Redis Transactions: MULTI/EXEC, WATCH and Lua Scripts

The idea: one thread makes a group of commands atomic

A Redis server runs every command on one main thread, one after another (see how Redis serves many connections with one thread). So a single command is always atomic: nobody sees it half done. A transaction stretches that to a group of commands. After MULTI, the server only queues the client's commands. EXEC then runs the whole queue in one go, and no other client's command can run in between.

The page has two clients, A and B, and a keyspace with alice = 100, bob = 50 and tag = "vip". Pick a client and a command and press Send, or run a demo. On the right you see what the server keeps for this: the flags and the mstate queue in each client struct, the watched_keys dict, the keyspace, the Lua engine and the order in which the main thread ran everything. Commands take no time on the page; only the endless script moves a clock, in 1-second steps.

MULTI, QUEUED, EXEC

• MULTI sets the CLIENT_MULTI flag on the client struct. It locks nothing.
• Every later command (except EXEC, DISCARD, MULTI, WATCH) is checked, appended to the client's mstate and answered +QUEUED. It has not run: other clients still see the old values.
• EXEC runs the queued commands one after another and answers one array with a reply per command. EXEC is one command for the event loop, so the main thread finishes the whole queue before it reads anything else.
• DISCARD drops the queue without running it, clears CLIENT_MULTI and unwatches every key (Demo 7: DISCARD drops the queue).

In Demo 1: MULTI/EXEC transfer, B waits, A moves 30 from alice to bob. B's GET alice between the queued commands still reads 100. When A's EXEC and another GET alice from B arrive in the same loop turn, B's command stays in its socket until both writes are done, and it reads 70. B never sees alice debited but bob not yet credited.

Two kinds of errors, and no rollback

	Queue-time error	Runtime error
Example	INCRBY bob (argument missing), unknown command, WATCH inside MULTI	INCR tag where tag is "vip"
Found	when the command is queued (processCommand)	when EXEC runs it
Effect	CLIENT_DIRTY_EXEC; EXEC answers -EXECABORT and runs nothing	that one entry of the array is an error; the other commands are applied
Demo	Demo 3: queue-time error → EXECABORT	Demo 2: runtime error, no rollback

Redis has no rollback. In Demo 2 the DECRBY before the failing INCR tag and the INCRBY after it both stay applied. The Redis docs give the reason: runtime errors are programming errors (a wrong type), which should be caught in development, and skipping rollback keeps Redis simple and fast. A transaction in Redis means "all or nothing runs, and nobody runs in between", not "all or nothing takes effect" as in SQL.

WATCH: optimistic check-and-set

A transaction cannot read a value and decide on it: queued commands return QUEUED, not values. To spend 30 only if alice has at least 30, the client reads before MULTI. Another client may change alice between that read and EXEC. WATCH catches this:

1. WATCH alice adds the client to the server's watched_keys dict (key → list of clients) and alice to the client's own list. Nothing is locked.
2. Any write to alice, by any client, calls touchWatchedKey, which sets CLIENT_DIRTY_CAS on every client watching it.
3. EXEC on a dirty client runs nothing and answers (nil). The client starts again from WATCH.
4. EXEC and DISCARD unwatch all keys; so does UNWATCH.

In Demo 4: WATCH, a write aborts, retry, A reads 100, then B spends 50. A's EXEC fails, A reads 50, checks again and its second EXEC succeeds: alice ends at 20, never below 0. This is optimistic locking: nobody waits, and a client pays with a retry only when there was a conflict.

Lua scripts and functions: read, decide and write in one command

EVAL script numkeys key… arg… runs a Lua script on the main thread as one command. Inside, redis.call() runs Redis commands and returns their results, so the script can read, branch and write. Other clients wait until it returns, exactly as for EXEC. The check-and-set needs no WATCH and no retry (Demo 5: Lua check-and-set (EVAL)):

local bal = tonumber(redis.call('GET', KEYS[1]))
if bal < tonumber(ARGV[1]) then return 0 end
redis.call('DECRBY', KEYS[1], ARGV[1])
redis.call('INCRBY', KEYS[2], ARGV[1])
return 1

In the demo, B's DECRBY alice 50 arrives together with A's EVAL. It runs only after the script: the script moves 30 (alice 70, bob 80), then B's write leaves alice at 20. SCRIPT LOAD + EVALSHA sha1 runs a cached script by its SHA1 digest. Since Redis 7, FUNCTION LOAD stores named Lua functions in a library that is kept in the RDB/AOF and replicated, and FCALL name numkeys key… arg… runs one. They run atomically in the same way. Pass every key in KEYS so Redis Cluster can check that they share a slot.

The price: a long script blocks everyone

A script or a long transaction holds the only command thread. Until busy-reply-threshold (5000 ms by default, named lua-time-limit before Redis 7) Redis treats it like one slow command and answers nobody. After the threshold it logs a warning and, between Lua instructions, starts serving events again, but only to say no: every command gets -BUSY Redis is busy running a script, except SCRIPT KILL, FUNCTION KILL and SHUTDOWN NOSAVE.

• SCRIPT KILL works only if the script has not written anything yet: Redis sets a flag and the Lua hook raises an error, so the script's caller gets -ERR Script killed by user with SCRIPT KILL (Demo 6: long script → BUSY, SCRIPT KILL).
• If the script has already written, killing it would break atomicity (half its writes would stay), so Redis answers -UNKILLABLE. The only way out is SHUTDOWN NOSAVE, which loses everything since the last save.

Which to use

Need	Use
Several writes that must not interleave with other clients	MULTI … EXEC
Read, decide, write; conflicts are rare	WATCH + MULTI/EXEC + retry
Read, decide, write; conflicts are common or logic is complex	A Lua script (EVAL / EVALSHA) or a function (FCALL)
Fewer round trips only, atomicity not needed	Pipelining (not atomic)
Mutual exclusion across many steps or many servers	A lock with SET key value NX PX (see distributed locks)

See also the Redis event loop for why one thread serves all clients, distributed locks for locks built on SET NX and released with a Lua script, Redis Cluster (all keys of a transaction or script must be in one hash slot) and Redis Sentinel, the other Redis internals pages (the dict, sorted sets, expiry and eviction, persistence, replication, encodings, streams), and MVCC and isolation levels for how SQL databases give transactions with rollback.

What the page leaves out

Network delay and real timing (commands take no time; the script clock is scaled to 1-second steps, and real Redis checks for events every 100,000 Lua instructions); UNWATCH and RESET; queuing an endless script inside MULTI (real Redis allows it); the -UNKILLABLE case and SHUTDOWN NOSAVE (described above, not animated); EVALSHA, SCRIPT LOAD, EVAL_RO, FUNCTION LOAD/FCALL and script flags such as no-writes; how EXEC and scripts are written to the AOF and replicas (wrapped in MULTI/EXEC, effects replication); keys expiring or being evicted while watched (they count as modified); WATCH in Redis Cluster; and blocking commands such as BLPOP, which do not block inside a transaction.

References

Redis documentation: transactions (MULTI, EXEC, DISCARD, WATCH, errors, no rollback)

Redis documentation: scripting with Lua (EVAL, EVALSHA, SCRIPT KILL, busy scripts)

Redis documentation: Redis functions (FUNCTION LOAD, FCALL)

Redis source: multi.c (queueMultiCommand, execCommand, watchForKey, touchWatchedKey)