Event Loop

Every screen froze at once

An online auction house runs its live evening sales on one Node.js process. About 6,000 bidders keep a WebSocket open to it. A bid travels up the socket, the server compares it with the lot's highest bid, and the new price is pushed to everyone watching. A busy night means 2,000 bids a second, at about 0.2 ms of server work each.

Then the complaints arrive: every few minutes the price freezes for most of a second, and then a burst of bids lands at once. Not on one bidder's screen but on all of them, at the same moment. Slow home connections would hit some bidders and not others; everyone stalling together points at something they all share, and here that turns out to be the server. To find it, start with how one process serves 6,000 connections at all.

6,000 sockets, one thread

The classic design gives each connection a thread that sleeps until its bidder sends something, so 6,000 threads would spend nearly all their time parked, each holding its own stack. This process works the other way round. Its sockets are non-blocking: reading from one that has no data returns at once instead of putting the thread to sleep. So the thread never waits on one connection. When it has nothing to do, it makes one call asking the operating system which of the 6,000 sockets have something ready, and sleeps until at least one does or the next timer is due. That asking-and-dispatching part has an article of its own, Reactor, and the whole cycle is the event loop:

1. The loop asks the operating system which sockets are ready, waiting at most until the next timer is due.
2. Each ready socket and each due timer becomes an event in the loop's queue: a bid arrived on socket 4127, lot 18's countdown expired.
3. The loop takes the next event and calls the handler registered for it.
4. The handler returns, and the loop takes the next event, until the queue is empty and it goes back to asking.

Diagram components: Event, Event, Queue, Event Loop, Handler, Handler.

Socket reads and timers become events in one queue. The loop takes them in turn and runs each handler to the end before taking the next.

Real implementations keep several queues, which affects the order of ready events but not the rule that they run one at a time. At 2,000 bids a second and 0.2 ms each, the bids alone keep this loop busy 400 ms of every second, and it sleeps through most of the rest.

No lock on the highest bid

The bid handler is about one line: if (bid.amount > lot.highest) { lot.highest = bid.amount; broadcast(lot) }. Bids of 510 and 505 arrive in the same millisecond on a lot at 500. With a thread per connection, both could read 500 before either writes, and the lower bid could overwrite the higher: the lost update that Mutex and semaphore opens with. On the loop that cannot happen, because a handler runs to its end before the next one starts, which is called run to completion. Whichever bid is taken first sets the price, the second is compared against it, and lot.highest needs no lock.

That holds for state only the loop's handlers touch. If some other thread can also write it, it is shared memory again, and it needs the locks the loop let you skip.

An await splits the handler

Then a new rule: every bid must be saved to the database before it counts. The handler becomes if (bid.amount > lot.highest) { await db.saveBid(bid); lot.highest = bid.amount }. The await hands the thread back to the loop until the save finishes, and the rest of the handler runs later, as another event; what it waits on is the subject of Future / Promise. Other events can now run between the two pieces. One possible order:

1. The 510 handler checks 510 > 500, starts its save and gives up the thread.
2. The 505 handler checks 505 > 500, still true, starts its save and gives up the thread.
3. 510's save finishes, and its second half sets the highest bid to 510.
4. 505's save finishes, and its second half sets the highest bid to 505.

The lower bid wins, on one thread. Run to completion protects each piece between two awaits, not the handler as you read it, so a check made before an await may be stale after it. Decide and apply in the same uninterrupted stretch: save the bid as an attempt, then, after the await, check that the lot is still open and bid.amount > lot.highest, and update only if both still hold.

800 milliseconds of one handler

Now the freeze. When a lot closes, the server renders the winner's invoice as a PDF with a synchronous library: about 800 ms of CPU inside one handler. Run to completion, the property that made the bid handler safe, means nothing else runs on the loop for those 800 ms: no other bids, no price broadcasts, no timers.

The numbers match the complaint. At 0.2 ms a bid, the loop can handle about 5,000 bids a second, and it receives 2,000. During the render about 1,600 bids arrive and wait, and one that arrived as the render started waits the full 800 ms. When the render ends, the backlog shrinks by about 3,000 a second (5,000 cleared minus 2,000 still arriving), so it takes roughly another half second to drain, and the price updates go out in a burst. Every bidder froze together because every bidder waits on the same thread.

A synchronous file read or a JSON.parse of tens of megabytes does the same. So the design rests on one rule: a handler must never block the loop, neither by waiting for I/O nor by computing for long. In queueing terms the loop is a single server, so its waits climb steeply as it nears fully busy; Servers, not utilization, decide the queue shows why.

Sending the work away

The fix is not a faster PDF library but taking the work off the thread. The close handler sends the job (lot number, winner, price) to a small pool of worker threads, worker_threads in Node.js, or to a separate process through a queue, and returns at once. When a worker finishes, its result reaches the loop as one more event, invoice ready for lot 18, and a short handler emails it. The loop now spends a fraction of a millisecond on the invoice.

Diagram components: Event Loop, Handler, Queue.

The close handler hands a copy of the job to a worker and returns. The finished invoice joins the queue like any other event.

Two conditions keep this honest. The worker gets a copy of what it needs and never writes the loop's state; once it writes lot.highest, the no-lock argument above is gone. And the pool's queue needs a bound, or a burst of closing lots only moves the backlog from the loop into that queue; Thread Pool covers sizing the pool and what a full queue should do.

The countdown that ran long

Each bid resets a 10-second countdown that closes the lot, and a countdown that expired during a render waited for the thread, so its lot closed up to about 800 ms late. A timer's delay means roughly not before, rather than exactly at, with no upper limit while the loop is busy. Code that cares about the deadline checks the clock where it decides instead of trusting the timer to be on time: the bid handler refuses a bid once Date.now() >= lot.closesAt.

Seven idle cores

One loop is one thread, so all the bid handling runs on one core of eight; the usual way to use the rest is eight processes, each with its own loop, as nginx runs one event loop per worker process. Then every bidder on lot 18 must reach the process that holds lot 18's state, or that state moves to a shared store, where run to completion no longer protects it. How a loop compares with locks or a single owner is the subject of Concurrency patterns.

What to draw in the Event Loop challenge

Redraw the auction server for the Event Loop challenge, within eight nodes: bid and countdown events feeding a Queue, the loop taking the next one, the handlers, and the invoice going out to a worker Handler and coming back as an event. The test of the drawing is whether someone reading it can see which handler must never block and why the Mutex / Lock node in the palette is not there.