Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Concurrency patterns

Concurrent work either shares memory behind locks, hands data over to a single owner, or runs on one thread that never waits. Where each choice moves the difficulty, and which pattern answers which problem.

Who is allowed to touch room 7?

A chat server keeps, in memory, which users are in which room. Thousands of connections are open, and any of them can join a room, leave one, or post to it. The question that decides the design is: when two connections change room 7 at the same moment, how do they avoid stepping on each other's change?

There are three broad answers, each of which moves the difficulty somewhere different:

  • Share the memory and guard it. Threads read and write the room list directly, and a lock lets only one of them change room 7 at a time.
  • Hand the data over. No connection touches the room list. Each one sends a message, "user 81 joins room 7", to the single owner of that state, which applies the messages one at a time.
  • Run everything on one thread that never waits. One thread serves every connection, so no two handlers run at the same instant, and its handlers must never sit waiting for the network or a disk.

Sharing: direct access, and nothing checks the rules

The first answer is where most code starts, because it looks like ordinary programming: the room list is a map, and a handler calls rooms.get(7).add(user). The trouble arrives with the second thread. Adding a member reads the set, changes it and writes it back, and two threads doing that at once can lose one of the joins, or, with a set or map that is not built for concurrent use, damage its internal structure. The fix is a mutex around every access to the room. Mutex and semaphore covers how one works, how a forgotten release leaves every later caller waiting, and how two locks taken in opposite orders deadlock.

What this answer costs is discipline that nothing enforces. One handler that reads the map without the lock, or a new feature that takes the room lock and a user lock in the opposite order from everyone else, brings the bug back, and it usually shows up only under load. In return, threads work on the data in place, with no copying and no messages.

Handing over: one owner per piece of state

The second answer removes the sharing instead of guarding it. Room 7's member list belongs to exactly one owner. A connection that wants to join sends the owner a message and moves on; the owner takes messages from its inbox one at a time and is the only code that ever reads or writes the list. Your code needs no lock, because the one shared thing is the inbox, a queue somebody has already made safe for concurrent use.

One owner for room 7

Connections never touch the member list. They send messages to the room's owner, which applies them one at a time.

This holds only if what is handed over is not changed afterwards: a message carrying a reference to an object the sender keeps editing is shared memory again, without the lock. Send a copy, or an immutable value.

The simplest form is a fixed-size buffer between threads that produce work and threads that consume it, which is Producer-Consumer. Taken all the way, every piece of state gets an owner like room 7's, with its own inbox, and owners talk to each other only through messages; that design is called the actor model, and Actor Model follows one message through it.

Handing over changes the shape of concurrency problems rather than removing them. An inbox with no limit grows without bound when its owner falls behind. Two owners that each send the other a request and wait for the reply before doing anything else are stuck as surely as two threads holding two locks. And moving a user from room 7 to room 9 is now two messages to two owners, handled at different moments, so for an instant the user can be in both rooms or in neither. The gain is that these problems sit in the message flow, where they are easier to trace, instead of in the one line that forgot a lock.

One thread that never waits

The third answer avoids sharing by having only one thread. An event loop takes the next event from a queue (data arrived on connection 412, a timer fired) and runs its handler to completion before taking the next one. Handlers never run at the same instant, so room 7's list needs no lock: the handler that adds user 81 finishes before any other handler starts.

One thread can serve thousands of connections because it never waits on any single one of them; Event Loop covers how, and what that costs. A slow operation, such as saving the room to a database, is started and not waited for: the handler gets back a placeholder for the result, whose attached code runs later as another event on the loop; Future / Promise covers how that placeholder works.

The price is one rule, never block: a handler that computes for 200 ms or calls a synchronous file or database API makes every connection wait those 200 ms. And going without locks is safe only between awaits, the points where a handler hands the thread to others while it waits: a check made before one (room 7 still has space) may no longer hold after it.

Threads are a budget, whichever answer you pick

Sharing and handing over both run on threads, and a thread per connection gets expensive as the count grows, since each reserves its own stack. A fixed set of workers taking tasks from a shared queue is Thread Pool. It caps how many things run at once but says nothing about sharing: its tasks still need locks, or owners, for state they have in common.

Which problem, which pattern

  • Threads change the same data and must not interleave, or at most N may use a resource at once: Mutex and semaphore.
  • Work is produced at a different rate from the rate it is consumed: Producer-Consumer.
  • Many short tasks, and a cap on how many run at once: Thread Pool.
  • Many independent pieces of state (rooms, game sessions, devices) and no shared memory at all: Actor Model.
  • Very many connections that mostly wait on the network: Event Loop.
  • A result that will exist later, to pass around, chain or wait for: Future / Promise.
  • The same handover between services rather than threads, through a broker: Message queues & async workers.
TipReal systems mix the three answers; what matters is the order of decisions. Decide who owns each piece of state, then pick the mechanism, and before adding a lock, ask whether the state could have a single owner instead. Handing data over means fewer locks and fewer lock-ordering deadlocks, though not none.

Practising each answer

No Challenge grades the choice between these answers on its own; each pattern's Challenge is described in the article that owns it. The Actor Model challenge practises the second answer, room 7's owner multiplied into actors with mailboxes and a supervisor, and the article Actor Model says what to draw there; the one-thread answer's Challenge is in Event Loop.

Practice this