Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Wire Tap

A fixed step in a channel that copies every message to a second channel and passes the original on unchanged, so you can see what went down the channel without changing what is delivered.

Did the new price ever go out?

"Store 214 says the olive oil label shows €6.49 and the till charges €7.29. Did the new price ever go out?" Store operations forwarded the question to the pricing team, and nobody there could answer it from what they had.

The supermarket chain's pricing service decides price changes centrally and publishes one PriceChanged message per product per store on the price-changes channel: about 40,000 on a normal night. A label service takes each message and sends the new price to the electronic shelf labels through the base station in that store. The tills read prices from another system, which already had €7.29. Pricing's log said the change was published. The label service's log said nothing about product 50211 at store 214, which could mean it never received the message, or that it received it and logged nothing.

The team needed to see what actually went down price-changes without changing either end's code, and the quick ways in each change something. Adding logging means redeploying the very service under suspicion. Pointing a second reader at the channel is worse. price-changes is a queue, and a second consumer on a queue does not get its own copy of each message: it takes turns with the label service, and every message it reads is a price that never reaches a shelf.

A copy on the side

The answer was a step inserted into the channel itself. Pricing still publishes to price-changes. The new step reads from it, forwards each message unchanged to a new channel the label service was configured to read, with no change to its code, and puts a copy on price-changes.tap, where an inspection tool stores what it receives and lets someone search it by store and product. The label service gets the same messages with the same content, and does not know the step is there. That step is a Wire Tap, and it makes no decisions: every message, the same second destination. A router that works out, message by message, which receivers get a copy is a Recipient List.

Tap a channel for observation

The label service receives the same messages it did before. The copy goes to a channel of its own, and nothing that reads it is on the delivery path.

NoteThe name comes from the network tap, a device that copies the traffic on a network link to a monitoring port; the messaging cousin keeps its rule: observation must not change the observed flow.

A tap sees only what passes after it is in place, so it could not show the night before. Pricing republished store 214's changes from that evening, and the copy was there within seconds: product 50211, store 214, €7.29, while the label went on showing €6.49. That proved the republished message went down price-changes, not what happened to the original the night before, and it moved the search past the channel, to the label service and the store's base station, which had been offline since a power cut. The tap shows one channel; following one price change through every service it touched is Distributed tracing.

The observer that held up the shelves

The first version of the tap wrote each copy to the inspection tool's database before forwarding the original. On the Sunday before a promotion, with pricing sending several times its usual volume, the inspection database slowed to 40 milliseconds a write. Each original now waited for its copy, written one at a time, so the tap forwarded at most 25 messages a second, and when the doors opened some stores still showed last week's prices. The step added to watch the flow had become something the flow waited on. A step that sits in the path on purpose, able to hold messages back, is a Detour; a tap should not turn into one.

So the copy is handed off instead of written in line: the tap forwards the original and puts the copy on the tap channel, which buffers it while the inspection tool reads at its own pace. The question has only moved: when the tool is slow or down and the tap channel fills up, what does the tap do? It has two choices:

  • Drop copies. The tap discards the copies it cannot place, counts them, and keeps forwarding. The labels never wait, and the inspection record has holes.
  • Block. The tap waits for room before forwarding the next original. No copy is dropped for lack of room, and a stopped inspection tool now stops the labels.

For a tap that answers questions like store 214's, dropping is right, as long as the count of dropped copies is visible, so nobody reads a gap as a message that was never sent. On RabbitMQ, a length limit on the tap's queue does the dropping in the broker: by default, a full queue discards its oldest messages to make room for new ones. A tap queue with no limit and nobody reading it is worse than either choice. It grows until the broker runs short of memory or disk, and RabbitMQ then blocks every publishing connection, pricing's included, until the alarm clears. Because a tap nobody reads still costs capacity, the team switches its debugging tap on only during an investigation, a job for a Control Bus.

A copy that must not go missing

Two months later the legal team asked for what sounded like the same thing: a record of every price change sent to the shelves, kept for two years, to check a disputed shelf price against. It is a different tap. A debugging tap may lose copies. An audit tap may not, because the missing copy is the one somebody will ask about.

Losing nothing has a price, and the tap is where it is paid. The tap channel has to store its copies durably, so that a copy survives a broker restart and waits until the audit store confirms it has written it, and the tap has to see both the forwarded original and the copy confirmed as stored by their channels before it acknowledges the incoming message, or a crash between the two loses one of them. A crash just after that, before the acknowledgement, sends the message through again, so the audit record has to tolerate a duplicate copy. The audit store has to keep up with the busiest hour, not the average one, with room for a backlog while it is down. And once that room runs out, the only way left to lose nothing is to stop forwarding until there is space, which makes an audit outage a labels outage. The chain accepted that, sized the audit store for promotion nights and alerts on the tap channel's depth; the debugging tap stayed separate, still allowed to drop.

Watch outA copy carries everything the original carries: each PriceChanged holds the user id of the manager who approved it, so the tap channel and whatever stores the copies need access control as strict as the main channel's and a retention limit chosen for their own purpose, or the copies stored without that field.

When the channel already makes copies

All of this assumed a queue, where reading a message takes it away. On a Publish / Subscribe channel, which already delivers each message to every subscriber, a tap is one more subscriber, such as another Kafka consumer group on the topic or another queue bound to the RabbitMQ exchange, and the drop-or-block question moves to that subscription: a RabbitMQ queue still needs its limit, and a Kafka group that falls behind holds up nobody but loses the copies the topic's retention deletes before it reads them.

Tapping in the Challenge

The Messaging Endpoints & Broker Challenge asks for a Wire Tap that copies traffic to an inspection channel without disturbing delivery. Put the tap on the channel between the broker and the receiving Message Endpoint, give its copy edge a Message Channel of its own named as the inspection channel, and let the original edge carry on to the endpoint it reached before. Write on the copy edge whether the tap drops or blocks when that channel is full. The rest of the picture is the Challenge's own list: a Messaging Gateway in front of the application, Message Endpoints on the channels, a Channel Adapter for the external system, and the broker routing between them. Then cover the copy edge and the channel behind it: the delivery path should read exactly as it did.

Practice this