Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Content Filter

Each message on its way to one consumer loses the fields that consumer does not need or must not see. The rule is safest when it names the fields to keep, and the filter protects nothing that handled the message before it.

"Can't we just send them the record?"

A car-rental company has signed with a loyalty partner: drivers in the partner's programme earn points for every completed rental. At the kickoff, someone from the partner asks the obvious question. The rentals system already publishes a RentalCompleted record each time a car comes back, as JSON, and the partner's API accepts JSON. Why build anything? Subscribe them and send them the record.

Open one record and the answer is in it. Next to what the partner needs sit fields it must never receive:

  • loyaltyNumber, rentalId, returnedAt, distanceKm and vehicle.class, the five fields the partner uses to award points;
  • driver.licenceNumber, checked at the counter;
  • payment.cardToken, which the payment provider issued in place of the card number;
  • damageNotes, free text typed by staff at the return bay;
  • driver.homeAddress, and about fifteen more fields on the car, the branch and the charges.

The data-sharing agreement names five fields. The licence number and the home address are personal data the partner has no reason to hold. The card token is not the card number, but it still identifies one person's card, and the partner has no use for it. The damage notes are worse than they look: typed in a hurry, they hold whatever was said at the bay, from "scratch on rear bumper" to the name of the other driver in an accident.

So the answer is no. A second record shaped for the partner would work, but it would make the rentals team maintain a contract for one consumer; the company keeps one record for billing, fleet and the partner, and shapes the partner's copy on the way out.

Every rental goes through, smaller

A Content Filter sits on the path to one consumer and removes fields from each message before passing it on. Every message goes through it, and each one leaves smaller. The partner receives a message shaped for its context: the five fields, and nothing else.

Full record in, five fields out

The filter runs on the rental company's side, as the last step before the partner. The broker and its logs still hold the full record.

The filter also changes the shape. In the record, the vehicle class sits inside vehicle, next to the plate number and the last GPS position, and the charges are an array of line items. The partner reads one flat message with five values, so the filter lifts the class to the top level and leaves the rest of vehicle behind. Simplifying a deep structure is part of the filter's job; renaming fields into the partner's vocabulary, or kilometres into miles, belongs to a Message Translator.

Some rentals should not reach the partner at all, such as one by a driver who is not in the programme. Dropping a whole message is a Message Filter's job; a Content Filter lets every message through and only decides which fields travel with it. Adding fields a message lacks, the opposite move, is a Content Enricher.

Keep these, or drop those

The filter's rule can be written two ways. A denylist names the fields to drop: licence number, card token, damage notes, home address and the rest. An allowlist names the fields to keep: the five in the agreement. On the day they are written, both produce the same message.

They part the first time the rentals team changes its record. In spring it adds additionalDrivers, an array with each extra driver's name, date of birth and licence number, because the counter now registers them. Nobody on the rentals team thinks of the partner, and there is no reason they would. The denylist does not mention the new field, so it lets it through, and the partner starts receiving licence numbers of people who never joined its programme. The allowlist does not mention it either, so it drops it, and the partner receives exactly what it received the day before.

A denylist fails open: each field the producer adds reaches the consumer until someone notices and writes a rule, and a leak tends to be noticed late. An allowlist fails closed: a new field the partner would want stays out until someone adds it, and that arrives as a request rather than an incident. Across a trust boundary, write an allowlist. A denylist is reasonable when the consumer is trusted and the filter only trims bulk, such as dropping a large attachment from messages bound for an internal dashboard.

An allowlist can still leak. Keeping vehicle instead of vehicle.class keeps everything under it, including whatever is added there later: the plate today, a GPS track next year. Name the fields you keep at the deepest level, not the objects that contain them. And treat a kept field that has gone missing, or turned from a single value into an object, as an error. If the rentals team renames distanceKm, a filter that forwards a rental with no distance leaves the partner to award zero points or reject it, while one that stops and reports the missing field gets the rule fixed.

What the filter does not hide

The filter protects what comes after it, and only that. Everything the record passed through before the filter saw it whole:

  • the rentals channel, and the broker that stores its messages for as long as it keeps them;
  • the filter's own logs, if it writes the messages it receives there to help with debugging;
  • any copy taken on the way, such as a Wire Tap feeding a monitoring tool;
  • a message moved to a retry or dead-letter channel before the filter, which keeps the full record.

Those copies are why the filter belongs at the boundary you don't trust, on your side of it: the last step before anything outside sees the payload. A filter the partner runs behind its own API, however carefully written, starts after the full record has crossed, and once a payload leaves your system you cannot un-send the sensitive fields. Deciding which fields count as sensitive in the first place, across every system rather than in one filter's rule, is a job for Data governance: catalogs, lineage & classification.

Five fields in Message Transformation

In the Message Transformation Challenge the Content Filter is the step that strips the fields the target must not see. Draw it as the last node before the handler that stands for the target, and let its edges carry the argument: the edge into it says it receives every message, and the edge out of it lists the fields that survive, written as the allowlist rather than the word "filtered". Those two labels also meet the brief's last requirement, that the Content Filter trims fields from every message while a Message Filter would drop whole messages, and they cost no extra node in a Grid capped at 10 that also needs a Normalizer, a Message Translator and a Content Enricher.

Practice this