Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Normalizer

The same fact arrives from several sources in several formats; a Normalizer recognises each message's format, sends it to the translator for that format, and lets one canonical format leave, so N sources and M consumers need at most N + M mappings instead of up to N × M.

Billing wants one shape

An electricity network operator bills energy suppliers for every 15 minutes of consumption on every meter it runs. Its billing team needs each reading in one shape: which meter, which 15 minutes, how much energy, and whether the number was measured or estimated. The operator's meters come from three vendors, and each vendor's collection system reports in its own way.

Three ways to say 312 watt-hours

Here is one reading, meter 4471 between 14:00 and 14:15 local time on 14 January, as each vendor sends it:

  • Vendor A sends XML: a reading element with the meter id, the end of the interval as 2026-01-14T14:15:00+01:00, the energy in kilowatt-hours and a flag for estimated values.
  • Vendor B sends a CSV line, 4471;2026-01-14 14:00;312: the start of the interval, in local time, with no offset.
  • Vendor C sends JSON, { "meter": "4471", "ts": 1768395600, "wh": 312 }: the start of the interval in seconds since 1970, in UTC.

Same meter, same quarter hour, same 312 watt-hours. One timestamp marks the end of the interval and two mark the start; one carries its offset, one is UTC, one is local and says nothing. Vendor B's local times hide one more problem: on the last Sunday of October the operator's clocks go back from 03:00 to 02:00, so a local time such as 02:15 happens twice that night, and the line alone does not say which.

First: which format is this?

All three collection systems deliver into one intake channel. A Normalizer takes that mixed stream and lets a single format out of it. It works in two parts. The first recognises which format each message is in and sends it to the converter for that format. That is a routing step whose rule reads the message's content, so it is a Content-Based Router with formats as its destinations. The second part is one translator per format.

Three meter formats, one reading shape

The format check and the three translators together are the Normalizer. A message whose format nobody recognises goes to its own channel, not to a best guess.

Telling XML from JSON from CSV is easy, and it is not enough. In spring, vendor B updates its collection system, and its lines now carry the voltage before the energy: 4471;2026-01-14 14:00;231;312. It is still a semicolon CSV line. A format check that only looks at syntax hands it to the old CSV translator; if that translator does not check the column count, it reads the third column as the energy and bills 231 watt-hours for a quarter hour that used 312. No step reports an error; the only symptom is a bill slightly too low.

So recognition has to name a format and its version: vendor B, layout 2, not merely CSV. The best signal is one the sender sets on purpose, such as a content type, a header naming the vendor and layout, or a channel per vendor; when the source offers none, check the shape strictly, with the column count, a header line or a version field. A message that matches no known format goes to an Invalid Message Channel for a person to inspect. Sending it to the nearest translator turns an unknown format into wrong numbers.

Then: one translator per format

Each translator converts one vendor's format into the operator's own, making the kind of mapping decisions that Message Translator walks through. What the Normalizer adds is a single target that every translator aims at.

The reading everyone agrees on

That target is a canonical data model: the operator's own shape for the fact, independent of any vendor. For a reading it could be { "meterId": "4471", "intervalStart": "2026-01-14T13:00:00Z", "minutes": 15, "energyWh": 312, "quality": "measured" }, always in UTC, where no hour happens twice, always the start of the interval, always whole watt-hours.

Billing is not the only consumer. Network planning forecasts load from the same readings, the customer portal shows them to households, and a reporting job sends totals to the regulator. Without a canonical model, every consumer that reads every vendor's format needs its own conversion for each one: three sources and four consumers make 3 × 4 = 12 mappings, each repeating the time-zone work. With one, each source needs a translator into the canonical model and each consumer at most one mapping out of it, if its own model differs: 3 + 4 = 7. A fourth meter vendor then costs one new translator and a new rule in the format check, and no consumer changes.

What the canonical model costs

Someone has to agree on it. Billing wants watt-hours per quarter hour; planning would like the voltage that vendor B now sends; the portal only needs daily totals. The model needs an owner who decides what goes in, and every change to it has to be checked against every translator and every consumer, so it is versioned like any shared format, as Schema evolution: event formats & compatibility describes.

The failure that is easy to miss is a model built from what every source has in common. Vendor C does not report whether a reading was estimated, so a model limited to the fields all three vendors share would drop the quality field, and billing would charge estimated quarter hours as if they had been measured. The model should hold what its consumers need. Then quality stays, and vendor C's translator sets it to unknown rather than measured, because claiming a measurement nobody confirmed is worse than admitting the gap. The opposite drift is just as real: a model that takes every field any vendor ever sent becomes a long list of optional fields that no consumer can rely on. Nor should a new source reshape it: a field added only because one new vendor happens to send it sets the canonical shape drifting with every source, so the model changes when a consumer needs something new.

Meter vendors on a Grid

The Message Transformation challenge asks for sources sending the same fact in different formats and a Normalizer turning every variant into one canonical format. Draw two clients as two meter vendors and connect both to the Normalizer, with each incoming edge naming the format it carries and the outgoing edge naming the canonical reading. On this Grid the Normalizer node stands for the format check and its translators together, so spend no nodes on a translator per format: within its ten nodes the Challenge also wants a Message Translator, a Content Enricher and a Content Filter. Then put your design through the fourth-vendor test: one more client with one more labelled edge into the Normalizer, and nothing after it changes.

Practice this