Skip to content

Serialization 201

In this track you will learn how serialization mechanisms work. You will not only learn the names of formats. You will learn why they behave differently.

Jump
Before this Serialization 101 vocabulary
After this 301 production judgment · Dashboard
Lab depth 401 wire formats

These short essays sit between Serialization 101 and its three lenses on one side, and the later courses on the other. 301 covers production judgment. 401 is for people who implement codecs. When you care about measured numbers, you will also want the suite categories page and the Dashboard.

Theory alone does not decide what you should ship. Use these pages to build clear mental models of the mechanisms. Then check those models against real libraries. When you must choose under several production constraints at once, continue into the advanced courses as they become available.

How to use this track

  1. Skim Serialization 101 so the basic definitions and trade-off axes feel familiar.
  2. Optionally read one lens that matches your work: Historical, Data science, or Engineering.
  3. Work through the articles below when you need a clearer how or why for a mechanism.
  4. Open Serialization categories and the Dashboard for numbers measured on this benchmark runner.

Honesty rules (same as Serialization 101). There are no universal winners. Implementation quality often matters more than the brand name of a format. The shape of the payload can change costs a great deal. Flat, nested, sparse, and dense data all behave differently. Compare within one paradigm and one language when you can. Numbers that appear in prose are only illustrations. The Dashboard owns measured numbers for this benchmark runner.

Suggested order (MVP path)

The table below gives a suggested order for a first pass through Serialization 201. Each row states what you should be able to explain after reading that article.

Step Article You should be able to…
1 Memory layout Explain why dumping raw process memory is not a portable interchange format
2 Encode/decode cost Name the real cost centers (parsing structure, converting numbers, allocating, copying). Do not say only that “JSON is slow.”
3 Self-describing vs schema Say whether field identity lives in the payload itself or in a shared contract outside the message
4 Schema evolution Plan additive changes that keep older readers and writers working during a rollout
5 Dynamic vs IDL binary Choose a MessagePack/CBOR-class encoding versus a Protocol Buffers–class encoding for a given workload. IDL means interface description language. An IDL is a formal shared description of messages and field types.
6 Zero-copy Explain what “no deserialize” usually means in marketing language—and what that design still costs
7 Compression vs format Separate gzip-style compression on the wire from density that comes from the format itself

By module

Representation

These articles explain how values become bytes, and where the time and memory go when you encode or decode them.

Contracts & change

These articles explain where the meaning of a field lives, and how systems stay compatible when that meaning changes over time.

Families in practice

These articles compare common binary families and specialized layout designs you will meet in real systems.

Systems concerns

This article separates two mechanisms that people often conflate when they talk about “small payloads.”

Lab notebooks (Python / Colab)

Several articles have an accompanying notebook you can run locally or in Colab. The notebooks let you experiment with the same ideas in code.

Notebook Article
Encode/decode cost Encode/decode cost
Self-describing vs schema Self-describing vs schema
Schema evolution Schema evolution
Dynamic vs IDL binary Dynamic vs IDL binary
Compression vs format Compression vs format

Install notes live in the notebooks README.

Where to go next