Method
This section explains how this project measures serializers, not the theory of formats themselves. On the site it lives under the Method tab. Live interactive results live under the Dashboard tab.
If Learn / Serialization 101 answers “what is serialization?”, Method answers “how do we time it fairly, and what do the numbers mean?”
You do not need to be a performance engineer to read these pages. They are written for introductory computer science students and for anyone who wants a clear picture of the suite before diving into tables and plots.
Learning goals
After reading this section you should be able to:
- Say what a benchmark is measuring here (serialize and deserialize time, size, and correctness).
- Find the right page for layout, modes, statistics, metrics, test data, or published results.
- Compare serializers within one language (and ideally one category) instead of treating absolute times as a global ranking.
- Open a language Overview (what libraries we measure) and its Results (the measured numbers) under the Languages tab.
- Open the Dashboard for interactive charts and comparisons of the same analysis story.
For the ideas behind formats, start with Serialization 101 under Learn.
The big picture in one paragraph
Each programming language has a small program called a benchmark runner. The benchmark runner builds sample data, runs each serializer many times, checks that the data comes back correctly, and writes a spreadsheet-like file (CSV). A shared analysis program then cleans those runs, computes summaries, and produces the tables and charts you see on the site. Everyone shares the same data shapes and the same analysis rules so comparisons stay fair.
benchmark runner (C#, Python, Rust, …)
│ timed serialize + deserialize
▼
logs/<language>/*.csv
│
▼
analyze-benchmarks
│
├──► docs/<language>/results.md (tables)
└──► docs/analysis/plots/… (charts)Contribute
| Task | Guide |
|---|---|
| Add one library to an existing language | Adding a serializer |
| Add a new language tree | Adding a language |
Shared terms (quick)
| Say this | Not this | Means |
|---|---|---|
| data type | “fixture” | One of message, document, telemetry, strings, event |
| benchmark runner | “harness” | Per-language program that times serializers and writes the CSV |
| batch size N | — | How many instances in one serialize call (1 or 100) |
| category / family | — | JSON, schemaless binary, schema-driven, language-native |
| I/O mode (bytes / stream) | — | How the library was called (buffer vs stream), not payload size |
| run mode (smoke / full / …) | — | How heavy the experiment is (repetitions / intent) |
Full mode guide: Modes. Data-type glossary: Test data — vocabulary.
How to read this section
Suggested order for a first visit
- Architecture — lab design and what is timed
- Modes + Test data — what the columns mean
- Methodology — warmup, filters, uncertainty
- Metrics when a column name is opaque
- Claims and replication before publishing a blog or paper claim
| Page | What you will learn | Start here if you want… |
|---|---|---|
| Architecture | How the repository is organized and how timing works | The measurement design |
| Modes | I/O modes (bytes/stream) and run modes (smoke…research) | Why Results have two mode columns; which run preset to use |
| Categories | Four families of serializers (JSON, binary, schema-driven, native) | Fair “apples to apples” groups |
| Test data | The five sample data types and how sizes are chosen | What we serialize |
| Methodology | Warmup, outliers, confidence intervals, effect sizes, exploratory ranks | How CSVs become published numbers |
| Metrics | Names and meanings of every reported measurement | “What does this column mean?” |
| Claims and replication | L1 / L2 / L3: what you may claim from one run vs many | Honest generalization language |
| Adding a serializer | Drop-in checklist for one library | Author path |
| Adding a language | Checklist to plug in a new language benchmark runner | Extending the suite |
| Results summary | Links to each language’s numbers and how to regenerate them | The published snapshots |
Theory tracks: 101 · 201 · 301 · 401. Live charts: Dashboard.
Languages in this suite
Each language has two documentation pages:
- Overview — which serializers are registered, how they are grouped, and any caveats. This is the hand-written source of truth for what we measure.
- Results — generated tables and latency charts from a real run on a particular machine. Numbers can change when you re-run elsewhere.
| Language | Serializers (registered) | Overview | Results |
|---|---|---|---|
| C# | 38 | Overview | Results |
| Python | 16 | Overview | Results |
| Rust | 16 | Overview | Results |
| C | 20 | Overview | Results |
| JavaScript | 20 † | Overview | Results |
| Go | 19 | Overview | Results |
| Java | 18 | Overview | Results |
| C++ | 27+ | Overview | Results |
| Swift | 14 | Overview | Results |
† In JavaScript, simdjson is optional (it needs a native addon). If that addon fails to build, the rest of the run still continues without it.
Log language ids (the Language column in CSVs) use short names such as csharp, python, rust, c, javascript, go, java, cpp, and swift. Documentation folders sometimes differ (for example docs/c-sharp/ for C#).
To regenerate tables and plots after a local run, see Results summary — regenerating snapshots.
One rule for fair comparison
Prefer:
Same language + same category + same data type + same I/O mode
Absolute times across languages (for example “Python vs C++”) mix runtimes, garbage collectors, and allocators. Those numbers can still be informative as a rough direction, but they are not a precise ranking. Details live under methodology limitations.