Skip to content

Python

Python's dynamic nature makes serialization uniquely challenging. While it excels at developer productivity, the runtime overhead of object instantiation and the Global Interpreter Lock (GIL) can severely bottleneck high-throughput data processing pipelines.

Runtime

What it is

This suite measures CPython, the usual C implementation of the Python interpreter. Other implementations such as PyPy are not used here. CPython compiles source to bytecode and then runs that bytecode in a virtual machine. Almost every value is a heap object with a type pointer and a reference count. That design makes Python easy to write and expensive to decode into, because even a small integer is a full object.

The language is specified by the Python Language Reference. CPython is the reference implementation named there. Other implementations are listed in the reference introduction.

The Global Interpreter Lock (GIL) is a lock that lets only one thread run Python bytecode at a time inside a process. If a decode takes 10 ms of Python bytecode, the whole process is blocked for those 10 ms, unless the library releases the GIL while it works in C, C++, or Rust.

This suite
Interpreter CPython 3.12 or newer. Not PyPy or Jython.
Host toolchain uv (uv sync)
Prepare ./scripts/install-host-requirements.sh python
Run python/scripts/run-benchmarks.sh
Memory Reference counting plus a cyclic garbage collector. The GIL is present.

What this suite runs

python/pyproject.toml requires Python 3.12 or newer. The uv tool creates a local virtual environment and installs the packages listed in the lock file. The benchmark runner is ordinary CPython on the command line. It is not a web server such as FastAPI or Flask.

What changes the numbers

Libraries with a C or Rust core, such as orjson, msgspec, and protobuf, spend most of the timed path outside the interpreter. They can release the GIL while they parse bytes. Pure-Python paths, such as dill serialize and the official FlatBuffers Builder, stay inside the virtual machine and look much slower.

Deserializing into a dict is usually cheaper than building a class or a Pydantic model, because constructing Python objects is itself expensive. The tracemalloc tool under-counts allocations that happen inside C or Rust extensions.

Suite-specific gotchas

The official FlatBuffers Python package builds with a pure-Python Builder. The C++ and Rust FlatBuffers libraries are a different speed class. This suite measures the Python binding. See caveats.

pickle, cloudpickle, and dill work only in Python. Loading untrusted input with them can run arbitrary code.

h5py, netCDF4, and adios2 time the array data set (grid and grid_window in config/library/array.yaml). They do not run on the five suite types or the tensor data set. h5py uses the HDF5 core driver. netCDF4 writes a NetCDF-4 file. adios2 writes a serial BP5 directory. The window read is a hyperslab, a variable slice, or an ADIOS selection. arrow-ipc and flatbuffers run every type in the tensor data set. protobuf does not.

These times cannot be ranked against another language.

Where to go next

The steps to install the toolchain and run the benchmark are in python/README.md. The language overview is The Python interpreter. For the GIL and garbage collection in latency, see Latency tails and GC.

Benchmark runner

  • Directory: python/ (repository root)
  • Output: monorepo logs/python/YYYY-MM-DD-HHMMSS.csv (Language=python, times in nanoseconds)
  • Runner: python/scripts/run-benchmarks.sh (or project docs for modes)
  • Registration: python/src/benchmark/runner.py
  • Modes: bytes and stream

Serializers

Log name Category Package Native input (prepare_data) Stream mode Notes
amazon-ion Binary amazon-ion dict native Official Ion binary simpleion dump/load
arrow-ipc Columnar pyarrow rows → Table inside serialize adapted IPC stream in the bytes API. table_project reads f_float_0 only
avro Schema fastavro record dict adapted Compact schemaless size; dict/union path slower than protobuf C++
dagr-packed Schema generated (dagr build, runtime shipped in-tree) dataclass → generated @dataclass node adapted Pure-Python reflective runtime (eager to_bytes / restore); ~35–390× slower than protobuf (upb), size ≈ protobuf. See caveat
dagr-regular Schema generated (dagr build, runtime shipped in-tree) dataclass → generated @dataclass node adapted Same runtime and call path as dagr-packed; regular (vtable) nodes — evolvable, larger than packed. See layouts
dagr-frozen Schema generated (dagr build, runtime shipped in-tree) dataclass → generated @dataclass node adapted frozen (positional) nodes — no schema evolution. See layouts
dagr-frozen-packed Schema generated (dagr build, runtime shipped in-tree) dataclass → generated @dataclass node adapted frozen+packed (positional, self-sizing) — smallest Dagr row, no schema evolution. See layouts
cbor2 Binary cbor2 dict native IETF CBOR (RFC 8949)
py3amf-amf0 Binary Py3AMF dict adapted AMF0. Every number is a double
py3amf-amf3 Binary Py3AMF dict adapted AMF3. Small integers stay integers
cloudpickle Native cloudpickle dataclass native Extended pickle; same security caveats
dill Native dill dataclass native Graphs/dynamics; ser much slower than pickle (pure-Python dispatch)
flatbuffers Schema flatbuffers dataclass → Builder adapted Python Builder ser is slow; deser is zero-copy GetRootAs view
fory Binary pyfory dataclass adapted Native mode; types registered before timing
gld-kiln Binary gld-kiln dict adapted Kiln 0.2.0 positional encode / decode. A batch is one list
json JSON stdlib dict adapted Baseline text JSON
mashumaro JSON mashumaro dataclass adapted ORJSONEncoder/Decoder
msgpack Binary msgpack dict native Reference MessagePack
msgspec JSON msgspec Struct native (encode_into) Typed array-like Structs
msgspec-msgpack Binary msgspec Struct native Same Struct path, MessagePack
orc Columnar pyarrow rows → Table inside serialize adapted write_table default is uncompressed on pyarrow
orc-uncompressed Columnar pyarrow rows → Table inside serialize adapted Same writer, compression="uncompressed"
orjson JSON orjson dict adapted Rust core; conversion untimed
parquet Columnar pyarrow rows → Table inside serialize adapted Default page compression is Snappy. table_project reads one column
parquet-uncompressed Columnar pyarrow rows → Table inside serialize adapted Same writer, compression="NONE"
pickle Native stdlib dataclass native Cycles supported; unsafe untrusted
protobuf Schema protobuf Message adapted From suite protobuf schemas under schemas/v2/ / language generated modules
pydantic JSON pydantic BaseModel adapted Validation-oriented API models
rapidjson JSON python-rapidjson dict adapted C++ RapidJSON bindings
serpyco-rs JSON serpyco-rs + orjson dataclass adapted dump/load + orjson wire
yaml Text pyyaml dict text_on_stream safe_dump / safe_load
toon-format Text toon-format dict adapted Official TOON. UTF-8 bytes, not a token count
toons Text toons dict adapted TOON with a Rust core. UTF-8 bytes
ctoon Text ctoon dict adapted C99 core. Size is UTF-8 bytes

Specifics

Why each library exists, what problem it was written to solve, and how. Names link to the source repository (or the stdlib / in-tree path this suite times).

amazon-ion

Amazon Ion was created at Amazon as a rich, self-describing superset of JSON (text and binary) for internal services. The problem was JSON's limited types. ion-python is the official Python implementation.

arrow-ipc

Apache Arrow was created so analytic engines could share columnar batches without copying each one into a private layout. The problem was a convert-at-every-boundary tax. Arrow IPC is the stream format for those batches. This row times pyarrow's IPC stream writer and reader inside the suite's bytes mode, not the Arrow file format. Schema objects are selected in prepare. Row-to-column conversion stays inside serialize. table_project deserialize reads f_float_0 only. On the tensor data set this row runs every type. The matrix, the prompt, and the shard are IPC tensor messages. request is those prompt tensors plus one record batch in the same buffer. tensor_view returns a buffer inside the stream.

avro

fastavro is a Cython Avro reader/writer for Python. Avro was created so data systems could store compact records with the schema out of band. fastavro solves the CPython performance problem of the older pure-Python Avro stack.

dagr-packed

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the packed node layout (tagged, evolvable). It does not support the graph data type.

dagr-regular

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the regular node layout (vtable, evolvable). It also times the graph data type.

dagr-frozen

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the frozen node layout (positional, no evolution). It also times the graph data type.

dagr-frozen-packed

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the frozen+packed node layout (positional and inline, no evolution). It does not support the graph data type.

cbor2

cbor2 is a Python implementation of IETF CBOR (RFC 8949). CBOR was created as a binary JSON-like format for constrained devices and IETF protocols. cbor2 solves the Python side with a widely used encoder/decoder.

py3amf-amf0

Py3AMF is a Python 3 fork of PyAMF. It encodes and decodes Adobe Action Message Format. This row uses AMF0, the form RTMP still uses for commands. Every number is written as a double. The timed path calls pyamf.encode and pyamf.decode on a dict.

py3amf-amf3

The same library, with the AMF3 encoding. AMF3 can store a small integer more tightly than AMF0, and it can share repeated strings and objects. The timed path is the same encode and decode calls.

cloudpickle

cloudpickle extends pickle so cluster and serverless runtimes can ship functions and dynamic objects, not just importable classes. The problem was that pickle cannot serialize many constructs that job schedulers need. cloudpickle solves that by capturing more of the Python object model — with the same security caveats.

dill

dill extends pickle further for scientific Python: graphs, lambdas, and interpreter state. The problem was that pickle and even cloudpickle still failed on objects researchers actually use. dill solves that with a broader, mostly pure-Python save path.

flatbuffers

FlatBuffers was created at Google so games and clients could access serialized data without an unpack step. The problem was that protobuf-style decode allocated a full object graph. FlatBuffers solves it with a schema and a binary layout that can be traversed in place. On the tensor data set the Builder runs every type: tensor, tensor_view, prompt, shard, and request. tensor_view reads the float vector in place.

fory

Apache Fory (formerly Fury) was created for high-performance, cross-language serialization. The problem was that JVM-centric binary codecs and slow portable formats left a gap. Fory registers types and serializes with a compact binary protocol. This row uses pyfory in native mode on registered dataclasses.

gld-kiln

gld-kiln (leo-gan) is a positional little-endian codec. The timed path reuses one writer. There are no field tags on the wire. This row times encode and decode of the suite structs. On the tensor data set this row runs every type. tensor_view borrows the float32 payload and does not copy the matrix.

json

The CPython json module is the language's standard JSON encoder/decoder (RFC 8259). It was added so Python programs had a stdlib way to speak the web's data format. The implementation is a C-accelerated codec over Python objects; this suite times that path as the baseline.

mashumaro

mashumaro was written to serialize Python dataclasses to JSON and other formats with code generation, not runtime reflection on every call. The problem was slow dataclass codecs. This row times the ORJSON encoder/decoder path.

msgpack

msgpack-python is the official MessagePack implementation for Python. MessagePack was created to be as compact as a binary format and as simple as JSON. The package exposes pack/unpack (and stream APIs) on Python objects.

msgspec

msgspec was created as a high-performance serialization library for Python with typed structures, not just dicts. The problem was that fast JSON libraries still paid for untyped Python objects, and validation libraries were slow. msgspec solves it with array-like Structs and a compiled encode/decode path (JSON and MessagePack).

msgspec-msgpack

msgspec was created as a high-performance serialization library for Python with typed structures, not just dicts. The problem was that fast JSON libraries still paid for untyped Python objects, and validation libraries were slow. msgspec solves it with array-like Structs and a compiled encode/decode path (JSON and MessagePack). This row times the MessagePack encoder/decoder on the same Struct types.

orc

Apache ORC was created at Hive as a columnar stripe file with indexes and compression. This row times pyarrow.orc.write_table and read_table with no compression argument. pyarrow's default is uncompressed. Apache ORC's own C++ and Java writers default to Zlib; this binding does not. table_project passes columns=['f_float_0'] on read.

orc-uncompressed

This is the same pyarrow ORC writer as orc, with compression='uncompressed'. On pyarrow 25 that codec is also the default, so the bytes match orc. The name stays so the allow-list matches languages whose ORC writer defaults to Zlib.

orjson

orjson was written to give CPython a JSON codec that is both fast and correct. The problem was that stdlib json is slow on large payloads, while many speed-focused alternatives cut corners on Unicode, strictness, or types. orjson solves that with a Rust extension that implements RFC 8259 on dataclasses and common types.

parquet

Apache Parquet was created as a columnar file for scans that touch a few fields of many rows. The problem was row files that made every reader parse every column. This row times pyarrow.parquet.write_table and read_table with pyarrow's default page compression, Snappy. Encodings stay at the library default. table_project passes columns=['f_float_0'] on read.

parquet-uncompressed

This is the same pyarrow Parquet writer as parquet, with compression='NONE'. Encodings stay at the library default. The name is the override. Use this size when the question is layout rather than Snappy.

pickle

pickle is CPython's native object serializer. It exists so Python processes can snapshot almost any object graph. That power is also the problem: loading untrusted pickle can run arbitrary code. This row times the stdlib C implementation.

protobuf

Protocol Buffers were created at Google so many languages could share a compact, evolving binary contract without hand-written parsers. The problem was ad-hoc binary formats and verbose XML. Protobuf solves it with an IDL, generated code, and a documented tag/length wire format.

pydantic

Pydantic was created to validate and serialize Python data for APIs (later FastAPI). The problem was that ad-hoc dicts and hand-written validators were error-prone. Pydantic solves it with typed models and JSON dump/load (model_dump_json / TypeAdapter).

rapidjson

python-rapidjson wraps Tencent RapidJSON so CPython can use that C++ parser/generator. RapidJSON was written for speed with a SAX and DOM API. The Python binding exposes dumps/loads on dicts.

serpyco-rs

serpyco-rs is a fast dataclass JSON serializer (Rust core) for Python. The problem was that dataclass-to-JSON in pure Python is slow. It solves that by compiling the dump/load path and using orjson on the wire.

yaml

PyYAML is the long-standing YAML 1.1 processor for Python. YAML was created as a human-friendly superset of JSON for config and documents. PyYAML implements that model with dump/load.

toon-format

toon-format is the official Python codec for TOON, a text encoding of the JSON data model made so prompts can carry the same values in fewer tokens. The timed path is encode and decode. The Size column is the UTF-8 byte length. A token count is a separate measure and is not timed.

toons

toons is a Python TOON codec whose core is Rust. It speaks the same JSON data model through loads and dumps. The timed path is that text. The Size column is the UTF-8 byte length, not a token count.

ctoon

ctoon is a C99 TOON core. This row is the Python binding. The timed path is dumps and loads. The Size column is the UTF-8 byte length.

Call-path contract (fair timing)

Timed methods measure codec only on library-native values:

  1. prepare(name, type) — encoders, schemas, buffers (untimed)
  2. prepare_data(obj, …) — dataclass → dict / Struct / Message / Model (untimed)
  3. serialize_* / deserialize_* — encode/decode only (timed)

FlatBuffers is the exception where Builder construction is the serialize API (no separate Message type). Stream mode is native when the library has a real file/stream API; otherwise adapted (bytes then write / read then bytes).

For table, table_project, nested_table, and signal only, rule 8 of Timing honesty overrides the usual “convert in prepare” rule. The pyarrow rows keep the Arrow schema as a module constant and select it in prepare. serialize_bytes builds the pyarrow.Table with Table.from_pylist, then calls pyarrow.ipc.new_stream (arrow-ipc), pyarrow.parquet.write_table (parquet, parquet-uncompressed), or pyarrow.orc.write_table (orc, orc-uncompressed). table_project deserialize does not materialize the other columns: IPC uses IpcReadOptions.included_fields, and Parquet and ORC pass columns=["f_float_0"]. Row peers (orjson, protobuf, flatbuffers, avro) still convert to their native value in prepare_data. Their timed table_project path is a full decode plus project_f_float_0. These rows have no compliance decoder. The columnar configs time bytes only.

Caveats

Why avro size is great but ops/s lag protobuf

  • Size is real: schemaless Avro omits field names (schema out-of-band).
  • Speed is mostly library/runtime: timed path is only schemaless_writer/reader on a pre-built dict + cached parse_schema. fastavro (Cython) still loses to protobuf message SerializeToString by ~20–30× on nested types because of Python-dict field walks and null-unions.
  • Measurement: the benchmark runner no longer runs tracemalloc during timed ser/des (it was inflating alloc-heavy codecs ~2–3×).

Why flatbuffers and dill ops/s look low

  • flatbuffers (serialize): the official Python package builds with a pure-Python Builder. Expect ~100×+ slower ser than protobuf on the same POCO. C++/Rust FlatBuffers are a different performance class; this suite measures the Python binding.
  • flatbuffers (deserialize): timed path is zero-copy GetRootAs + a thin view. Full field materialization is not forced inside the timer (FlatBuffers' model: pay on field access).
  • dill (serialize): for ordinary importable dataclasses the wire size matches pickle, but dill's pure-Python save path (module/type discovery) is ~15–20× slower than C pickle. That is inherent; byref/recurse do not close the gap on these data types. Prefer pickle when you do not need dill's dynamic-object features.

Why dagr ops/s look low

  • Pure-Python runtime: the Dagr Python target is a reflective runtime (typed @dataclass layer over a generic node model); there is no C extension. serialize is <graph>.to_bytes(node) on nodes built in prepare_data, deserialize is <graph>.restore(bytes) back to the generated dataclasses (eager — Python has no lazy reader). Peer-smoke medians: ~20–75× slower than protobuf (upb C core) on single messages and ~40–125× on n=100 cells (protobuf's packed repeated double is close to a memcpy in C). Size is within a few % of protobuf, and smaller on strings.
  • Hot-path audit: cProfile shows no per-call imports or schema loading. The generated module caches the schema's per-node facts once (lookup, field plans, elidability). Its dataclasses feed the serializer directly and are built directly by restore, with no intermediate node graph, so what remains is the per-field Python work.
  • Layouts: all four dagr* rows run the same reflective runtime, so their ops/s differ only by the per-layout work in the runtime, not by a different code path.
  • Framing: N>1 cells encode N independent Dagr buffers in a u32 count + u32 len frame (same frame as the Rust dagr-packed row, used by all four dagr* rows), so n=100 carries 404 bytes of framing that protobuf's Batch* wrapper does not.

Other caveats

  • tracemalloc under-counts C/Rust extension allocations.
  • Fidelity is semantic, not strict type identity (dict vs dataclass, enum vs int, datetime ms truncation).

Benchmark runner: python/README.md. Families.

Numbers

Measured numbers for this language live on the Dashboard (pre-filtered). Claim level is L1 (one machine, one session) — see Claims and replication.