Skip to content

Rust

Rust serialization is dominated by the serde data model: libraries implement Serialize/Deserialize once, then plug in format backends. A second tier (rkyv, FlatBuffers, Cap’n Proto) targets zero-copy access.

Runtime

What it is

Rust compiles to native machine code. There is no virtual machine and no garbage collector. Memory is released when values go out of scope. That rule is called ownership. Because of it, a Rust microsecond is a different kind of number from a C#, Java, or Python microsecond.

The language is described by The Rust Reference. Rust does not publish an ISO standard. A short overview is Rust.

This suite
Edition Rust 2021
Host toolchain rustc and cargo, usually installed with rustup
Prepare ./scripts/install-host-requirements.sh rust
Run cargo build --release through rust/scripts/run-benchmarks.sh
Memory Ownership. No garbage collector.

What this suite runs

The --release flag turns on optimizations. A cargo run without --release uses the Debug profile and is not comparable to the Dashboard. prost code is generated at build time by build.rs. Most rows go through serde, which is Rust’s shared serialize-and-deserialize trait. A few libraries (minicbor, rkyv, nanoserde, speedy, prost, dagr) use their own traits or generated code instead.

What changes the numbers

Building without --release is the error that changes the numbers the most, because Debug Rust is far slower than optimized Rust. After that, the useful comparison is still inside one family: JSON with JSON, not JSON with a zero-copy schema codec.

rkyv deserialize on the timed path builds owned values so the suite can check fidelity. A pure zero-copy access of the archived bytes would be faster and is not what this row measures. simd-json only accelerates parse; serialize still goes through serde_json.

Suite-specific gotchas

Stream mode is native only where the serializer table says so. Elsewhere the stream path is the bytes path written through a cursor.

arrow-ipc bytes are an IPC stream, not an Arrow file and not suite stream I/O. Schema selection and Parquet writer properties are outside the timer. Building the RecordBatch, and filling the SBE flyweight, is inside the timed serialize. table_project deserialize returns only f_float_0 (Arrow StreamReader::try_new with field index 0, Parquet ProjectionMask::columns for f_float_0). Those rows have no compliance decoder. A comma in the serializer filter selects case-insensitive exact names; without a comma the filter stays a substring.

These times cannot be ranked against a garbage-collected language as one contest.

Where to go next

The steps to install the toolchain and run the benchmark are in rust/README.md. The language overview is The Rust Book.

Benchmark runner

  • Directory: rust/ (repository root)
  • Output: monorepo logs/rust/YYYY-MM-DD-HHMMSS.csv (Language=rust, times in nanoseconds)
  • Runner: rust/scripts/run-benchmarks.sh {smoke|all-single|full|research} or cargo run --release -- <reps>
  • Registration: rust/src/serializers/mod.rs (family modules under serializers/)

Serializers

Serializer Category Crate Native path Stream Notes
arrow-ipc Columnar arrow 60.0.0 IPC stream RecordBatch built in serialize adapted Bytes API is the IPC stream, not the file format. table_project uses StreamReader::try_new(_, Some(vec![0])). No compliance decoder.
bincode Binary bincode 2 Serde; config in prepare adapted Config not rebuilt per call
bitcode Binary bitcode Serde adapted Bit-packed
bson Document bson Serde adapted Document DB interop
ciborium CBOR ciborium Serde native Reused write buffer
scuffle-amf0 AMF scuffle-amf0 Serde native AMF0. No object references
flexbuffers FlexBuffers flexbuffers Serde adapted Schemaless FB family
fory Binary fory ForyStruct derive; native mode adapted Reused output buffer; suite types only
ion-rs Binary ion-rs Serde experimental-serde adapted to_binary allocates; no public serde into_writer
gld-kiln Binary gld-kiln 0.2.0 Kiln Value built in prepare adapted One struct, or one list of that struct for N>1. Schema is not rebuilt. encode_into appends to the harness buffer.
minicbor CBOR minicbor Direct Encode/Decode on structs adapted No MessagePack envelope
nanoserde Binary nanoserde SerBin/DeBin adapted Zero-dep style binary
parquet Columnar parquet 60.0.0 RecordBatch in serialize; props in prepare adapted Snappy page codec. Encodings not overridden. table_project: ProjectionMask::columns(..., ["f_float_0"]). No compliance decoder.
parquet-uncompressed Columnar parquet 60.0.0 Same writer, compression off adapted Compression::UNCOMPRESSED only.
postcard Binary postcard Serde adapted no_std-friendly format
prost Schema prost + build Protobuf messages in prepare adapted De-facto Rust Protobuf (no Google-owned Rust runtime; prost-build + fixture/shared protos)
rkyv Zero-copy rkyv 0.8 Full Archive on structs adapted Timed deser materializes owned T for fidelity
rmp-serde MessagePack rmp-serde to_vec_named adapted Named maps
dagr-packed Schema generated dagr_benchmark_v2 Direct builder into a reused DagrBuilder; lazy reader → domain adapted Direct value structs built in prepare (like prost's messages); timed encode is write_into only; bytes copied out because Dagr writes back-to-front
dagr-regular Schema generated dagr_benchmark_v2 Generated arena built in prepare; timed arena serializer into a reused DagrBuilder; lazy reader → domain adapted regular (vtable) node layout; no direct builder exists for it
dagr-frozen Schema generated dagr_benchmark_v2 Generated arena built in prepare; timed arena serializer into a reused DagrBuilder; lazy reader → domain adapted frozen node layout; no direct builder exists for it
dagr-frozen-packed Schema generated dagr_benchmark_v2 Direct builder into a reused DagrBuilder; lazy reader → domain adapted frozen+packed node layout; same call path as dagr-packed
sbe Binary schema sbe-tool 1.40.2 Flyweight filled in serialize adapted Logged version is the generator, not crate 0.1.0. No nested_table. No compliance decoder.
serde_avro_fast Schema serde_avro_fast Serde one-pass datum; reused SerializerConfig native Prefer over official apache-avro (Value intermediate is multi-× slower than JSON on small records). Loads schemas/v2/avro/*.avsc. table_project full-decodes then keeps f_float_0.
serde_json JSON serde_json Serde Fixture native Baseline
simd-json JSON simd-json SIMD parse; ser via serde_json adapted Honest split responsibilities
sonic-rs JSON sonic-rs Serde-compatible SIMD JSON adapted Hot-path JSON
speedy Binary speedy Writable/Readable adapted Fast binary framework
toon-format Text toon-format Serde encode_default / decode_default adapted TOON text. Size is UTF-8 bytes
ctoon Text ctoon dumps / loads adapted C99 core. Size is UTF-8 bytes
tru Text tru encode / try_decode adapted Size is UTF-8 bytes

Specifics

Why each library exists, what problem it was written to solve, and how. Names link to the source repository (or the stdlib / in-tree path this suite times).

arrow-ipc

Apache Arrow was created so analytic engines could share columnar batches without copying each one into a private layout. The problem was a convert-at-every-boundary tax. Arrow IPC is the stream format for those batches. This row times arrow-rs 60.0.0 IPC stream writer and reader inside the bytes API, not the Arrow file format. The schema is chosen in prepare. The RecordBatch is built inside serialize. table_project passes field index 0 to StreamReader::try_new. No compliance decoder.

bincode

bincode is a compact binary format for serde. It was created so Rust programs could pack serde types without a public schema. It is a Rust-centric encoding, not a cross-language standard.

bitcode

bitcode is a bit-packed binary format for serde. It was written to squeeze serialized Rust values smaller than typical byte-aligned packers.

bson

BSON (Binary JSON) was created for MongoDB so documents could be stored and traversed without a text parse. Official language drivers implement that spec. This row times that library's serialize/deserialize path.

ciborium

ciborium is a CBOR implementation for serde (Enarx). CBOR is the IETF binary JSON-like format. The crate exists to give Rust a serde CBOR backend.

scuffle-amf0

scuffle-amf0 is a pure-Rust AMF0 encoder and decoder from the Scuffle RTMP stack. AMF0 is the command encoding inside RTMP. The timed path is to_bytes / from_slice and to_writer / from_reader through Serde. The crate does not write AMF0 object references, so a graph with a shared node or a cycle does not round-trip.

flexbuffers

FlexBuffers is the schemaless cousin of FlatBuffers. It was created so you can have a FlatBuffers-family binary without compiling a schema. The same Google repository implements it.

fory

Apache Fory (formerly Fury) was created for high-performance, cross-language serialization. The problem was that JVM-centric binary codecs and slow portable formats left a gap. Fory registers types and serializes with a compact binary protocol. This row uses the fory crate with ForyStruct-derived serializers in native mode and a reused output buffer.

ion-rs

Amazon Ion was created at Amazon as a rich, self-describing superset of JSON (text and binary) for internal services. The problem was JSON's limited types. ion-rs is the official Rust implementation. This row uses the crate's experimental serde feature (to_binary / from_ion).

gld-kiln

gld-kiln (leo-gan) is a positional little-endian codec. The timed path reuses one writer. There are no field tags on the wire. This row times encode and decode of the suite structs. On the tensor data set this row runs every type. tensor_view borrows the float32 payload and does not copy the matrix.

minicbor

minicbor is a compact, often no_std CBOR codec with its own Encode/Decode traits. The problem was serde overhead and no_std needs. It implements RFC 8949 directly on structs.

nanoserde

nanoserde is a tiny, dependency-light serializer for Rust (SerBin/DeBin). The problem was serde's compile-time and dependency weight in constrained crates. nanoserde generates a minimal binary path.

parquet

Apache Parquet was created as a columnar file for scans that touch a few fields of many rows. The problem was row files that made every reader parse every column. This row times arrow-rs 60 ArrowWriter. arrow-rs DEFAULT_COMPRESSION is UNCOMPRESSED, so this row sets Compression::SNAPPY. Encodings stay at the builder default. table_project uses ProjectionMask::columns on f_float_0. No compliance decoder.

parquet-uncompressed

This is the same arrow-rs Parquet writer as parquet, with Compression::UNCOMPRESSED. Encodings stay at the builder default. The name is the override. No compliance decoder.

postcard

postcard is a compact, no_std-friendly binary format for serde (James Munns). It was created for embedded Rust where alloc and self-describing formats are too heavy.

prost

prost is the de-facto Protocol Buffers implementation for Rust (tokio-rs). The problem was that Google does not ship an official Rust runtime. prost-build generates Rust from .proto and times encode/decode on those messages.

rkyv

rkyv is a zero-copy deserialization framework for Rust. The problem was that even fast binary codecs still allocate an owned value on decode. rkyv archives data so it can be accessed in place; this suite still materializes owned T for fidelity.

rmp-serde

rmp-serde is MessagePack for serde (msgpack-rust). MessagePack exists as compact binary JSON. This crate maps serde types to named MessagePack maps.

dagr-packed

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the packed node layout (tagged, evolvable). It does not support the graph data type.

dagr-regular

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the regular node layout (vtable, evolvable). It also times the graph data type.

dagr-frozen

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the frozen node layout (positional, no evolution). It also times the graph data type.

dagr-frozen-packed

Dagr ("Data Graph") is a schema-driven binary format that can store shared nodes and cycles, built on an arena model. One Python DSL schema generates the code for every target language (dagr build), so there is no runtime library: the suite commits the generated code from schemas/v2/dagr/schema.py. The five suite types are emitted in all four node layouts, one row each. The graph data type is emitted only for regular and frozen, because a packed reference cannot store the person ring. On suite types, prepare builds the native value and the timed call writes every field (direct builder or arena serializer) and reads them back. This row uses the frozen+packed node layout (positional and inline, no evolution). It does not support the graph data type.

sbe

SBE (Simple Binary Encoding) was created for word-aligned financial messages: a fixed block, then repeating groups, then variable-length data. The problem was tag/length codecs that touch every field on the hot path. This row fills the sbe-tool 1.40.2 Rust flyweight inside serialize. The logged version is the generator, not the vendored crate 0.1.0. It encodes table, table_project, and signal. nested_table is not an SBE body. The signal wire order is fixed fields, then the legs group, then symbol and venue. No compliance decoder.

serde_avro_fast

serde_avro_fast is a high-performance Avro datum codec for serde. Avro exists for compact, schema-driven records. This crate avoids the official apache-avro Value intermediate, which is multi-× slower on small records.

serde_json

serde_json is the standard JSON backend for Rust's serde. Serde was created so Rust types could implement Serialize/Deserialize once and plug in many formats. serde_json is the JSON instance of that idea.

simd-json

simd-json is a SIMD JSON parser for Rust (the simdjson port). The problem was parse speed vs serde_json. This row uses SIMD for parse; serialize still goes through serde_json.

sonic-rs

sonic-rs is a SIMD-oriented JSON library for Rust, in the same family as ByteDance sonic. The problem was JSON cost on the hot path. It offers a serde-compatible encode/decode.

speedy

speedy is a fast binary framework for Rust with its own Writable/Readable traits. It was written to beat generic serde binaries on the encode/decode hot path.

toon-format

toon-format is the official Rust codec for TOON, with Serde. The timed path is encode_default and decode_default of the suite value. The stream path writes that UTF-8 buffer. The Size column is the byte length, not a token count.

ctoon

ctoon is the Rust binding over the C99 TOON core. The crate has no Serde layer. The timed path builds that value tree and calls dumps and loads. The Size column is the UTF-8 byte length.

tru

tru is the published crate for Dicklesworthstone's TOON codec. The library name is toon. The timed path is encode and try_decode of the JSON value. The Size column is the UTF-8 byte length.

Call-path contract (same idea as Python)

prepare(fixture)                 # untimed: config, kind, prost convert
for rep:
  serialize_bytes / stream       # timed
  deserialize_bytes / stream     # timed
  fidelity(expected, actual)     # untimed

Caveats

These notes explain odd-looking correctness or speed edges on Rust only:

  • prost maps ISO timestamps through millisecond integers; the benchmark runner allows date-string drift on types that carry timestamps (message, event, document, telemetry).
  • rkyv timed deserialize builds owned values for comparison; a pure zero-copy access path would be faster.
  • simd-json serialize still goes through serde_json (the crate focuses on parse speed).
  • flatbuffers / capnp: not registered yet (codegen weight); flexbuffers partially covers FB-family schemaless use.
  • Stream mode is native only where noted; others are adapted bytes+cursor.

Also: rust/README.md. Families.

How to rank

Compare libraries inside one language and one standard. JSON with JSON is an implementation comparison. JSON with Protocol Buffers or SBE is a standard comparison. Rank in bytes mode only (the in-memory buffer API — not “payload size in bytes”). Stream mode is left out of this ranking.

The table is a family map of this roster. It is not the ranking key.

Family Members
JSON serde_json, simd-json, sonic-rs
Rust-centric binary bincode, postcard, bitcode, nanoserde, speedy
Schema / zero-copy flexbuffers, rkyv, prost, sbe
Columnar arrow-ipc, parquet, parquet-uncompressed
Schemaless binary (interop) bson, ciborium, minicbor, rmp-serde, scuffle-amf0

Numbers

Measured numbers for this language live on the Dashboard (pre-filtered). Claim level is L1 (one machine, one session) — see Claims and replication.

Design choices

  1. Prepare outside the loop — configs, buffers, prost messages, kind tags.
  2. Optimal APIs — crate-recommended encode/decode; no pretty-print.
  3. Dual mode — bytes and stream with StreamMode metadata.
  4. Concrete types for non-Serde stacks (minicbor, rkyv, nanoserde, speedy, prost).