Comparison rules
This lab publishes slices, not a winner. Breaking these rules produces a chart that looks decisive and is wrong.
Never
-
Never publish one “fastest queue” number. A library that wins 1P1C handoff of a 256-byte ticket can lose at 4 KiB, under 4P4C, or when the queue is bounded. Report by category × pattern × payload.
-
Never rank across languages. A C ring and a Python deque are not the same experiment. Runtimes, GCs, and clocks differ. Cross-language times are directional only.
-
Never put a broker on the same plot as an in-process queue. Redis, Kafka, RabbitMQ, NATS, ZeroMQ on localhost measure client + serialize + loopback + server. That is category N, a system bench. It does not belong next to
deque-lock. -
Never rank Thread against Async.
queue.Queueandasyncio.Queueanswer different questions. Use the dashboard Category filter. -
Never treat a scheduler as a queue. JavaScript
p-queuelimits concurrency. It does not hand a payload from a producer to a consumer. It lives under Other, not A. -
Never invent a multi-worker cell. If the library cannot run four producers and four consumers, skip 4P4C. Do not wrap a 1P1C structure in a mutex and log it as 4P4C.
Always
- Compare inside one language and one communication category.
- Say 1P1C / 4P4C in prose. The CSV still says
bytes/4p4c. - A failed fidelity check is an error, not a speed win.
- Warmup index 0 stays in the raw CSV; analysis drops it.
Where this is enforced
| Surface | What it does |
|---|---|
| Benchmark design | Tests and how to read a result |
| Queue categories | T / A default; P / S / D opt-in; N unpublished |
| Claims | What a published dashboard payload means |
| Dashboard Category filter | Hides the other communication model |
Experiments 3–4 (contention, backpressure) are designed, not shipped. Do not add them by widening experiment 1’s question.