Benchmark result format
The comparison workbench can read a JSON file from your computer. The file is processed in memory in that tab; it is not uploaded or persisted. Keep private result files outside the public site sources.
Prepare a local file
Use an object containing schema_version: 1 and a results array. An empty file is valid:
{"schema_version": 1, "results": []}
Each result uses the fields below. The public register supplies the benchmark identifiers, protocol versions, and allowed metric identifiers and units. This page intentionally supplies no sample scores that could be mistaken for measurements.
| Field | Required content |
|---|---|
id |
Unique nonempty record identifier. |
benchmark_id |
Identifier from the register. |
protocol_version |
Registered protocol version; changes to methods require a new version. |
comparison_group |
Contributor-assigned identifier for a matched experiment. It is a compatibility assertion, not independent verification. |
reference_id |
Versioned reference or dataset subset identifier; use immutable checksums when available. |
context |
Physical parameters, initial/boundary conditions, duration, reference subset, and any aggregation or preprocessing choices. Keep identical across matched runs. |
implementation, revision |
Method name and immutable software/model revision. |
evidence |
author-run for your own experiment, published for a source-reported result, or reproduced for a result independently rerun by the contributor. These labels are self-declared. |
source_url |
HTTPS evidence URL; required for published. For local reproductions without a public URL, describe retained evidence in notes. Never include credentials in URLs. |
recorded_at |
Real calendar date in YYYY-MM-DD format. |
settings |
Object with nonempty hardware, precision, and timestep strings. Document solver options and other controls in context. |
sample_count |
Positive integer: number of independent trials represented. Explain what counts as independent in the uncertainty note. |
uncertainty_note |
Treatment of reference uncertainty, run variability, intervals, and dependence—or explicit reasons uncertainty was not estimated. |
metrics |
Nonempty object mapping registered metric IDs to finite numeric values, in the registered units. Omit unmeasured metrics; do not replace them with zero. |
notes |
Optional text explaining exclusions, retained evidence, or qualifications. |
The initial metrics are nonnegative error, time, or cost quantities. Negative values, unknown fields, unknown metrics, invalid dates, duplicate IDs, or incompatible protocol versions are rejected. Candidate benchmarks cannot accept scores until their data and protocols are finalized. Text fields must be nonempty and no longer than 2,000 characters. A local file may contain up to 500 records and must be no larger than 2 MiB.
Comparison rules
Baseline differences require exact agreement on benchmark_id, protocol_version, comparison_group, reference_id, context, and all three settings fields. This conservative rule prevents accidental comparison across different inputs or hardware. It does not establish that the declarations are true or that uncertainty permits a scientific conclusion.
Use separate comparison groups for different physical regimes, precision choices, dataset splits, or evaluation definitions. Do not rename a modified evaluation as the original benchmark. If a method cannot expose a metric, leave it absent and explain the limitation.
Publication boundary
Local imports never become part of the public register automatically. A public result requires a separate source review, exposure review, and site release. Anything included in the deployed HTML, JavaScript, or JSON is public even if a control hides it.
The comparison page disables the site's analytics integration and permits its result interface to fetch only same-origin resources. Other site pages retain the existing visit counter. Do not paste private results into URLs or link fragments.