Submission states¶
The status values a submission carries, and what moves it between them.
States¶
| Status | Set by | Meaning |
|---|---|---|
REVIEW_PENDING |
submissions create |
Submission created and bundle uploaded; awaiting review |
WITHDRAWN |
submissions withdraw |
Retracted; bundle deleted |
FINALIZED |
Review workflow (server-side) | Review complete; submission accepted |
PUBLISHED |
Review workflow (server-side) | Results published in the MLPerf leaderboard |
Only the first two are set by the CLI. The others are set server-side by the review workflow.
stateDiagram-v2
[*] --> REVIEW_PENDING: submissions create
REVIEW_PENDING --> WITHDRAWN: submissions withdraw<br/>or 10 business days without<br/>responding to an objection
REVIEW_PENDING --> FINALIZED: all objections resolved<br/>or retracted
FINALIZED --> PUBLISHED: next cohort
WITHDRAWN --> [*]
PUBLISHED --> [*]This list may be incomplete
These four are the states documented by the CLI. Whether the review workflow reports additional states — rejected at Week 0, in dispute, invalidated, embargoed — is not documented in any source. Tracked as C5 in Open questions.
Review phases¶
Status is coarse; the review phase is what actually governs your obligations.
| Phase | Window | Status during |
|---|---|---|
| Automated compliance | Week 0 | REVIEW_PENDING |
| Peer review | Weeks 1–3 | REVIEW_PENDING |
| Objection resolution | Weeks 4–6 | REVIEW_PENDING |
| Dispute resolution | From Week 6, ~5 weeks | REVIEW_PENDING — does not finalize until the dispute concludes |
| Finalized | — | FINALIZED |
| Published | Next cohort | PUBLISHED |
See How submission works and After you submit.
Tags a published result can carry¶
Distinct from status. A result can carry:
| Tag | Meaning |
|---|---|
| peer review pending | Provisionally published before review completed. Removed at finalization |
| Preview — Available by [date] | Preview publication status, with its 180-day deadline |
| Invalidated | Removed from the active results page, retained in the historical archive with a description of the error |
| Withdrawn | Post-finalization withdrawal — removed from active results, retained in the archive |
Pre-finalization withdrawal leaves no record
Withdrawing before finalization removes the submission entirely from both the active results page and the historical archive. No record of the provisional publication is retained. Withdrawing after finalization retains it in the archive marked Withdrawn.
Where the publication status categories fit¶
Available, Preview and RDI are publication status, not submission state. A submission has
one of each. See Publication status.
Result IDs¶
Assigned by MLCommons at publication, not chosen by you and not part of the submitted bundle.
<major>.<minor>.<cohort>.<model_id>.<dataset_id>.<entry>
Example: 1.0.0.deepseek-r1.mmlu-pro.7 — the seventh published DeepSeek-R1 result on MMLU-Pro under
the v1.0 rules.
Result IDs are stable and never reused. A result that is superseded, withdrawn or invalidated keeps its ID in the historical record, and the replacement gets a new entry number.
Two identifiers, two purposes
The submission ID is an opaque hash generated by the pipeline so tooling can track a bundle through upload, review and amendment. The result ID is human-readable and identifies one published Pareto curve — one system, one benchmark model, one dataset.
Historical record¶
MLCommons keeps every version of every Pareto curve, aligned to cohorts. The active results page shows the latest finalized version; older versions remain accessible on request, and superseded points are labelled with the cohort in which they were replaced.
Last verified against: mlcommons/endpoints_policies@v1.0_rules_dev (a7ec3cc) and
mlcommons/endpoints-submission-cli@main (f48ca84), 2026-09-19.