Submission Workflow¶
Eight steps, in order, from an empty directory to a submission in the review queue. Each page links to the next, and each ends with a Verify section so you can check the step worked before moving on.
flowchart LR
R[1 Register] --> I[2 Install] --> P[3 Plan curve] --> X[4 Run points]
X --> D[5 Author disclosures] --> V[6 Validate] --> S[7 Submit] --> A[8 Review]
V -.->|errors| D
| Step | What it produces |
|---|---|
| Before you begin | A decision to proceed, or to stop before it costs anything |
| 1. Register and get a token | A PRISM API key in mlc_… format |
| 2. Install the tools | Reference client, submission CLI, checker |
| 3. Plan your Pareto curve | The concurrency levels you'll run |
| 4. Run the measurement points | One run folder per point, plus an accuracy run |
| 5. Author the disclosure files | system_desc.json and point.yaml per point |
| 6. Validate locally | A clean submission-checker report |
| 7. Submit | An uploaded bundle and a review PR |
| 8. After you submit | A finalized, published result |
Start step 1 before anything else. It's the only step that waits on someone outside your organisation, and no turnaround time is published.
Two steps that catch people out
Step 3. Choosing C_max badly means re-running points, and each point is a 600- or
1,200-second run.
Step 5. Two required files are hand-authored, and no tool generates them for you.
These rules are a working draft
MLPerf Endpoints v1.0 rules are on an active development branch. Sections marked
[TENTATIVE], [WIP], [WG Open Item] or [WG Decision Required] upstream can still
change. Before committing hardware time to a submission, read
Open questions and WIP rules.