2. Install the tools¶

Produces: a working inference-endpoint and endpoints-submission-cli, including its checker.

Before you begin

Completed 1. Get access and an API key
Python 3.12+ available for the reference client
Python 3.10+ available for the submission CLI

What you'll do¶

Install the reference client (mlcommons/endpoints) — runs the benchmark
Install the submission tools (mlcommons/endpoints-submission-cli) — registers runs, assembles and validates the bundle
Confirm all three CLIs respond

There are three distinct CLIs and it is worth fixing which is which now:

CLI	Package	Does
`inference-endpoint`	`mlcommons/endpoints`	Drives load at your endpoint and writes a run folder
`endpoints-submission-cli`	`endpoints-submission-cli`	Registers runs, assembles and uploads a submission
`endpoints-submission-cli check-submission`	same package	Validates a bundle against the automated compliance rules

Steps¶

1. Install the reference client¶

Do not modify the source

For Client-on-Prem submissions the reference client must be used without source-code modification, built from a commit the review committee can access. Anything you want to change goes in the YAML config. Review can detect modifications (§2.1.1 of the rules).

uv (recommended)pip + venv

git clone https://github.com/mlcommons/endpoints.git
cd endpoints
uv sync

Commands then run as uv run inference-endpoint ….

git clone https://github.com/mlcommons/endpoints.git
cd endpoints
python3.12 -m venv venv && source venv/bin/activate
pip install .

Note this does not use uv.lock, so dependency versions may differ from the lockfile. After activating the venv, commands run without the uv run prefix.

Record the commit SHA you built from — you will need it for disclosure:

git -C endpoints rev-parse HEAD

2. Install the submission tools¶

pipFrom sourceuv

pip install endpoints-submission-cli

git clone https://github.com/mlcommons/endpoints-submission-cli.git
cd endpoints-submission-cli
pip install -e ".[dev]"

uv sync --extra dev

Both endpoints-submission-cli and submission-checker come from this one package.

Optionally, install the results visualizer to preview your curves locally before you submit:

pip install mlperf-viz

See Results visualizer.

3. Install `gh`¶

The gh CLI is required for creating, updating and withdrawing submissions. Install it and authenticate with gh auth login.

4. Smoke-test the client against a local server¶

Before pointing anything at real hardware, confirm the client works end to end against the bundled echo server:

pip install endpoints-submission-cli

The compliance checker comes with it, as endpoints-submission-cli check-submission. The CLI's README still describes a separate submission-checker command, but the package doesn't install one (B16 in Open questions).

Verify¶

Both must respond:

uv run inference-endpoint --version
endpoints-submission-cli --version

And the region calculator should produce sensible boundaries — this is the tool you will use in the next step:

python -c "from submission_checker.cli import main; main()" regions --max-concurrency 1024 --min-concurrency 16

Next¶

→ 3. Plan your Pareto curve

Problems? See Troubleshooting.