The config file¶

The config file serves as the input to the mlperf-sysinfo tool. The fields inside it determines which fields get collected, execution path of the tool , and what shape the output file takes(As determined by the particular benchmark rules).

Which profile¶

Profile	For
`endpoints`	MLPerf Endpoints submissions
`inference`	MLPerf Inference submissions
`training`	MLPerf Training submissions

profile: defaults to endpoints.

Getting one¶

init writes a commented starter config for the profile you name, containing only the fields that profile reads:

mlperf-sysinfo init endpoints     # writes sysinfo.yaml
mlperf-sysinfo check -c sysinfo.yaml

check names every field still missing or still holding starter text, and reaches every machine the config mentions, before you spend a capture finding out.

Every CHANGEME below is a field check will stop on.

What init endpoints writes

# MLPerf system description -- endpoints profile
#
#   mlperf-sysinfo check   -c sysinfo.yaml     # validate + reach everything
#   mlperf-sysinfo capture -c sysinfo.yaml     # collect and write the file
#
# Anything the tool can detect for itself -- CPU, memory, accelerators, node
# counts, framework version, parallelism and batch size -- is deliberately
# absent below. Only what cannot be detected belongs in this file.

profile: endpoints

output:
  dir: results/sysinfo

system:
  name: CHANGEME                 # e.g. H100x8_vLLM
  shortened_name: CHANGEME       # at most 20 characters, e.g. H100x8
  availability: available        # available | preview | rdi
  accelerator: cuda              # cuda | rocm | xpu | none
  cooling: air                   # air | liquid | passive

nodes:
  include_local: false           # is this machine part of the system under test?
  ssh:
    - user@node1
    # - user@node2:2222
  ssh_key_preconfigured: false

  # Optional. Only needed for disaggregated setups, where nodes serve
  # different functions. 'match' is compared case-insensitively against the
  # detected accelerator model name.
  # groups:
  #   prefill:
  #     - { match: NVIDIA H100, count: 2 }
  #   decode:
  #     - { match: NVIDIA H100, count: 5 }

serving:
  # The endpoint under test, and also probed for the framework name/version.
  # Rules 8.2 allow a description instead of a URL, for a hosted endpoint with
  # no public address -- e.g. "Managed endpoint, us-east-1, no public URL".
  url: http://node1:8000
  node: user@node1               # where the server process runs
  log: /tmp/serving.log          # server stdout/stderr must be redirected here
  framework: auto                # auto | vllm | sglang | trtllm

submission:
  division: standardized         # standardized | serviced | rdi
  # container_link: https://...  # container the submission ran in
  notes:
    hardware: ""                 # becomes hw_notes on every node type
    software: ""                 # becomes sw_notes on every node type
    # other_hardware: ""

# How the stack was configured for this run. The parallelism degrees and batch
# size are read from serving.log, so they are not listed here -- these three
# cannot be detected from anything on the machine.
run:
  # node_config: "prefill: 2x H100; decode: 6x H100"   # defaults to a summary of nodes.groups
  # config_summary_notes: ""     # anything the parallelism fields do not capture
  # link_config: https://github.com/myorg/submission/tree/main/configs

What init inference writes

# MLPerf system description -- inference profile
#
#   mlperf-sysinfo check   -c sysinfo.yaml     # validate + reach everything
#   mlperf-sysinfo capture -c sysinfo.yaml     # collect and write the file
#
# Writes the flat field set the MLPerf Inference submission checker expects.
# Anything detectable is absent below on purpose -- only what cannot be
# detected belongs in this file.

profile: inference

output:
  dir: results/sysinfo

system:
  name: CHANGEME                 # e.g. 8xH100_TRT
  category: datacenter           # datacenter | edge
  availability: available        # available | preview | rdi
  accelerator: cuda              # cuda | rocm | xpu | none
  cooling: air
  # type_detail: ""

nodes:
  include_local: true            # single machine: describe the one running this
  ssh: []                        # add user@host entries for a multi-node system
  ssh_key_preconfigured: false

submission:
  submitter: CHANGEME
  contact: CHANGEME@example.com
  division: closed               # closed | open
  notes:
    hardware: ""
    software: ""

What init training writes

# MLPerf system description -- training profile
#
#   mlperf-sysinfo check   -c sysinfo.yaml     # validate + reach everything
#   mlperf-sysinfo capture -c sysinfo.yaml     # collect and write the file
#
# Writes the flat field set mlperf_logging/system_desc_checker validates.
# Anything detectable -- CPU, memory, accelerators, node count, OS, software
# stack -- is deliberately absent below. Only what cannot be detected belongs
# in this file.
#
# The output is named after system.name, because a training submission stores
# it as <submitter>/systems/<system_name>.json.

profile: training

output:
  dir: results/sysinfo

system:
  name: CHANGEME                 # e.g. dgx-h100-n8 -- also the output filename

  # Training accepts exactly these four, and nothing else. Plain "available"
  # is not one of them: training splits it into on-premise and cloud.
  availability: Available on-premise
  #   Available on-premise
  #   Available cloud
  #   Preview
  #   Research, Development, or Internal (RDI)

  accelerator: cuda              # cuda | rocm | xpu | none
  cooling: air                   # air | liquid | passive

  # How the nodes are wired to each other. No probe can see past the local
  # NIC. A single-node system should say so rather than leave it blank.
  networking_topology: CHANGEME  # e.g. "rail-optimized fat tree, 8x400G per node"

nodes:
  include_local: true            # single machine: describe the one running this
  ssh: []                        # add user@host entries for a multi-node system
  ssh_key_preconfigured: false

training:
  # The framework and version the run used. It lives inside the container
  # image, which this tool never opens.
  framework: CHANGEME            # e.g. "NVIDIA PyTorch Release 25.04"
  # Optional short tag for that build. Written only when set.
  # framework_name: ngc25.04_pytorch

submission:
  submitter: CHANGEME
  division: closed               # closed | open
  notes:
    hardware: ""                 # becomes hw_notes
    software: ""                 # becomes sw_notes

`output` — where the file lands¶

Key	Type	Default	Notes
`dir`	path	`.`	Relative paths resolve against the config file, not the cwd
`file`	string	profile's own	Output filename

Only the deliverable is written to dir. Everything the collection layer produces goes into dir/.mlperf-sysinfo/ — see Architecture.

`${VAR}` — secrets stay out of the file¶

Any ${VAR} anywhere in the config is replaced from the environment, including inside lists. An unset variable is a config problem, reported against its path:

  ✗ power.redfish.password -> ${BMC_PASSWORD}    unset    not found in the environment

`extends` — share org defaults¶

# ~/.mlperf/org.yaml
submission:
  division: standardized
run:
  link_config: https://github.com/myorg/submission/tree/main/configs

# sysinfo.yaml
extends: ~/.mlperf/org.yaml
system:
  name: H100x8_vLLM

The child wins. Nested maps merge; lists replace wholesale. A chain that loops back on itself stops at a depth limit rather than following it round forever:

error  'extends' nested more than 8 deep -- is there a cycle?

Empty, unset, and absent¶

Three states that look alike in YAML and are not the same thing:

Written as	Means
`cooling: air`	Set
`cooling:`	Unset. Stays unset; nothing is guessed
`ssh:` with every entry commented out	Absent. Treated as if the key were not there

The last one is why deleting the final entry under nodes.ssh — or under run: — is not an error.

Emptying nodes.ssh is an error when nothing else is left to look at

An absent ssh list is only fine while some other machine is still named. With include_local: false and no serving.node either, there is nothing to collect from, and the config is rejected before any node is contacted:

$ mlperf-sysinfo check -c sysinfo.yaml
error  sysinfo.yaml: config is not valid
  (root): Value error, nothing to collect from: set nodes.include_local to true,
    list at least one target under nodes.ssh, or set serving.node

Leftover starter text¶

A config still carrying starter text has not been filled in, and check refuses it. Two patterns are recognised anywhere in the file:

anything containing changeme (so CHANGEME@example.com counts)
anything matching Insert … here or <…>

The insert rule is anchored on purpose, so genuine prose survives:

notes:
  hardware: "insert card in slot 3 before boot"   # fine, not starter text

Every string is scanned, not only the fields your profile requires — starter text in any field still reaches the submission file. The check report calls these placeholder.

Unknown and misspelled options¶

check reports an error for any option it does not recognise, and names a real option where one is close enough:

$ mlperf-sysinfo check -c sysinfo.yaml
error  sysinfo.yaml: config is not valid
  system.categry: unknown option. Did you mean "category"?
  nodes.include-local: unknown option. Did you mean "include_local"?
  submission.divison: unknown option. Did you mean "division"?

Matching ignores case, and treats - and _ as the same character.