Skip to content

prepare-training-data-bert

Automatically generated README for this automation recipe: prepare-training-data-bert

Category: MLPerf benchmark support

License: Apache 2.0

  • CM meta description for this script: _cm.json
  • Output cached? True

Reuse this script in your project

Install MLCommons CM automation meta-framework

Pull CM repository with this automation recipe (CM script)

cm pull repo mlcommons@cm4mlops

cmr "prepare mlperf training data input bert" --help

Run this script

Run this script via CLI
cm run script --tags=prepare,mlperf,training,data,input,bert[,variations] [--input_flags]
Run this script via CLI (alternative)
cmr "prepare mlperf training data input bert [variations]" [--input_flags]
Run this script from Python
import cmind

r = cmind.access({'action':'run'
              'automation':'script',
              'tags':'prepare,mlperf,training,data,input,bert'
              'out':'con',
              ...
              (other input keys for this script)
              ...
             })

if r['return']>0:
    print (r['error'])
Run this script via Docker (beta)
cm docker script "prepare mlperf training data input bert[variations]" [--input_flags]

Variations

  • Group "implementation"

    Click here to expand this section.

    • _nvidia (default)
      • ENV variables:
        • CM_TMP_VARIATION: nvidia
    • _reference
      • ENV variables:
        • CM_TMP_VARIATION: reference
Default variations

_nvidia

Script flags mapped to environment

  • --clean=value → CM_MLPERF_TRAINING_CLEAN_TFRECORDS=value
  • --data_dir=value → CM_DATA_DIR=value

Native script being run

No run file exists for Windows

Script output

cmr "prepare mlperf training data input bert [variations]" [--input_flags] -j