compile-model-for.qaic
Automatically generated README for this automation recipe: compile-model-for.qaic
Category: AI/ML optimization
License: Apache 2.0
- CM meta description for this script: _cm.json
- Output cached? True
Reuse this script in your project
Install MLCommons CM automation meta-framework
Pull CM repository with this automation recipe (CM script)
cm pull repo mlcommons@cm4mlops
Print CM help from the command line
cmr "qaic compile model model-compile qaic-compile" --help
Run this script
Run this script via CLI
cm run script --tags=qaic,compile,model,model-compile,qaic-compile[,variations] [--input_flags]
Run this script via CLI (alternative)
cmr "qaic compile model model-compile qaic-compile [variations]" [--input_flags]
Run this script from Python
import cmind
r = cmind.access({'action':'run'
'automation':'script',
'tags':'qaic,compile,model,model-compile,qaic-compile'
'out':'con',
...
(other input keys for this script)
...
})
if r['return']>0:
print (r['error'])
Run this script via Docker (beta)
cm docker script "qaic compile model model-compile qaic-compile[variations]" [--input_flags]
Variations
-
No group (any combination of variations can be selected)
Click here to expand this section.
_bert-99
- ENV variables:
- CM_COMPILE_BERT:
on
- CM_QAIC_MODEL_TO_CONVERT:
calibrate_bert_mlperf
- CM_QAIC_MODEL_COMPILER_PARAMS_BASE:
-aic-hw -aic-hw-version=2.0 -execute-nodes-in-fp16=Add,Div,Erf,Softmax -quantization-schema=symmetric_with_uint8 -quantization-precision=Int8 -quantization-precision-bias=Int32 -vvv -compile-only -onnx-define-symbol=batch_size,1 -onnx-define-symbol=seg_length,384 -multicast-weights -combine-inputs=false -combine-outputs=false
- CM_QAIC_MODEL_COMPILER_ARGS: ``
- CM_COMPILE_BERT:
- ENV variables:
_bert-99.9
- ENV variables:
- CM_COMPILE_BERT:
on
- CM_QAIC_MODEL_TO_CONVERT:
bert_mlperf
- CM_QAIC_MODEL_COMPILER_PARAMS_BASE:
-aic-hw -aic-hw-version=2.0 -convert-to-fp16 -vvv -compile-only -onnx-define-symbol=batch_size,1 -onnx-define-symbol=seg_length,384 -combine-inputs=false -combine-outputs=false
- CM_QAIC_MODEL_COMPILER_ARGS: ``
- CM_COMPILE_BERT:
- ENV variables:
_resnet50
- ENV variables:
- CM_COMPILE_RESNET:
on
- CM_QAIC_MODEL_TO_CONVERT:
compile_resnet50_tf
- CM_QAIC_MODEL_COMPILER_PARAMS_BASE:
-aic-hw -aic-hw-version=2.0 -quantization-schema=symmetric_with_uint8 -quantization-precision=Int8 -output-node-name=ArgMax -vvv -compile-only -use-producer-dma=1
- CM_COMPILE_RESNET:
- ENV variables:
_retinanet
- ENV variables:
- CM_COMPILE_RETINANET:
on
- CM_QAIC_MODEL_TO_CONVERT:
calibrate_retinanet_no_nms_mlperf
- CM_QAIC_MODEL_COMPILER_ARGS:
-aic-enable-depth-first
- CM_QAIC_MODEL_COMPILER_PARAMS_BASE:
-aic-hw -aic-hw-version=2.0 -compile-only -enable-channelwise -onnx-define-symbol=batch_size,1 -node-precision-info=<<<CM_ML_MODEL_RETINANET_QAIC_NODE_PRECISION_INFO_FILE_PATH>>> -quantization-schema-constants=symmetric_with_uint8 -quantization-schema-activations=asymmetric -quantization-calibration=None
- CM_COMPILE_RETINANET:
- ENV variables:
-
Group "batch-size"
Click here to expand this section.
_bs.#
- ENV variables:
- CM_QAIC_MODEL_BATCH_SIZE:
#
- CM_QAIC_MODEL_BATCH_SIZE:
- ENV variables:
_bs.1
- ENV variables:
- CM_QAIC_MODEL_BATCH_SIZE:
1
- CM_QAIC_MODEL_BATCH_SIZE:
- ENV variables:
-
Group "calib-dataset-filter-size"
Click here to expand this section.
_filter-size.#
-
Group "mlperf-scenario"
Click here to expand this section.
_multistream
_offline
_server
_singlestream
(default)
-
Group "model-framework"
Click here to expand this section.
_tf
-
Group "nsp"
Click here to expand this section.
_nsp.14
_nsp.16
_nsp.8
_nsp.9
-
Group "percentile-calibration"
Click here to expand this section.
_pc.#
- ENV variables:
- CM_QAIC_MODEL_COMPILER_PERCENTILE_CALIBRATION_VALUE:
#
- CM_QAIC_MODEL_COMPILER_QUANTIZATION_PARAMS:
-quantization-calibration=Percentile -percentile-calibration-value=<<<CM_QAIC_MODEL_COMPILER_PERCENTILE_CALIBRATION_VALUE>>>
- CM_QAIC_MODEL_COMPILER_PERCENTILE_CALIBRATION_VALUE:
- ENV variables:
-
Group "quantization"
Click here to expand this section.
_no-quantized
- ENV variables:
- CM_QAIC_MODEL_QUANTIZATION:
no
- CM_QAIC_MODEL_QUANTIZATION:
- ENV variables:
_quantized
(default)- ENV variables:
- CM_QAIC_MODEL_QUANTIZATION:
yes
- CM_QAIC_MODEL_QUANTIZATION:
- ENV variables:
Default variations
_quantized,_singlestream
Script flags mapped to environment
--register=value
→CM_REGISTER_CACHE=value
Native script being run
No run file exists for Windows
Script output
cmr "qaic compile model model-compile qaic-compile [variations]" [--input_flags] -j