Summary
About
Collective Mind (CM) is a small, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface to make it easier to build, run, benchmark and optimize applications across diverse models, data sets, software and hardware.
CM is a part of Collective Knowledge (CK) - an educational community project to learn how to run emerging workloads in the most efficient and cost-effective way across diverse and continuously changing systems.
CM includes a collection of portable, extensible and technology-agnostic automation recipes with a common API and CLI (aka CM scripts) to unify and automate different steps required to compose, run, benchmark and optimize complex ML/AI applications on any platform with any software and hardware.
CM scripts extend the concept of cmake with simple Python automations, native scripts
and JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are
continuously extended by the community and MLCommons members
to run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux
and any other operating system, in a cloud or inside automatically generated containers
while keeping backward compatibility.
CM scripts were originally developed based on the following requirements from the MLCommons members to help them automatically compose and optimize complex MLPerf benchmarks, applications and systems across diverse and continuously changing models, data sets, software and hardware from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors: * must work out of the box with the default options and without the need to edit some paths, environment variables and configuration files; * must be non-intrusive, easy to debug and must reuse existing user scripts and automation tools (such as cmake, make, ML workflows, python poetry and containers) rather than substituting them; * must have a very simple and human-friendly command line with a Python API and minimal dependencies; * must require minimal or zero learning curve by using plain Python, native scripts, environment variables and simple JSON/YAML descriptions instead of inventing new workflow languages; * must have the same interface to run all automations natively, in a cloud or inside containers.
Maintainers
Resources
- CM v2.x (2022-cur) (stable): installation on Linux, Windows, MacOS ; docs ; popular commands ; getting started guide
- CM v3.x aka CMX (2024-cur) (stable): docs
- MLPerf inference benchmark automated via CM
- Run MLPerf for submissions
- Run MLPerf at the Student Cluster Competition'24
- Examples of modular containers and GitHub actions with CM commands:
- GitHub action with CM commands to test MLPerf inference benchmark
- Dockerfile to run MLPerf inference benchmark via CM
License
Citing CM project
If you found CM automations useful, please cite this article: [ ArXiv ], [ BibTex ].
You can learn more about the motivation behind these projects from the following presentations:
- "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ ArXiv ]
- ACM REP'23 keynote about the MLCommons CM automation framework: [ slides ]
- ACM TechTalk'21 about Collective Knowledge project: [ YouTube ] [ slides ]
Acknowledgments
The Collective Mind (CM) automation framework was originally developed by Grigori Fursin, as a part of the Collective Knowledge educational initiative, sponsored by cTuning.org and cKnowledge.org, and contributed to MLCommons for the benefit of all. This open-source technology, including CM4MLOps/CM4MLPerf, CM4ABTF, CM4Research, and more, is a collaborative community-driven project made possible by our amazing volunteers, collaborators, and contributors!