Skip to content

LLama 2

The benchmark reference for LLama 2 can be found in this link, and here is the PR for the minified benchmark implementation: link.

This video explains all the following steps:

IMAGE ALT TEXT HERE

Project setup

An important requirement is that you must have Docker installed.

# Create Python environment and install MLCube Docker runner 
virtualenv -p python3 ./env && source ./env/bin/activate && pip install pip==24.0 && pip install mlcube-docker
# Fetch the implementation from GitHub
git clone https://github.com/mlcommons/training && cd ./training
git fetch origin pull/749/head:feature/mlcube_llama2 && git checkout feature/mlcube_llama2
cd ./llama2_70b_lora/mlcube

Inside the mlcube directory run the following command to check implemented tasks.

mlcube describe

Extra requirements

Install Rclone in your system, by following these instructions.

MLCommons hosts the model for download exclusively by MLCommons Members. You must first agree to the confidentiality notice.

When finishing the previous form, you will be redirected to a Drive folder containing a file called CLI Download Instructions, follow the instructions inside that file up to step: #3 Authenticate Rclone with Google Drive.

When finishing this step a configuration file for Rclone will contain the necessary data to download the dataset and models. To check where this file is located run the command:

 rclone config file
 ```

 **Default:** `~/.config/rclone/rclone.conf`

Finally copy that file inside the `workspace` folder that is located in the same path as this readme, it must have the name `rclone.conf`.

### MLCube tasks

* Core tasks:

Download dataset.

```shell
mlcube run --task=download_data -Pdocker.build_strategy=always

Train.

mlcube run --task=train -Pdocker.build_strategy=always
  • Demo tasks:

Download demo dataset.

mlcube run --task=download_demo -Pdocker.build_strategy=always

Train demo.

mlcube run --task=demo -Pdocker.build_strategy=always

Execute the complete pipeline

You can execute the complete pipeline with one single command.

  • Core pipeline:
mlcube run --task=download_data,train -Pdocker.build_strategy=always
  • Demo pipeline:
mlcube run --task=download_demo,demo -Pdocker.build_strategy=always