To build and run docker, use the following commands
./docker/build.sh
To run docker
./docker/run_docker.sh
We are working with `llama3.2-1B-Instruct' model. Basically to run this on our computer, you need to go to llama3 website and signup.
Once you have signed up, you can list the available models for downloads
# List models
llama model list # --show-all
Select a model that you want to download
llama model download --source meta --model-id llama3.2-1B-Instruct
# prompts for download url
Once the models are downloaded, you want to convert it to huggingface format to use hugging face libraries
python3 models/llama/convert_to_hf_format.py --input_dir llama/checkpoints/Llama3.2-1B-Instruct --output_dir llama/checkpoints/Llama3.2-1B-Instruct_hf --llama_version 3.2 --model_size 1B
Look at the script for more details. To run llama model now, please run
python3 models/llama/chat_on_llama.py --checkpoint llama/checkpoints/Llama3.2-1B-Instruct_hf/