A short example script to run the smallest size of Google Gemma3 large language model (270M parameters).
These steps tested on:
- Ubuntu 24.04.02 LTS
- Python 3.12.3
- NVidia RTX A2000 GPU (Ampere)
- CUDA 12.8
Get an access token from your HuggingFace account and paste it to this command.
HF_TOKEN=<your token>Create a Python environment and activate it.
sudo apt update
sudo apt upgrade
sudo apt install python3-venv
cd
python3 -m venv venv_gemma3_270m
source ./venv_gemma3_270m/bin/activateInstall package requirements.
python3 -m pip install --upgrade pip
cd gemma3_270m
python3 -m pip install -r requirements.txtpython3 main.py --hf_token=${HF_TOKEN}Example run.
The generated text will change each run: the model decoding is stochastic.
$ python main.py --hf_token=${HF_TOKEN}
Device set to use cuda:0
Model: google/gemma-3-270m-it
Precision: torch.bfloat16
================================================================================
Input prompt: What causes climate change?
Climate change is caused by human activities that release greenhouse gases into the atmosphere. These gases trap heat and warm the planet.These steps tested on:
- MacBook Air M3 16GB
- macOS 15.6.1
- Python 3.12.6
- PyTorch 2.8.0
Get an access token from your HuggingFace account and paste it to this command.
HF_TOKEN=<your token>Create a Python environment and activate it.
brew install venv
cd
python3 -m venv .venv_gemma3
source ./.venv_gemma3/bin/activateInstall package requirements.
python3 -m pip install --upgrade pip
cd gemma3_270m
python3 -m pip install -r requirements.txtpython3 main.py --hf_token=${HF_TOKEN}Example run.
The generated text will change each run: the model decoding is stochastic.
$ python3 main.py --hf_token=${HF_TOKEN}
Device set to use mps
Model: google/gemma-3-270m-it
Device: mps:0
Precision: torch.bfloat16
================================================================================
Input prompt: What causes climate change?
Climate change is caused by human activities, primarily the burning of fossil fuels.