Overview:
This notebook provides instructions to run the latest version of the Gemma 4 model using Google Colab with full quantization from unsloth. The model has 31B parameters and uses full precision.
Requirements:
- Python 3.8+ installed in Colab
- Install required packages via pip: `pip install -U torch==1.12.1 torchvision==0.12.1 torchaudio==0.12.1
- Enable the GPU runtime by running: `import os; os.environ["CUDA_VISIBLE_DEVICES"] = "0"
Instructions:
- Download this notebook: Gemma 4 Colab Notebook
- Open the JSON file in a text editor and copy its content into a new Python script file.
- Upload the script to a Colab notebook and execute it.
- The notebook will download the model, perform full quantization, and run it on the GPU.
Note:
This is a prototype and may contain errors. Please test it in a safe environment before deploying it.