Ahma-3B-exl2-6bpw / setup_steps.txt
kyynaama's picture
Upload folder using huggingface_hub
1e9bf1c verified
raw
history blame contribute delete
385 Bytes
Setup Runpod account and add few dollars.
Setup RTX 4090 with
Disk 50GB/50GB
Select image:
runpod/pytorch:2.2.0-py3.10-cuda12.1.1-devel-ubuntu22.04
pip install --upgrade --force-reinstall --no-cache-dir torch==2.1.1 triton --index-url https://download.pytorch.org/whl/cu121
pip install "unsloth[cu121-ampere] @ git+https://github.com/unslothai/unsloth.git"
--> run notebook