matotezitanka/proteus-pytorch repository overview

⁠PROTEUS+STYX — Parameter Golf Competition Image

PyTorch + CUDA 12.8 + FlashAttention 3 (Hopper) + FineWeb data pre-loaded.

Built for the OpenAI Parameter Golf Challenge⁠ by Light Speed Up⁠.

⁠What's Inside

• PyTorch + CUDA 12.8 + cuDNN 9
• FlashAttention 3 (Hopper-native)
• FineWeb 10B dataset (80 train shards + validation) at /workspace/data/
• Tokenizer (1024 BPE) at /workspace/data/tokenizers/
• Jupyter Lab on port 8888
• SSH on port 22

⁠Quick Start (RunPod)

Use as your pod container image.

cd /workspace git clone https://github.com/openai/parameter-golf.git⁠ cd parameter-golf torchrun --standalone --nproc_per_node=8 train_gpt.py

Data is pre-loaded — no download wait.

⁠Competition Contributions

• Compliance guide⁠
• LeakyReLU slope sweep⁠ — 0.9 beats 0.5
• N-gram eval cache⁠ — 0.85 BPB
• PR #769⁠ — Submission

Built with PROTEUS+STYX by Light Speed Up

1. Click Save.
2. Then go to https://hub.docker.com/repository/docker/matotezitanka/proteus/general⁠ and set the description to:

DEPRECATED — Use matotezitanka/proteus-pytorch instead.