siiai/siirl-metax repository overview

⁠ siiRL: Shanghai Inovation Institute RL Framework for LLM Post-Training

| 📄 Paper⁠ | 📚 Documentation⁠ | 💬 Contact Us⁠ |

siiRL is a novel, fully distributed reinforcement learning (RL) framework designed to break the scaling barriers in LLM post-training. Developed by researchers from Shanghai Innovation Institute, siiRL tackles the critical performance bottlenecks that limit current state-of-the-art systems.

By eliminating the centralized controller common in other frameworks, siiRL delivers near-linear scalability, dramatic throughput gains, and unprecedented flexibility for RL-based LLM development.

⁠🚀 Highlights

• Near-Linear Scalability: The multi-controller paradigm eliminates central bottlenecks by distributing control logic and data management across all workers, enabling near-linear scalability to thousands of GPUs.

• SOTA Throughput: Fully distributed dataflow architecture minimizes communication and I/O overhead, achieving SOTA throughput in data-intensive scenarios.

• Flexible DAG-Defined Pipeline: Decouple your algorithmic logic from the physical hardware. With siiRL, you can define complex RL workflows as a simple Directed Acyclic Graph (DAG), enabling rapid, cost-effective, and code-free experimentation.

• Cross-Hardware Compatibility: siiRL now officially supports Huawei's Ascend NPUs, providing a high-performance alternative for training and inference on different hardware platforms.

• Proven Performance & Stability: Extensively benchmarked on models from 7B to 72B, siiRL delivering excellent performance across a wide range of tasks. Its advantages are particularly evident in data-intensive workloads such as long-context and multi-modal training.

⁠📚 How to Use

Please refer to GitHub: https://github.com/sii-research/siiRL⁠