StepFun released the weights of its Step 5 Preview model on October 15, keeping the open-sourcing date it announced when the model launched in September. The release includes BF16 and FP8 checkpoints on Hugging Face and ModelScope, a technical blog describing how the model was built, and supporting open-source developer tools.
Step 5 Preview is a sparse mixture-of-experts model designed for long-horizon agent tasks. It has 600B total parameters, of which 27B are active per token, and supports a context window of up to 1 million tokens. StepFun says it performs well in coding, software engineering, professional knowledge work and finance. The company also says the model currently ranks second worldwide on OpenRouter's weekly and daily usage charts.
Developers can serve the model through an OpenAI-compatible API with vLLM or SGLang, or run offline inference with Transformers. According to the model card, the FP8 checkpoint fits on a single node of four B300 GPUs or eight H200 GPUs, while the BF16 checkpoint needs eight B300 GPUs or two eight-GPU H200 nodes. StepFun notes that the full BF16 sharded workflow for Transformers has not yet been validated end to end.
The weights are not under a standard open-source license. They are released under the StepFun Model License 1.0, which the company describes as Apache-2.0-style but with additional attribution and commercial-license conditions.
The technical blog explains how StepFun co-designed the architecture and training for long agent runs. The model uses a narrow but deep design with 92 layers, which the company says gives more computation steps for implicit multi-hop reasoning while keeping parameter count and compute in check. To cut the cost of attention over a million-token context, it uses Sparse GQA, which combines block-level indexing with sparse computation. Indexing in blocks of 8 tokens reduces the cost of indexing and top-k selection to one-eighth.
For training stability in such a deep network, StepFun keeps key states and parameters in FP32 precision and applies fine-grained gradient clipping to curb activation growth and gradient spikes. Reinforcement learning ran across about a million verifiable environments covering software engineering, scientific reasoning, machine learning research and professional knowledge work. StepFun also built context compression into end-to-end policy training, so the agent learns to summarize task state and continue working, and paired it with finer-grained reward attribution for long trajectories.
SEE ALSO: StepFun Launches Step 5 Preview: 600B Sparse MoE, 1M Context, Weights Open Oct 15 · StepFun Step 5 Preview: ~600B MoE Long-Horizon Agent; Open Weights Due Oct 15 · StepFun's Step 5 Preview Comes to Tencent WorkBuddy, Free to Use Exclusively Until October 25