站内导读

中文

定位:Ricursive Intelligence 推出的 650 亿参数稠密 Transformer 语言模型,目前为预览版。 核心能力:在 12 万亿 token 上完成预训练,支持最长约 100 万 token(1,048,576)的上下文;除 BF16 版本外,还提供 NVFP4 量化版和基座模型版本可选。 适用人群:需要长上下文开源大模型做推理、微调或研究的团队。许可证为 Apache-2.0,在 Hugging Face 上收藏 487。

README · 详细文档

来源 · 查看原文


license: apache-2.0 ---

Ripple-65B Preview

Ricursive Logo

BF16 | NVFP4 | Base Model | DFlash2 | Launch Blog | License: Apache 2.0

Ricursive Intelligence introduces Ripple-65B Preview, a 65B parameter dense Transformer model pretrained on 12 trillion tokens with support for up to 1Mi (1,048,576 tokens) of context. Ripple-65B Preview delivers strong results on agentic coding, scoring 80.52 on TerminalBench 2.1 and surpassing significantly-larger US open-weight models such as Reflection Beam 501B (80.1), Meta Muse Spark Open 816B (76.4), Thinking Machines Inkling 975B (63.8), and NVIDIA Nemotron 3 Ultra 550B (56.4).

Our Preview release includes an instruction-tuned model in BF16 and NVFP4 numerics, and a BF16 base model. This repo contains the BF16 instruction-tuned model.

Starting from an initial Olmo 3 baseline, we iteratively improved all aspects of the training recipe including pretraining scaling laws, model architecture, data, and posttraining stack. Our final Ripple v1 model ladder improves Compute Efficiency (CE) on code by 16x relative to the Olmo 3 ladder, with the advantage further increasing with scale up to 55x for the full hero run.

Ripple-65B was codesigned to be trained and served efficiently on NVIDIA hardware. For our first model, a key objective was to make design decisions which reduce pretraining and posttraining infrastructure complexity while still enabling strong reasoning and agentic performance. Therefore, we chose to train a modernized dense Transformer model rather than use Mixture-of-Experts (MoE) like other labs. This decision quickly paid off, and we were able to train a strong model from scratch with a small team that scaled from 1 to 5 people during development, while maintaining solid MFU (40%) throughout pretraining and posttraining with simple sharding, no custom kernels, and no quantization (BF16).

Ripple-65B uses a thin-and-deep model aspect ratio, with a narrow d_model (5120) and 80 total layers. We switch from the Olmo 3 llama-like architecture to a Gemma 3N-like architecture to improve training stability at scale. To improve reasoning performance, we overprovision the number of attention heads while increasing the SWA-to-global layer ratio and reducing SWA window size to compensate for the increased cost. In addition, we added FFN activation clipping, gated attention, global NoPE, and value residuals.

The end result is a model that is especially capable at reasoning and agentic tasks given its model size, matching or exceeding other US open-weight models with more than 10x the number of total parameters. At only 65B total parameters, the model can fit on consumer hardware such as the NVIDIA DGX Spark. To encourage local model development, we provide an NVFP4 checkpoint and colab to help you get started. The model is also fast at token generation when running on datacenter GPUs, reaching 250 tok/s on 2xB200 batch=8 prior to quantization or custom kernels.

Pre-training. Ripple-65B Preview is pretrained from scratch on our 12T text-only predominantly-English and code corpus at 8K sequence length. Ripple-65B Base Preview is a pretrain-only checkpoint without any midtraining or synthetic QA data.

Post-training. We use a standard post-training recipe consisting of midtraining, long context, SFT, long context SFT, and mixed domain RL. Our Preview release has gone through relatively light posttraining, with a focus on agentic coding.

We show a brief overview of each stage below:

StageTokensStepSeqlenBatch Size
Pretrain12T877,5008K1408 -> 3072
Midtrain200B8,00064K384
Long Context6.5B3892M8
SFT183B7,274128K192
LC SFT13.5B8022M8
RLN/A700128K2048

Instruct Model Benchmarks

We compare Ripple-65B Preview against currently-available US open-weight frontier models. All models are tested at full BF16 precision with maximum effort levels.

Each lab typically only provides a limited subset of available benchmarks. Our table below shows official model card scores where available, and we use our own in-house eval flow for missing evals (italized). For SWE-bench Pro v2.0 Hard and DeepSWE 1.1, we use Terminus 2 for Ripple-65B Preview. In other cases where the harness is not specified, we use OpenCode. A higher number is better for all benchmarks.

Replicating benchmark scores has been challenging, particularly for modern agentic evals, due to the number of factors which can cause eval scores to shift. For example, we found that Terminal-Bench 2.1 scores were universally lower (with the exception of Muse Spark Open) when using the official Terminus 2 harness at 128K sequence length with summarization. Therefore, our results represent a best-effort attempt at a fair comparison.

BenchmarkRipple-65B
Preview
Nemotron 3
Ultra 550B
Inkling
975B
Muse Spark
Open 816B
Reflection
Beam 501B
Model
ArchitectureDenseMoEMoEMoEMoE
Total params65B550B975B816B501B
Active params65B55B41B42B23B
Pre-training tokens12T20T45TN/A23.8T
Knowledge
MMLU89.692.091.392.692.4
MMLU-Pro84.286.887.387.976.5
GPQA Diamond85.287.087.289.090.6
Math
AIME 202497.194.297.597.996.7
AIME 202598.394.291.7100.099.6
AIME 202695.095.497.198.397.7
HMMT Feb 202690.580.386.393.993.9
Coding & Agentic
Terminal-Bench 2.1 (self-reported)80.556.463.876.480.1
Terminal-Bench 2.1 (Terminus 2)80.555.156.278.770.8
SWE-bench Verified76.571.977.678.781.6
SWE-bench Pro v1.061.4246.954.358.266.1
SWE-bench Pro v2.0 Hard79.130.739.672.476.5
DeepSWE 1.144.30.21.860.043.3
SciCode (Inspect AI)39.444.646.158.549.2
τ³-bench banking31.2722.623.714.436.1
AutomationBench (Public)36.48.414.642.439.3
MCP Atlas71.6762.276.089.280.8
Safety
FORTRESS Adversarial83.879.378.094.292.1
FORTRESS Benign95.589.195.982.686.4
StrongREJECT97.598.298.694.795.8
Instruction Following
IFBench79.281.779.878.679.6
Long Context
AA-LCR75.065.478.380.778.2

Base Model Benchmarks

We show traditional pretraining evals comparing Ripple-65B Base Preview against a selection of open-weight base models. Although we would have liked to compare against the same set of models throughout the model card, many companies are now choosing to only open-source their instruction-tuned models.

With the rise of the importance of posttraining, many modern LLM recipes now blend posttraining data earlier into the training process, including during pretraining. For our first model, we kept our pretraining data clean with minimal synthetic data, focusing on high quality natural data rather than aggressively targeting evals. We instead added significantly more synthetic data during midtraining, which allowed us to quickly iterate on our synthetic data recipe without blocking pretraining.

Ripple-65B Base Preview is a pretrain-only checkpoint. The base model overall beats Poolside Laguna M.1, and is comparable with DeepSeek V3.1 and V4 Flash on reasoning, math, and code. Nemotron 3 Ultra dominates on traditional pretraining evals, likely due to their heavy use of task-seeded synthetic data. However, its pretraining eval performance did not translate to modern agentic tasks, likely due to their synthetic data being out of distribution relative to these tasks.

We report official model results where available. For our own model and for missing evals, we use our internal fork of Olmes.

Benchmark (Metric)# ShotsRipple-65B
Preview
Laguna
M.1 225B
DeepSeek
V3.1 671B
DeepSeek
V4 Flash 284B
Nemotron 3
Ultra 550B
Model
Architecture—DenseMoEMoEMoEMoE
Total params—65B225B671B284B550B
Active params—65B23B37B13B55B
Training tokens (T)—12T30T+15.6T32T+20T
World Knowledge
MMLU (MC acc)5-shot83.776.688.388.789.1
MMLU-Pro (CoT EM)5-shot61.552.257.868.379.1
AGIEval-En (acc)3-5-shot74.466.281.983.978.7
GPQA main (CoT EM)5-shot37.731.730.844.049.6
GPQA-Diamond (CoT EM)5-shot38.427.837.444.453.0
C-Eval (MC acc)5-shot69.659.390.792.182.2
SuperGPQA (MC acc)5-shot34.224.543.146.552.9
SimpleQA-Verified (EM)25-shot10.110.817.330.127.1
Language & Reasoning
BBH (CoT EM)3-shot85.679.687.286.988.0
BBEH (EM)1-shot17.914.218.425.421.2
DROP (F1)1-shot77.866.881.288.687.6
HellaSwag (acc_norm)0-shot84.686.286.885.789.1
HellaSwag (acc_norm)10-shot87.288.189.488.890.5
ARC-Challenge (MC acc)25-shot95.790.895.195.897.4
OpenBookQA (acc_norm)0-shot47.448.650.448.048.6
PIQA (acc_norm)0-shot83.684.885.284.783.8
WinoGrande (acc)5-shot83.483.283.281.479.3
Code & Math
HumanEval (pass@1, n=20, T=0.2)0-shot70.983.469.869.583.8
MBPP Sanitized (pass@1, n=32, T=0.2)3-shot81.581.175.274.686.0
BigCodeBench (pass@1, n=5, T=0.8)3-shot49.250.559.956.855.6
GSM8K (CoT EM)8-shot88.580.479.690.888.1
GSM1K (CoT EM)8-shot88.479.781.883.290.2
MATH (flex EM)4-shot59.152.559.757.482.0
MGSM (CoT EM, 11-lang avg)8-shot78.265.481.785.787.7

Citation

@misc{ricursive2026ripple65b,
      title={Ripple-65B: Pushing the Frontier of Local Intelligence},
      author={Ricursive Intelligence},
      year={2026},
}

Contact

For questions and comments, please contact us at info@ricursive.com. To learn more, please visit our website.

相关项目 · HuggingFace

查看全部「HuggingFace」项目 →