Cost Calculator
Plug in your parameters. Compare Tinker vs self-hosted.
Cookbook presets
?Each 'group' is one prompt that gets G rollouts — effectively the batch size. Cookbook defaults: 128 (RL), 100 (math_rl), 8 (harbor_rl).
groups_per_batch in cookbook
64
?Rollouts per prompt. Called group_size in the cookbook. More rollouts = lower variance advantage estimates but higher cost. Cookbook defaults: 16 (RL), 4 (math_rl/harbor_rl), 8 (code_rl).
16
?Total RL iterations. Cookbook recommends ≥100 steps. math_rl shows gains at 180 steps, code_rl at 100 steps.
180
?max_tokens in the cookbook — max completion length per rollout. Cookbook READMEs: 512 (math_rl MATH), 1024 (math_rl GSM8K), 8192 (harbor_rl), 24576 (code_rl). ScaleRL uses 14K-32K for reasoning. Actual average is often lower than max.
512
?Average tokens in each training prompt. Varies by task: ~100-256 for math, 512-1024 for code with test cases.
200
?Completions typically grow 1.5-5x over training as the model learns longer reasoning chains. R1-style reasoning models can see 4-5x growth. This averages the cost across the run.
1.5x
?lora_rank in the cookbook. Default 32 for RL, 64 for RLHF. Does not affect cost — LoRA params are ~0.5% of base model params so the FLOP difference is negligible. Tinker also charges a fixed per-train-token rate regardless of rank. Rank matters for quality and VRAM, not price.
?How many optimizer updates to take on the same batch of rollouts before generating new ones (num_substeps in cookbook). Each step drifts the policy further from the rollout policy. Practice: default 1 (safe), 2–4 (common), 16 (DAPO — aggressive). Tülu 3 reduces from 4 to 1 at 405B specifically for stability. Training cost scales linearly with this value.
1
for rollout?The inference engine used for rollout generation determines MBU (Model Bandwidth Utilization) — how efficiently the GPU's memory bandwidth is used during batched autoregressive decode. Higher MBU = faster rollouts = lower self-hosted cost. This only affects the self-hosted estimate. Tinker abstracts the engine away.
PagedAttention, continuous batching. ~60% MBU at BS=1 (Databricks). Most stable. Used by OpenRLHF, veRL, Unsloth.RadixAttention prefix caching. ~29% faster than vLLM (PremAI 2026). Best for RLVR shared-prefix workloads.No continuous batching, no PagedAttention. 14-24x slower than vLLM at large batch. TRL default.Manual — set the slider below.
MBU 70%
low due to LoRA overhead?Model FLOPs Utilization during the training (backward pass) phase. RL training has much lower MFU than SFT because of multi-phase overhead: the GPU alternates between rollout generation, log-prob computation, reference model inference, and gradient updates. With LoRA, only ~0.5% of parameters update but gradients still flow through the full model. Colocated setups (actor + vLLM on same GPUs) further reduce MFU due to memory contention. This only affects self-hosted cost.
Single-phase training, no rollout contention. 30-40% on H100 with FSDP. Source: Stas Bekman ml-engineering benchmarks.Proper GPU timesharing (CPU offload between phases). 8-15% end-to-end. Training kernels alone: ~14%. Source: Yotta Labs (veRL on MI300X).No offloading — actor + vLLM compete for GPU memory. 1-5% range. We measured 1.6% on 8xB200. Source: veRL issues #965, #2628.Manual — set the slider below.
MFU 10%
?Colocated: one GPU pool time-shares between inference and training. Disaggregated: separate pools running simultaneously — no idle time, but you pay for both. RollArt, PipelineRL, AReaL use disaggregated at scale.
One shared GPU pool.?Total GPUs shared between inference and training. Min 2 to hold Qwen3-8B.
8
?In synchronous RL, GPUs sit idle while waiting for rollouts (inference) to finish before training. Async RL overlaps them: start generating the next batch while the current batch trains. This increases GPU utilization but introduces off-policyness — rollouts were generated by an older policy. k=0 is fully synchronous (on-policy). k=1 overlaps generation with training (double buffer). k≥2 runs multiple batches in flight with stale-trajectory abort. Tinker: AsyncConfig(max_steps_off_policy=k). See /quality for the full spectrum.
k = 1
1-step off-policy (double buffer)
Generate batch N+1 while training on batch N. One step of staleness — within PPO clip range. Eliminates tail waste and sandbox idle.
Tinker: StreamMinibatchConfig(groups_per_batch=B, num_minibatches=N) — on-policy, overlaps sampling with training. Also: veRL, SLiME, verifiers-rl.
?code_rl and harbor_rl require a sandbox per rollout to execute generated code and verify rewards. math_rl has no sandbox. code_rl: single-turn, ~45s per rollout. harbor_rl: multi-turn (10 turns × ~30s/turn ≈ 5 min per rollout). Modal: $0.0000131/core/sec + $0.00000222/GiB/sec. Daytona: $0.0000140/vCPU/sec. SandboxFusion (local Docker) = $0.
Off — math_rl uses no sandbox. Enable for code_rl or harbor_rl (Modal, Daytona, or SandboxFusion).
?Models multi-turn agentic tasks (SWE-bench-style) where each turn adds context. Creates quadratic prefill cost growth — the #1 cost driver for agentic RL.
Off — standard flat completion model. Enable for SWE-bench-style multi-turn agentic tasks.
Tinker
Per-token pricing
$99
Rollout (sample tokens)($0.4/M tok)$47(118.0M tok)
Training tokens($0.4/M tok)$47(118.0M tok)
Prefill tokens($0.13/M tok)$5(36.9M tok)
Self-hosted
8x H100_SXM @ $2.69/GPU/hr, 70% rollout efficiencyeasy to get
$39.25
GPU cost (training)$28.49
Rollout gen time20 min
Training time1.3 hrs
GPU-hours14.6
Min GPUs needed2
Setup overheadmin+$10.76
Checkpoint storage
?LoRA adapter weights only (bf16, no optimizer state). Estimated as total_params × (rank/32) × 0.5% × 2 bytes. Storage charged at $0.1/GB-month.Checkpoint size
191 MB
Cost/month
$0.02
Cost for 3 mo
$0.06
3 mo
Total rollout tokens
0.1B
Total rollouts
184,320
Tokens per step
0.5M
Rollout % (self-hosted wall-clock)
25%