The four primitives

Tinker exposes four API operations — enough algorithmic control for most RL workflows, with the infrastructure complexity abstracted away:

These four operations map directly to RLVR's three phases. sample() handles rollout generation, forward_backward() handles the policy gradient update, and optim_step() applies the update to LoRA weights. Reward scoring (Phase 2) happens in your own code — for RLVR it's just checking math correctness or running unit tests.

Per-token pricing makes cost transparent

Three token types, three prices:

This separation is powerful: you can see exactly how much you're spending on rollout vs training. On GPU-hours, those costs are entangled in a single bill where idle time during phase transitions is invisible overhead.

What the abstraction absorbs

Behind these four API calls, Tinker handles an enormous amount of infrastructure complexity that you'd otherwise build yourself:

• Inference engine — vLLM/SGLang-style continuous batching, prefix caching, KV cache management
• Multi-GPU parallelism — FSDP, tensor parallelism, expert parallelism for MoE models
• LoRA adapter management — loading, swapping, batching across users via SGMV kernels
• Async RL scheduling — overlapping rollout with training via configurable off-policy tolerance
• Hardware optimization — FP4/FP8 quantization, speculative decoding, hardware-specific tuning

Setting this up from scratch takes 2-4 weeks of ML engineering time. At $75/hr, that's ~$6,000 before you generate a single training token. If you expect to run many training iterations, you can amortize this setup cost — but ongoing maintenance (keeping up with framework changes, debugging distributed training issues, tuning for new model sizes) means the engineering overhead never fully goes to zero. This is the hidden cost that makes Tinker competitive even when its per-token price exceeds raw GPU-hour costs.

Built-in loss functions

Tinker provides several loss functions out of the box, plus a custom loss interface for arbitrary differentiable objectives:

GRPO is not a separate loss function — it's importance_sampling (or ppo) with group rollouts and per-group advantage normalization in your own code. The Tinker cookbook training scripts do exactly this.

The abstraction spectrum

The trade-off is control vs. operational burden. Frameworks like veRL and SLiME give you full control over the RL loop but require managing GPU clusters, tuning parallelism, and debugging distributed failures. Managed APIs abstract all of that away at the cost of less flexibility.

GPU-hours hide the real cost

Self-hosted RLVR is billed in GPU-hours — a unit that tells you almost nothing about what you're paying for. Your 8xH100 node spends 80-90% of its time doing inference (rollout generation), but you're paying training-tier prices for what is fundamentally an inference workload. Phase transitions waste cycles. Synchronization barriers waste cycles. The hourly bill doesn't distinguish any of this.

Tinker's bet: per-token pricing is the right abstraction for post-training, just as it was for inference. You stop paying for idle time. Cost becomes deterministic and proportional to actual work.

An important nuance: Tinker is fundamentally a general-purpose trainer and sampler, not an RL framework. It trades peak efficiency for generality. The four primitives (sample, forward_backward, optim_step, save_state) are building blocks for any ML training loop — the RL loop (rollout → reward → update) is a higher-level program you write on top of these primitives. This is by design: Tinker targets “PyTorch users” who think in terms of forward passes and gradient steps, rather than RL-specific concepts. The tradeoff is that a purpose-built RL system (like veRL or SLiME with tight Megatron integration) can achieve higher hardware utilization for specific RL workloads.