0.8.0 — Contracts + Evals for RubyLLM
Narrative repositioning + small API additions. Internal architecture unchanged: no Step::Base refactor, no breaking changes to existing DSL.
Added
thinking(effort:, budget:)class macro onStep::Base— mirrorsRubyLLM::Agent.thinkingsignature exactly. Stored as{ effort:, budget: }hash; reader returns the hash; supports:defaultreset semantics; superclass inheritance likemodel/temperature. The convenience aliasreasoning_effort(:low)is implemented asthinking(effort: :low)— single normalized state, not separate ivar.- Adapter wiring for
with_thinking— whenthinkingis set on the Step class, OR whenreasoning_effort:is passed through context, OR when an attempt config inretry_policy escalate(...)carriesreasoning_effort:, the RubyLLM adapter resolves the effective{ effort:, budget: }hash and forwards it viachat.with_thinking(**)— provider-agnostic (supports OpenAIreasoning_effortAND Anthropic extended-thinking budget). Precedence: per-attempt / contextreasoning_effortoverrides class-levelthinking[:effort]; budget is taken from class-levelthinking[:budget]. Behavioural change vs 0.7.x:reasoning_effortis now forwarded viawith_thinkinginstead ofwith_params. Same wire-level OpenAI parameter; provider-agnostic Anthropic support is now automatic.
Dependencies
ruby_llmconstraint bumped from~> 1.0to~> 1.12—Chat#with_thinkingis the canonical path for reasoning effort + extended thinking; it shipped in RubyLLM 1.12. Adopters onruby_llm < 1.12need to bump RubyLLM before upgrading this gem to 0.8.0.
Changed
- Tagline + README opening — repositioned around "Contracts + Evals for RubyLLM". New "Relation to RubyLLM::Agent" section explicitly frames Step as a sibling abstraction (same niche as Agent, wider contract), not an alternative or foundation. README does not claim "Step uses Agent under the hood" — current call path is
Step → Runner → Adapters::RubyLLM → RubyLLM.chatdirectly. TokenEstimatordocumented as heuristic — module docstring expanded with explicit "±30% accuracy" framing. Refusal messages fromLimitCheckernow include(heuristic ±30%)suffix so adopters know the pre-flight number is estimated, not measured. RubyLLM 1.14 also has no pre-flight tokenizer;RubyLLM::Tokensis post-hoc only.CostCalculatorrepositioned in docs — module narrative reframed from "cost calculator" to "fine-tune pricing registry + lookup with fallback chain". Math methods (compute_cost,token_cost, etc.) were already private; this release makes the docs match. Public API surface unchanged:register_model,unregister_model,reset_custom_models!,calculate.output_schemareframed in docs — described as "wrapper aroundRubyLLM::Schema+ client-side validation step", not a standalone feature. The schema language is identical to whatRubyLLM::Agent.schemaaccepts; the difference is what wraps it.- README retry framing —
retry_policy escalate(...)(model escalation on validation failure) is the marketed default.retry_policy attempts: N(same-model retry) stays in the API for backward compat and niche cases (subjective criteria, multi-step pipelines, weaker models) but is no longer marketed as a recommended default. Empirical basis: four small experiments across PDF quiz generation, GSM8K math (n=30 + n=120), and multi-constraint schedule generation found no useful lift for nano-class models on tasks with clear correctness criteria.
Documentation
- New disambiguation paragraphs in
prompt_ast.md(Step.input_typevsRubyLLM::Agent.inputs;Prompt::Buildermulti-role DSL vs Agent ERB single-string template loader),testing.md(Step.observevsChat#on_end_message/on_tool_call),output_schema.md(relation toAgent.schema), andoptimizing_retry_policy.md(orthogonal model + thinking dimensions). getting_started.mdrefusal message example updated to include the new(heuristic ±30%)suffix.
Issues closed
- #11 (Optimizer is blind to same-model attempts) — closed after empirical experiments.
attempts: Nretry stays in API; not marketed as a default. - #6 (Production cost reporting) — already implemented in 0.7.x; close confirmed.
Not in this release (deferred)
output_schemaProc form for runtime-input-aware schemas (parity withAgent.schemaProc form). Additive, low-risk; deferred to 0.9 to keep 0.8 scope tight.- H4 (Step composing
RubyLLM::Agentinternally as config holder) — verified feasible but ROI insufficient for current adopter base; trigger-based revisit, no calendar commitment.