Model Providers

Every LLM in Helikon sits behind one trait: Model, from paigasus-helikon-core. There are no per-provider traits. Capability differences (streaming, tool calling, structured output, vision, prompt caching, …) are surfaced through a flag struct, ModelCapabilities, rather than split interfaces. Four adapters ship today — OpenAiModel, AnthropicModel, BedrockModel, and GeminiModel — and switching between them is a single line.

The `Model` trait

#![allow(unused)]
fn main() {
use async_trait::async_trait;
use futures_core::stream::BoxStream;
use paigasus_helikon::core::{
    CancellationToken, Model, ModelCapabilities, ModelError, ModelEvent, ModelRequest,
};

#[async_trait]
pub trait Model: Send + Sync {
    async fn invoke(
        &self,
        request: ModelRequest,
        cancel: CancellationToken,
    ) -> Result<BoxStream<'static, Result<ModelEvent, ModelError>>, ModelError>;

    fn capabilities(&self) -> ModelCapabilities;

    fn provider(&self) -> &str; // GenAI `gen_ai.provider.name`, e.g. "openai"
    fn model(&self) -> &str;    // GenAI `gen_ai.request.model`, e.g. "gpt-4o"
}
}

invoke and capabilities are the only methods an implementor must define. provider and model are provided methods with defaults ("unknown" and "" respectively); the shipped adapters override them so traces carry the real provider and model id.

invoke takes a ModelRequest and returns a stream of ModelEvents. The agent loop drives this trait; you rarely call invoke yourself — you hand a Model to an LlmAgent and let the loop run it.

Carrier types

These live in paigasus_helikon::core (re-exported from paigasus-helikon-core) and cross the model boundary:

ModelRequest — the request envelope: accumulated messages, the tools the model may call this turn, and a ModelSettings of provider-tuning knobs.
ModelSettings — temperature, top_p, max_output_tokens, a tool_choice (ToolChoice), a response_format (ResponseFormat), and OpenAI Responses' previous_response_id.
ResponseFormat — Text, JsonObject, or JsonSchema { name, schema, strict }. Used to request structured output; see Structured Output.
ModelEvent — the streaming union: TokenDelta, ReasoningDelta, ToolCallDelta, Usage { input_tokens, output_tokens, cached_input_tokens, reasoning_tokens }, and the terminal Finish { reason } (a FinishReason).
ModelCapabilities — the per-instance capability flags below.
ModelError — Unavailable, RateLimited, ContextLengthExceeded, Refused, Transport, Other. The loop does not auto-retry on these.

ModelRequest, ModelSettings, ModelEvent, ModelCapabilities, ResponseFormat, ToolChoice, FinishReason, and ModelError are all #[non_exhaustive], so new fields and variants are additive.

`ModelCapabilities`

A Copy flag struct, stable per Model instance, that tells the loop what the provider can do: streaming, tools, parallel_tool_calls, structured_output, server_managed_state, reasoning, vision, audio, prompt_caching. Construct from empty() (or default()) with chained const builders:

#![allow(unused)]
fn main() {
use paigasus_helikon::core::ModelCapabilities;

let caps = ModelCapabilities::empty()
    .with_streaming()
    .with_tools()
    .with_structured_output();
}

The shipped adapters

Four provider adapters ship today. All are published on crates.io and reached through the facade behind a feature flag. Each exposes a Model implementation plus a builder.

OpenAI — `paigasus-helikon-providers-openai`

Reached as paigasus_helikon::openai::OpenAiModel behind the openai feature (alias providers-openai). Covers the Chat Completions and Responses APIs. The builder reads OPENAI_API_KEY from the environment.

#![allow(unused)]
fn main() {
use paigasus_helikon::openai::OpenAiModel;

let model = OpenAiModel::chat("gpt-5-mini").build()?;
}

OpenAiModel::chat(id) returns an OpenAiModelBuilder; build() yields a Result<OpenAiModel, BuildError>.

Anthropic — `paigasus-helikon-providers-anthropic`

Reached as paigasus_helikon::anthropic::AnthropicModel behind the anthropic feature. Covers the Messages API. The builder reads ANTHROPIC_API_KEY from the environment, and the crate also exports the Anthropic-specific settings types CacheStrategy and ExtendedThinking.

#![allow(unused)]
fn main() {
use paigasus_helikon::anthropic::AnthropicModel;

let model = AnthropicModel::messages("claude-sonnet-4-6").build()?;
}

AnthropicModel::messages(id) returns an AnthropicModelBuilder; build() yields a Result<AnthropicModel, BuildError>.

Bedrock — `paigasus-helikon-providers-bedrock`

Reached as paigasus_helikon::bedrock::BedrockModel behind the bedrock feature. Covers the Converse streaming API — tool use and multi-turn conversations across Anthropic Claude, Amazon Nova, Amazon Titan, Meta Llama, Mistral, and Cohere model families hosted on AWS Bedrock. Structured output (forced-tool synthesis) is supported on Anthropic Claude, Amazon Nova, and Mistral; Titan, Llama, and Cohere degrade to plain text responses.

Disambiguation: this is the Bedrock Converse model provider. It is distinct from paigasus-helikon-runtime-agentcore, which is the Bedrock AgentCore runtime host — a separate crate implementing the AgentCore container contract; see Runtimes.

Construction

The simplest path loads AWS configuration from the standard credential chain:

use paigasus_helikon::bedrock::BedrockModel;

let model = BedrockModel::from_env("anthropic.claude-3-5-sonnet-20241022-v2:0").await?;

from_env is async because it may fetch IMDS or SSO tokens. The credential chain is lazy — auth failures surface at invoke() time, not here.

For explicit configuration, use the synchronous builder:

use aws_config::BehaviorVersion;
use paigasus_helikon::bedrock::BedrockModel;

let sdk_cfg = aws_config::defaults(BehaviorVersion::v2026_01_12())
    .region(aws_config::Region::new("us-east-1"))
    .load()
    .await;

let model = BedrockModel::converse("amazon.nova-pro-v1:0")
    .sdk_config(&sdk_cfg)
    .build()?;

BedrockModel::converse(id) returns a BedrockModelBuilder; build() yields a Result<BedrockModel, BuildError>. You can also inject a pre-constructed aws_sdk_bedrockruntime::Client via .client(c) for full control.

Family-gated structured output

The bedrock provider detects the model family from the Bedrock model ID. For families that support Bedrock's forced-tool-choice — Anthropic, AmazonNova, and Mistral — ResponseFormat::JsonSchema is synthesized as a hidden tool call: the model is forced to invoke an internal reserved tool whose input schema is the user-supplied JSON Schema, and the result is returned as a structured JSON response. For other families (AmazonTitan, Llama, Cohere, Unknown) a JsonSchema format degrades silently to a text response.

Cross-region inference profile prefixes (us., eu., ap., apac.) are stripped before family detection, so model IDs such as us.anthropic.claude-3-7-sonnet-20250219-v1:0 are correctly identified as Anthropic.

Reasoning content surfaced by the model (e.g. extended-thinking responses from Claude) is delivered as ModelEvent::ReasoningDelta in the stream — no extra configuration is required on this provider.

Tool-input schema rewriter

Bedrock's Converse API validator rejects JSON Schemas containing $ref/$defs, oneOf/anyOf/allOf, or meta-keywords such as $schema, format, and examples. The crate exposes rewrite_tool_schema(schema, Ruleset::Strict), which runs automatically on every tool spec before the request is sent:

$ref references are inlined recursively (cycles and excessive depth are terminated with a permissive {"type": "object"}).
oneOf/anyOf/allOf combinators (e.g. serde-generated tagged enums) are collapsed into a flat object whose properties union the variants' fields and whose shared tag key becomes an enum of variant names.
Unsupported meta-keywords are stripped.

This means schemars-derived schemas for tagged enums and generic structs pass Bedrock's validator without any manual schema adjustment.

Credentials and TLS

The provider uses the standard AWS credential chain via aws-config (environment variables, ~/.aws/credentials, SSO, IMDS, ECS task role, …) and the aws-lc-rs TLS backend — the same crypto provider the workspace already uses via reqwest/async-openai. Do not enable the AWS SDK ring feature alongside this crate; a second CryptoProvider panics.

Model ids are illustrative — swap them for any model your account can reach if the provider rejects the id.

Gemini — `paigasus-helikon-providers-gemini`

Reached as paigasus_helikon::gemini::GeminiModel behind the gemini feature. Covers the Gemini streaming API — tool use, structured output, and multi-turn conversations against Google's Gemini model family. Structured output is native (responseSchema) — the provider does not use forced-tool synthesis.

Note: Gemini rejects requests that combine structured output with active function calling. If your ModelRequest sets a ResponseFormat::JsonObject or ResponseFormat::JsonSchema while tools is non-empty or tool_choice is anything other than None, the provider returns a conflict error before sending.

Transports

Two transports are available:

Developer API — authenticates with an API key from the environment:

#![allow(unused)]
fn main() {
use paigasus_helikon::gemini::GeminiModel;

// Reads GEMINI_API_KEY or GOOGLE_API_KEY.
let model = GeminiModel::from_env("gemini-2.5-flash")?;
}

Or explicitly via the builder:

#![allow(unused)]
fn main() {
use paigasus_helikon::gemini::GeminiModel;

let model = GeminiModel::developer("gemini-2.5-flash")
    .api_key("your-api-key")
    .build()?;
}

Vertex AI — authenticates with a bearer token or TokenProvider:

use paigasus_helikon::gemini::GeminiModel;

// vertex-adc feature: reads GOOGLE_CLOUD_PROJECT + GOOGLE_CLOUD_LOCATION (default: "global").
// ADC credentials are discovered from the gcloud credential chain.
let model = GeminiModel::vertex_from_env("gemini-2.5-flash").await?;

Or with an explicit static bearer token:

#![allow(unused)]
fn main() {
use paigasus_helikon::gemini::GeminiModel;

let model = GeminiModel::vertex("gemini-2.5-flash", "my-project", "us-central1")
    .bearer_token("ya29.your-token")
    .build()?;
}

Enable the vertex-adc feature for Application Default Credentials via gcp_auth:

[dependencies]
paigasus-helikon-providers-gemini = { version = "0.1", features = ["vertex-adc"] }

JSON-Schema sanitizer

A JSON-Schema sanitizer runs automatically on every schema passed to responseSchema. It inlines $ref references, converts [T, "null"] type arrays to nullable: true, replaces const with single-item enum, renames oneOf to anyOf, strips unsupported keywords ($schema, additionalProperties, unevaluatedProperties, patternProperties, examples, default, $defs/definitions), and drops format values Gemini doesn't recognize for their type (keeping e.g. enum/date-time). Schemars- derived schemas for typical Rust structs and enums pass without manual adjustment.

Limitations

Remote-URL images, audio parts, and non-text tool-result parts are silently dropped during request translation. Reasoning streaming (ModelEvent::ReasoningDelta) is not yet emitted by this provider.

Model ids are illustrative — swap them for any model your account can reach.

Switching providers is one line

Because all adapters implement the same Model trait, the only code that changes between providers is the construction line. Everything downstream — the agent, its tools, the run context, the streamed result — is identical. Compare the two budgeting-assistant examples (budget_assistant_openai.rs and budget_assistant_anthropic.rs); the agent, the #[tool] functions, and the run are byte-for-byte the same, and the diff is one line:

// OpenAI
let model = OpenAiModel::chat("gpt-5-mini").build()?;

// Anthropic — same agent, same tools, same run
let model = AnthropicModel::messages("claude-sonnet-4-6").build()?;

// Bedrock (async construction — loads the AWS credential chain)
let model = BedrockModel::from_env("anthropic.claude-3-5-sonnet-20241022-v2:0").await?;

// Gemini Developer API
let model = GeminiModel::from_env("gemini-2.5-flash")?;

#![allow(unused)]
fn main() {
let agent = LlmAgent::builder::<()>()
    .name("budget-assistant")
    .model(model) // ← the only thing that differs is how `model` was built
    .instructions("You are a budgeting assistant. ...")
    .tools(tools![lookup_spending, budget_status])
    .build();
}

.model(model) accepts anything that implements Model, so your own adapter slots in the same way — implement invoke and capabilities on a type and the loop will drive it.

Enabling the providers

[dependencies]
paigasus-helikon = { version = "0.3", features = ["openai", "macros"] }
# or, for Anthropic:
# paigasus-helikon = { version = "0.3", features = ["anthropic", "macros"] }
# or, for Bedrock:
# paigasus-helikon = { version = "0.3", features = ["bedrock", "macros"] }
# or, for Gemini:
# paigasus-helikon = { version = "0.3", features = ["gemini", "macros"] }

Feature names are kebab-case (openai, anthropic); the pub use aliases are snake_case (openai, anthropic, providers_openai). See Workspace Layout for the full feature map and Crates for the published crate list.

Retrying transient errors

Provider calls can fail transiently — rate limits (RateLimited), 503s (Unavailable), or dropped connections (Transport). Per ADR-10 the agent loop never auto-retries; retry is an opt-in composition-layer concern.

paigasus-helikon-runtime-tokio provides a RetryingModel<M> decorator: it wraps any Model and retries those transient variants with exponential backoff and jitter. It is configured by wrapping the model (not via RunConfig), and is disabled unless you wrap.

use std::time::Duration;
use paigasus_helikon_runtime_tokio::{RetryPolicy, RetryingModel};

let policy = RetryPolicy::new()
    .max_attempts(4)
    .base_delay(Duration::from_millis(250));
let resilient = RetryingModel::new(model, policy);

Retry covers connection establishment: a retryable error that arrives before any content has streamed is retried; once tokens or tool-call deltas have been emitted, a later error is surfaced rather than retried (output can't be un-emitted). RateLimited { retry_after_ms } waits at least the provider's hint, and backoff sleeps abort promptly on run cancellation.

Paigasus Helikon