AI Observability — LiteLLM, Helicone, Langfuse

The "I have a working local AI stack — now I want to know what it's doing, set budgets, and trace bad outputs" tier. LiteLLM, Langfuse, and Helicone are the FOSS / self-host picks. Phoenix (Arize) rounds out the major options.

For the upstream stack see overview; for cloud / API mixed see cost math and cloud rentals. For the broader observability category see observability.

LiteLLM

★ ★ LiteLLM (github.com/BerriAI/litellm, MIT) — multi-provider LLM proxy with logging, budgets, fallback, and rate-limiting.

OpenAI-compatible proxy — point any OpenAI SDK at LiteLLM; LiteLLM routes to whichever backend (OpenAI, Anthropic, Bedrock, OpenRouter, Ollama, vLLM, anything).
Per-key budgets and rate-limits. Critical for multi-user / family / team setups.
Logging — to Postgres, S3, Helicone, Langfuse, etc.
Model routing — fallback chains ("try Claude, fallback to local Qwen on error").
Prompt caching for supported providers.
MIT license; self-host friendly. Also a paid hosted version, but the OSS proxy is full-featured.
★ ★ The center of a serious self-host AI stack.

# litellm config.yaml example
model_list:
  - model_name: gpt-5
    litellm_params:
      model: openai/gpt-5
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude
    litellm_params:
      model: anthropic/claude-sonnet-4-7
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: local-qwen
    litellm_params:
      model: ollama/qwen2.5-coder:32b
      api_base: http://localhost:11434
 
router_settings:
  routing_strategy: simple-shuffle
  fallbacks:
    - claude: [local-qwen]

Point Aider, Cline, Continue.dev, Open WebUI at LiteLLM, get one-source-of-truth logs, budgets, and routing.

Langfuse

★ ★ Langfuse (github.com/langfuse/langfuse, MIT for OSS / EE for some features) — observability + evals platform.

Trace / span model — every LLM call traced with inputs, outputs, latency, cost, metadata.
Eval framework — score traces with LLM-as-judge or custom Python; track quality regressions.
Prompt management — version / A/B test prompts.
Self-host with Docker Compose; hosted SaaS available.
MIT for the core; some enterprise features behind a license.
★ ★ The default OSS LLM observability platform.

Helicone

★ Helicone (github.com/Helicone/helicone, Apache 2.0) — proxy-based observability.

Drop-in proxy in front of OpenAI / Anthropic; logs every call with no code change.
Self-host or hosted.
Caching, rate-limiting, prompt management.
Less feature-rich than Langfuse for evals; simpler to start.

Phoenix (Arize)

Phoenix (github.com/Arize-ai/phoenix, ELv2) — observability + RAG-tracing-focused.

Especially good for RAG debugging — visualises retrieval quality.
Self-host friendly.
Different license shape (Elastic License v2, not strict OSS).

OpenLLMetry

OpenLLMetry (github.com/traceloop/openllmetry, Apache 2.0) — instrumentation library for LLM apps that emits OTel traces.

Integrates with any OTel backend (Jaeger, Tempo, Datadog, Honeycomb).
Auto-instruments OpenAI, Anthropic, LangChain, LlamaIndex.

What to actually instrument

★ Cost per call. $/M token math — multiply input + output. LiteLLM logs this.
★ Latency. First-token-time vs. total time matters for chat UX.
★ Error rate / fallback rate. How often is your fallback chain firing?
★ Tool-call success rate. For agent workloads — JSON parse failures, tool-call malformed, etc.
★ User feedback / thumbs up-down. Correlate to traces.
★ Eval scores. LLM-as-judge on output quality, periodically.

Patterns

All clients → LiteLLM → providers. One source of truth.
LiteLLM → Langfuse for traces. Every call gets a trace.
Pre-deploy eval — Langfuse evals over a fixed dataset before promoting a new model / prompt.
Budget caps — LiteLLM enforces; alerting on approaching limits.

Honest scoping

Single-user personal stack — you probably don't need observability beyond Open WebUI's chat history. Don't over-engineer.
Family / small-team self-host — LiteLLM is worth setting up just for budget enforcement.
Production / business workload — LiteLLM + Langfuse is the canonical pair.

Pick this if…

Multi-provider proxy with budgets: ★ ★ LiteLLM.
LLM observability platform with evals: ★ ★ Langfuse.
Quick proxy-flavoured logging: Helicone.
RAG debugging: Phoenix.
OTel-native instrumentation: OpenLLMetry.
Personal single-user stack, no obs needed: stay with Open WebUI's built-in chat history; don't over-build.

AI Observability — LiteLLM, Helicone, Langfuse

On this page