Library of the Week — LangFuse

LangFuse — open-source observability and evaluation platform for LLM applications

GitHub · Language: TypeScript/Python · License: MIT (core)

What it does

Langfuse gives you full tracing, logging, and evaluation for LLM pipelines — from individual model calls to multi-step agent traces. It’s built for teams who need to debug production issues, run evals, and track costs without wiring up their own telemetry stack. Works self-hosted or via their cloud offering.

Why it stands out

• Trace-first design: Every LLM call, tool invocation, and retrieval step is captured as a structured span with latency, token counts, and cost attribution — not just flat logs. You can reconstruct exactly what happened in a complex agent run.
• Integrated eval loop: You can attach human feedback, run model-based scoring, or pipe in dataset evals all within the same interface — no separate eval tool required.
• Framework-agnostic: Native integrations for LangChain, LlamaIndex, OpenAI SDK, and plain HTTP — if a framework emits spans, Langfuse can ingest them. The Python observe decorator works anywhere.
• Self-hostable with zero lock-in: Docker Compose deploy in minutes; your traces stay on your infrastructure. Competitors like Arize or Helicone push harder toward their cloud.

Quick start

from langfuse.decorators import observe, langfuse_context
from openai import OpenAI

client = OpenAI()

@observe()
def summarize(text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-5.4-nano",
        messages=[{"role": "user", "content": f"Summarize: {text}"}],
    )
    langfuse_context.update_current_observation(
        usage={"input": response.usage.prompt_tokens,
               "output": response.usage.completion_tokens}
    )
    return response.choices[0].message.content

result = summarize("Langfuse traces every step of your LLM pipeline.")

Set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_HOST as env vars — that’s it. The trace appears in your dashboard immediately.

When to use it

• You’re running agents or multi-step RAG pipelines and need to understand which step is slow, hallucinating, or burning tokens.
• You want a unified place to run evals, review traces, and collect human feedback without stitching together four different tools.
• You’re self-hosting and need full data residency — Langfuse’s Docker path is genuinely first-class, not an afterthought.

When to skip it

• If you’re already deep in the OpenTelemetry ecosystem and have Grafana/Jaeger wired up, adding Langfuse may duplicate your tracing infrastructure rather than simplify it.
• For pure batch offline evals with no production tracing need, Braintrust or Ragas are more focused tools.

The verdict

Langfuse has quietly become the default observability layer for teams building serious LLM products — it’s one of the few tools that handles tracing and evals without feeling stitched together. The self-host story is strong enough that even security-conscious teams can adopt it without compromise. If you’re moving a prototype toward production and still flying blind on what your pipeline is actually doing, start here.