Library of the Week — Miroir

Miroir — wait, let me pick the right one.

LM Format Enforcer — enforce output schemas on any local LLM at the token level

GitHub · Language: Python · License: MIT

What it does

LM Format Enforcer intercepts the token logits of a locally-running LLM and masks any token that would violate your schema — before it’s sampled. The result is structurally valid JSON, regex-matched strings, or custom grammars, guaranteed, without post-processing retries. It targets developers running open-weight models (Llama 4, Qwen3.6, Kimi K3, etc.) through HuggingFace Transformers, vLLM-compatible runtimes, or llama-cpp-python.

Why it stands out

• Token-level enforcement, not retry loops — rather than generating and validating after the fact, it zeroes out logit probabilities for invalid continuations at every decoding step, making schema violations structurally impossible
• Pydantic-native — pass a BaseModel subclass directly; the library derives the character-level automaton from your schema automatically, no DSL to learn
• Broad backend support — integrates with HuggingFace pipeline, transformers LogitsProcessor, and llama-cpp-python’s LogitsProcessorList with near-identical APIs across backends
• Regex and custom grammars too — beyond JSON Schema, you can enforce arbitrary regular expressions or write a custom CharacterLevelParser, making it useful for structured non-JSON outputs like dates, codes, or templated strings

Quick start

from pydantic import BaseModel
from lmformatenforcer import JsonSchemaParser
from lmformatenforcer.integrations.transformers import (
    build_transformers_prefix_allowed_tokens_fn,
)
from transformers import pipeline

class MovieReview(BaseModel):
    title: str
    rating: int
    summary: str

pipe = pipeline("text-generation", model="your-local-model")
parser = JsonSchemaParser(MovieReview.schema())
prefix_fn = build_transformers_prefix_allowed_tokens_fn(pipe.tokenizer, parser)

result = pipe(
    "Review the movie Inception in JSON:",
    prefix_allowed_tokens_fn=prefix_fn,
    max_new_tokens=200,
)
print(result[0]["generated_text"])
# Always valid JSON matching MovieReview — no retries needed

When to use it

• Local/self-hosted inference — when you’re running open-weight models and can’t rely on an API provider’s native structured output endpoint
• Strict schema compliance — applications where a single malformed response is unacceptable (pipelines that parse output programmatically with no fallback)
• Regex-constrained generation — extracting fixed-format strings (ISBNs, phone numbers, dates) where a JSON schema would be overkill

When to skip it

• If you’re calling hosted APIs (GPT-5.6 Sol, Claude Opus 5, Gemini 3.6 Flash), those providers offer native structured outputs server-side — adding this layer adds complexity for no benefit
• The logits-masking approach adds per-token CPU overhead; at very high throughput on large models, dedicated constrained-decoding inference servers (like those with Outlines baked in) may be more efficient

The verdict

LM Format Enforcer is the cleanest solution for guaranteed structured output when you control the inference stack. The Pydantic integration is genuinely seamless, and the character-level automaton approach is the right abstraction — it’s schema enforcement all the way down, not wishful prompting. If your team is serving open-weight models and tired of writing retry-and-parse wrappers, this belongs in your stack.