Where randomness meets reason
Tag
31 posts
SafeStart catches a real but overlooked safety regression: MoE expert deactivation for speed silently degrades refusal behavior, with a cheap fix.
Silent and Costly: LLM agents silently waste money through mis-integrated prompt caching — this paper shows how to detect and fix the bug class.
Fine-tuned small models for RAG query clarification beat frontier-class systems by learning when to ask, what to ask, and when to stay silent.
Which MCP tool metadata actually drives model selection — name, description, or parameter schema — measured across 367K real tools.
Coding agents trust rule files blindly — this paper shows attackers can exploit that to trigger package hallucination and slip malicious dependencies into generated code.
Practitioners building multi-objective agentic workflows can now route at inference time across accuracy, cost, latency, and robustness without retraining separate generators.
4-bit quantization choice leaks fine-tuned PII at different rates — GPTQ-style methods outperform GGUF on privacy, independent of bit width.
KV-cache repair after edits: a new paper shows patching only contiguous changed blocks beats importance-scoring by 2–3× on latency with near-zero quality loss.
Token-level SFT masking on math reasoning: a small-team paper showing which tokens actually hurt training, with released analysis tools.
KV cache memory hogs in hybrid LLMs traced to per-head residency imbalance — a new budget allocator cuts waste without accuracy loss.
Coding agents using function-level execution signals for process supervision—patch-level test feedback outperforms line and trace approaches by 8+ points on HumanEval variants.
Looped LLMs cut tool-calling errors by maintaining hidden state across API calls — a quiet fix for multi-step agent reliability that costs zero extra parameters.
Enterprise RAG hits an 80%→27% accuracy cliff when all constraints must hold simultaneously — here's the benchmark that quantifies it.
Self-repair in code agents adds feedback overhead but rarely beats simply sampling again — a controlled study quantifies exactly when the extra tokens are wasted.
Structured output silently collapses answer diversity across 44 LLMs — JSON-only prompts skew which answer a model picks, not just how it formats the response.
AI agent tool sets that ossify at deployment lose compounding value — this paper shows how to grow them automatically from task execution logs.
SWE-Router cuts LLM agent costs by routing easy GitHub issues to cheaper models mid-trajectory, not just at task start, using execution signals as routing features.
Quantization inflates reasoning model token counts by up to 4×, a hidden cost accuracy benchmarks miss entirely.
Routing accuracy for 110+ agents drops sharply past 50 tools — new study maps the degradation curve and identifies three recovery strategies that work today.
RAG corpus poisoning drops 60-80% when you add chunking+reranking — but most attack evals skip these standard pipeline stages entirely.
Activation probes detect credential exfiltration *before* the LLM outputs any tokens — combined with honeytokens and multi-turn leakage tracking, with no model changes required.
76 malicious skills confirmed in 3,984 audited AI agent marketplaces — credential theft, backdoor installation, and data exfiltration found hiding in plain sight.
Code cleanliness measurably changes how well coding agents complete tasks — a controlled minimal-pair study with released dataset and reproducible methodology.
LLM agent memory degrades when consolidated: episodic traces outperform summarized lessons across 5 agentic tasks.
TSCG shows small LLMs (4–14B) drop tool-call failures by compiling JSON schemas into natural-language descriptions before inference.
Rewriting tool descriptions at deployment time—not training time—can recover 20-40% of function-calling accuracy lost to poorly written API docs.
Visualizing LLM output distributions reveals hidden modes, edge cases, and prompt sensitivity that single-sample evaluation completely misses.
Lossless prompt compression via dictionary encoding lets LLMs analyze repeated data at a fraction of token cost — no external tools, just in-context learning.
Structured state-space models finally beat transformers at document retrieval — here's what the Mamba-based RAG benchmark actually shows.
KV cache compression that cuts memory 40–60% with under 1% accuracy loss — here's the technique your inference stack probably isn't using yet.
SCoRe trains a single LLM to catch and fix its own mistakes via RL — 15.6% better on math, 9.1% on code, no multi-model pipeline needed.