Model Releases

  • prism-ml/Bonsai-8B-gguf 🤖 8B model pushed down to 1-bit quantization via GGUF, specifically targeting llama.cpp compatibility with CUDA/Metal hooks. Useful if you’re trying to run inference on constrained edge hardware, though you’ll want to benchmark quality drops carefully before trusting it with anything critical.
  • Hcompany/Holo3-35B-A3B 🤖 MoE vision-language fine-tune built on Qwen3.5, optimized for GUI automation and screen-understanding tasks. If your team is moving past brittle XPath selectors toward actual computer-use agents, this one actually tracks interface state instead of hallucinating button coordinates.

Open Source Releases

  • NVIDIA/Model-Optimizer 🛠️ Unified compression library bundling quantization, pruning, distillation, and speculative decoding with TensorRT-LLM/vLLM integrations. Standardizes the usual optimization grind, though it pretty firmly anchors you to NVIDIA’s serving ecosystem.
  • LMCache/LMCache 🛠️ KV cache acceleration layer focused on memory reuse to cut recomputation overhead for long-context and multi-turn chats. If your inference bills are bleeding out on redundant attention passes, this plugs the leak without forcing a full rewrite of your serving config.
  • allenai/OLMo-core 🛠️ PyTorch backbone for the OLMo ecosystem, providing modular training loops and data utilities for open-weight LLMs. Good if you want reproducible training from scratch without stitching together a dozen fragmented research repos.
  • Claude Code v2.1.90 Release 🛠️ Adds /powerup for interactive feature walkthroughs and an env var to keep the marketplace cache alive in offline setups. Also extends protected directory support to .husky, so your pre-commit hooks survive automated code edits.
  • cellcog 1.16.0 🛠️ Python SDK for building agents with any-to-any communication models, using fire-and-forget async WebSocket notifications for state tracking. Handy if you prefer decoupled agent messaging over synchronous RPC, assuming you’re ready to debug async state drift.
  • pocketteam 1.0.4 🛠️ Multi-agent library for autonomous IT ops with modular skills and safety guardrails, wired into GitHub Actions for self-healing pipelines. Looks like a way to automate incident response without accidentally granting root access to production on a bad Tuesday.

Research Worth Reading

  • Decision-Centric Design for LLM Systems 📄 Argues for splitting control logic from content generation to improve inspectability and constraint enforcement. A solid architectural reminder that treating LLMs as deterministic routers instead of monolithic text generators saves you from debugging production prompt spaghetti.
  • Self-Routing: Parameter-Free Expert Routing from Hidden States 📄 Drops the learned router in MoE layers by deriving expert assignments directly from hidden states. Cuts parameter overhead and training complexity while keeping routing competitive, which should make MoE deployment cheaper for teams tired of routing bottlenecks.
  • Two-Stage Optimizer-Aware Online Data Selection for Large Language Models 📄 Framework for streaming data selection during fine-tuning that factors in step-dependent sample utility and optimizer geometry. Moves beyond static offline filtering to actually match gradient updates, useful if your fine-tuning runs are drowning in low-signal tokens.
  • ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving 📄 Open-source multi-model router that uses bandit algorithms to balance cost and quality under drifting conditions like price hikes or model regressions. If you’re running production API routing and tired of manually swapping providers when latency spikes, this automates the triage.
  • Open, Reliable, and Collective: A Community-Driven Framework for Tool-Using AI Agents 📄 Introduces OpenTools, separating tool invocation accuracy from actual tool correctness to fix reliability bottlenecks in agentic loops. Offers community validation patterns for tool use, directly addressing the classic “it called the API correctly, but the API returned garbage” problem.
  • Signals: Trajectory Sampling and Triage for Agentic Interactions 📄 Methodology for efficiently sampling and categorizing non-deterministic agent trajectories post-deployment. Gives you a structured way to spot failure modes without manually reading ten thousand JSON dumps of an agent arguing with itself.

AI Dev Tools

  • microsoft/agent-framework 🛠️ Official framework for orchestrating single and multi-agent workflows across Python and .NET, with standardized state and tool-calling primitives. Useful if your stack already lives in Azure/MS ecosystems, though it brings the usual enterprise abstraction tax.

Today’s Synthesis

If you’re still routing production workloads by chaining prompts and hoping for deterministic behavior, it’s time to decouple control flow from content generation. The Decision-Centric Design for LLM Systems paper nails why treating LLMs as monolithic text generators creates unmaintainable spaghetti, while Self-Routing: Parameter-Free Expert Routing from Hidden States shows how to skip learned routing layers entirely and derive expert assignments directly from model internals. Pair that architectural shift with ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving , and you’ve got a practical blueprint for stripping out prompt overhead, routing cheaply at the hidden-state level, and dynamically swapping providers before latency or price spikes bite. Stop asking your models to parse JSON and manage application state inside a single forward pass. Extract routing decisions into a lightweight classifier or explicit state machine, reserve heavy generation tokens for actual reasoning, and let online bandit algorithms handle the provider juggling. Refactor your agent orchestration this quarter, and your inference bills, deployment configs, and on-call pager will all breathe easier.