This Week In AI Research (🗓️ 9-16 July 26)
The top 10 AI research papers and releases this week (Kimi K3, Inkling, WanSong v1.0, Bonsai 27B, and many more)
1. Kimi K3
Moonshot AI released Kimi K3, the world's first open 3T-parameter class model, designed for frontier intelligence across long-horizon coding, knowledge work with native vision, and reasoning.
The model has 2.8T parameters and comes with a 1M-token context window. It uses Kimi Delta Attention, Attention Residuals, and a highly sparse MoE architecture (Stable LatentMoE) that activates 16 of 896 experts, resulting in 2.5× better scaling efficiency than Kimi K2.
While its overall performance still remains behind the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol, Kimi K3 shows frontier-level performance across diverse benchmarks, consistently outperforming other tested models.
Some limitations of the model are that it performs less reliably when earlier reasoning steps or important context are missing. It also tends to act too proactively when user instructions are unclear, making unexpected decisions on the user's behalf.
Read the technical blog using this link.
Before we move forward, I want to introduce you to my book, ‘LLMs In 100 Images’.
It is a collection of 100 easy-to-follow visuals that describe the most important concepts you need to master to understand LLMs today.
I’m offering a limited-time 30% discount, which you can claim using the button below.
2.xHC: Expanded Hyper-Connections
Hyper-Connections (HC) expand the residual stream of Transformers into N multiple parallel streams, which enables memory scaling beyond model width and depth, while Manifold-Constrained HC (mHC) stabilizes this approach at scale.
However, scaling mHC beyond N=4 leads to diminishing performance gains and rapidly increasing training cost because very little information is written back across the growing number of streams, while residual-mixing generation scales cubically with N.
This research paper addresses this by introducing xHC (Expanded Hyper-Connections), the first HC-family method to achieve meaningful expansion beyond N=4. xHC does this by maintaining 16 readable streams, providing dense access to the full residual state, while updating only 4 streams at each layer.
xHC improves performance across knowledge, reasoning, coding, and Chinese-language benchmarks across MoE models. To reach the same training loss, standard Transformers and mHC require 1.50× and 1.19× as much compute as xHC, respectively.
Read more about this research paper using this link.
3. VideoChat3
VideoChat3 is a fully open (model weights, training code, training strategy, and complete training datasets), 4B-parameter multimodal model with generalist video-centric capabilities.
VideoChat3 uses the Inflated 3D Vision Transformer (I3D-ViT) and adaptive frame resolution to efficiently compress video while preserving spatiotemporal information, and reduces the cost of processing video inputs during training and inference.
Trained on roughly 3 million instruction samples, VideoChat3 achieves a rare balance of broad generalization and computational efficiency, outperforming prior open-source models with equal or larger parameter counts while using only 4B parameters.
Read more about this research using this link.
4. Inkling
Thinking Machines Lab released Inkling, an open-weight model for reasoning, coding, tool use, and multimodal tasks. It uses the Mixture-of-Experts architecture with 975B total parameters and 41B active parameters, and supports a context window of up to 1M tokens.
Alongside this, the team released a preview of Inkling-Small, a lighter-weight model with 12B active parameters, trained with a similar recipe, that achieves strong performance with even lower cost and latency.
Inkling comes with adjustable reasoning effort and can be fine-tuned through the Tinker platform.
While it is not the strongest overall model available today (open or closed), its diverse capabilities make it a good open-weights base for customization.
Read the technical blog using this link.
5. AIDE²: The First Evidence of Recursive Self-Improvement
AIDE2 is a recursive self-improvement (RSI) system that uses two autoresearch loops:
An inner loop that functions as a normal autoresearch agent, optimizing code against an eval.
An outer loop that optimizes the inner-loop agent’s harness code.
Over 100 autonomous outer-loop iterations running for eight days, AIDE2:
Discovered 7 successive improved versions of AIDE
Developed a new search algorithm
Reduced the prompt size by 16×
Outperformed a manually tuned agent that had been iterated on for two years, on several unseen tasks under a fixed compute budget
Reduced reward hacking on a held-out GPU benchmark from 63% to 34%
The team describes this as “Level 1” recursive self-improvement because the system improved itself more efficiently than human researchers. However, it did not reach “Level 2” or “Ignition,” because the improved agent was not conclusively better at running the outer self-improvement loop, and its evolved code became complex and difficult to maintain.
Read the technical blog using this link.
6. WanSong v1.0
WanSong is Alibaba’s pure diffusion-based model for generating commercial-grade, multilingual songs up to 5 minutes long. Trained on more than six million hours of audio, the model produces complete audio in a single run and separately outputs vocals and background music.
On a multilingual and multi-genre benchmark, WanSong achieved the lowest reported pronunciation error and the highest musicality score compared with LeVo, Mureka V7.6, and Suno V5.
Read more about this research using this link.
7. Bonsai 27B
Bonsai 27B is PrismML's new multimodal model based on Qwen3.6 27B, capable of multi-step reasoning, structured tool calls, vision tasks, and agentic loops for computer use.
Models of this size typically require 54GB of memory at 16-bit precision (27B × 2 bytes per parameter). This far exceeds the memory of any typical phone.
Bonsai 27B changes this with its two variants:
Ternary Bonsai 27B: Uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, leading to 1.71 effective bits per weight. This totals 5.9 GB of memory, which makes it suitable for running on an everyday laptop.
1-bit Bonsai 27B uses binary {−1, +1} weights with FP16 group-wise scaling, leading to 1.125 effective bits per weight. This totals 3.9 GB, which means it fits and runs comfortably on an iPhone 17 Pro Max.
Ternary Bonsai 27B achieves 95% of the full-precision baseline, and 1-bit Bonsai 27B achieves 90% across 15 benchmarks in knowledge, reasoning, math, coding, instruction following, tool calling, and vision.
Read the technical blog using this link.
8. Mach-Mind-4-Flash
This research presents Mach-Mind-4-Flash, a 35B-parameter Mixture-of-Experts (MoE) agentic model (3B activated) that, through post-training optimization alone, achieves performance on par with or surpassing that of 100B-parameter-class models.
Its three-stage training pipeline combines:
A unified RL and on-policy distillation infrastructure, with dynamic multi-teacher scheduling and operator-level acceleration, which gives it a 17% end-to-end training speedup
Multiple domain-specific RL experts trained in parallel across Reasoning, General, and Agent tracks, then combined into a single generalist using Multi-Teacher On-Policy Distillation (MOPD)
Hybrid Median-length Policy Optimization (HMPO), which compresses reasoning chains by 19–46% with not more than a 0.7-point loss in accuracy.
Mach-Mind-4-Flash scores competitively matching or outperforming models with 10-30× its activated size while using substantially less inference compute.
Read more about the research using this link.
9. Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values
This research identifies a type of misalignment in LLMs called Covert value leakage. This is when the information provided by an LLM is influenced by its own values, and this influence is never disclosed to the user.
For example, when a user asks whether they are considering investing in an AI company and wants to know how likely the AI bubble is to pop, Claude Opus 4.8 gives a lower probability when the company under consideration is Anthropic rather than OpenAI, without disclosing this influence to the user.
The research shows that models are influenced by different types of values, including:
Preferences for morally good outcomes
Preferences for the company that developed them
Preferences for some human leisure activities over others
Models differed in how openly they revealed these biases. For example, Claude models falsely claim to provide unbiased answers during chain-of-thought reasoning, whereas Qwen models explain how their values bias their answers.
Read more about this research using this link.
10. On-Policy Delta Distillation
On-policy distillation (OPD) is an RL post-training method that trains a student model based on its own responses using token-level feedback from a stronger teacher model. This encourages the student model to generate tokens that the teacher prefers.
This research paper improves OPD by introducing a new distillation reward, called the ‘delta signal’, instead of the teacher’s output distribution.
The delta signal is the difference between the teacher model and its base model prior to post-training for reasoning. This helps capture and transfer the reasoning skills gained during the teacher model’s post-training rather than copying all of its knowledge and style. This new approach is termed On-Policy Delta Distillation (OPD2).
Experiments across mathematics, science, and code-reasoning benchmarks show that OPD2 consistently outperforms conventional on-policy distillation. However, it requires access to both the teacher and its base model and increases training time compared with standard OPD.
Read more about this research using this link.
This newsletter edition is completely free to read. Show your love by liking it, restacking it, and sharing it with others! ❤️
Join the paid tier today to get access to all posts in this newsletter, including:
and so many more!