AI8/10
Satya Nadella Outlines Microsoft's MAI Models and Cost-Efficient Routing
Satya Nadella's article argues that optimizing cost-to-outcome matters as software gains marginal cost, describing Microsoft's MAI model family. He says MAI models now outperform some frontier models on product tasks with fewer tokens and are being routed across GitHub Copilot, Excel and Outlook.
Original post · 3 min read
X ArticleFrontier Diffusion & Control
In a world where software has real marginal cost for the first time, how do we ensure frontier benefits are diffused across the entire ecosystem?
The key is to optimize the cost-to-outcome frontier in real world context. In practical terms, that means using the right model for each task, and optimizing the context, skills, tools, and agent harness around it.
This is the motivation behind our MAI model family. These models have been built ground up with clean data lineage and optimized for learning transfer from generalist to specialized skills in enterprise RLEs. We continue to make rapid progress in this pursuit.
We can now take saturated frontier capabilities and deliver them at scale and at lower cost through models optimized for high-usage products, while continuing to use frontier models for frontier needs. We are proving this out across our first party products, and thereby creating a template for every other AI native, SaaS, or Enterprise company out there.
In our products, frontier models from OpenAI and Anthropic are part of the orchestration system alongside MAI. But the model is only one part of the hill-climbing system. Harness, memory, context, tools, skills, user interactions, etc. all shape the evals and performance of these agentic systems.
The other key criteria to ensure that you are in control, is your evals should continue to hill climb even when any given model has been removed. Therefore we build RLEs where models learn inside the product system and are rewarded for completing the tasks customers actually care about. We train models against the actual product harness, interactions, and outcomes they will encounter. And strategically ensure that the harness, memory, context, skills are externalized outside of the model.
Product-specific evals and model independence give us the control and a direct hill to climb, and to keep refining until we reach the right quality-cost target. We are now seeing MAI models outperform general-purpose frontier models in many use cases while using a fraction of the tokens.
We believe the biggest opportunity is to optimize all of these layers together in the products where the world works every day. And we are beginning to route traffic across our first-party surfaces to MAI whenever our models match or outperform frontier alternatives.
We are seeing promising early results across GitHub Copilot, Excel, and Outlook and are beginning to take the same approach across Copilot Chat, PowerPoint, and more. And all these results will only get better as the entire system keeps hill-climbing!
What we are doing across our first party products is also what every enterprise customer can be doing in their real world agentic systems with their proprietary evals, their proprietary RLEs, workflows, and context. We are making all this available as part of Foundry and our toolchain.
Read more here: microsoft.ai/news/hill-climbing-mai-models-for…
The key is to optimize the cost-to-outcome frontier in real world context. In practical terms, that means using the right model for each task, and optimizing the context, skills, tools, and agent harness around it.
This is the motivation behind our MAI model family. These models have been built ground up with clean data lineage and optimized for learning transfer from generalist to specialized skills in enterprise RLEs. We continue to make rapid progress in this pursuit.
We can now take saturated frontier capabilities and deliver them at scale and at lower cost through models optimized for high-usage products, while continuing to use frontier models for frontier needs. We are proving this out across our first party products, and thereby creating a template for every other AI native, SaaS, or Enterprise company out there.
In our products, frontier models from OpenAI and Anthropic are part of the orchestration system alongside MAI. But the model is only one part of the hill-climbing system. Harness, memory, context, tools, skills, user interactions, etc. all shape the evals and performance of these agentic systems.
The other key criteria to ensure that you are in control, is your evals should continue to hill climb even when any given model has been removed. Therefore we build RLEs where models learn inside the product system and are rewarded for completing the tasks customers actually care about. We train models against the actual product harness, interactions, and outcomes they will encounter. And strategically ensure that the harness, memory, context, skills are externalized outside of the model.
Product-specific evals and model independence give us the control and a direct hill to climb, and to keep refining until we reach the right quality-cost target. We are now seeing MAI models outperform general-purpose frontier models in many use cases while using a fraction of the tokens.
We believe the biggest opportunity is to optimize all of these layers together in the products where the world works every day. And we are beginning to route traffic across our first-party surfaces to MAI whenever our models match or outperform frontier alternatives.
We are seeing promising early results across GitHub Copilot, Excel, and Outlook and are beginning to take the same approach across Copilot Chat, PowerPoint, and more. And all these results will only get better as the entire system keeps hill-climbing!
What we are doing across our first party products is also what every enterprise customer can be doing in their real world agentic systems with their proprietary evals, their proprietary RLEs, workflows, and context. We are making all this available as part of Foundry and our toolchain.
Read more here: microsoft.ai/news/hill-climbing-mai-models-for…
♥ 2.5K · ⟲ 396 · 👁 759.5KView on X ↗
