Mistral Medium 3.5 Released: What It Means for Self Hosted AI

Published April 29, 2026

Mistral AI released Medium 3.5 today. It’s a 128 billion parameter dense model with open weights, a 256k context window, and the ability to self host on as few as four GPUs. For teams running AI on their own infrastructure, this is a big deal. A flagship class model that’s actually deployable outside hyperscaler data centers.

Here’s what the release means, what hardware you need to run it, and where it fits with everything else out there.

Mistral Medium 3.5: Quick Specs

Source: Mistral AI official announcement

Why This Release Matters

For the past two years, the highest performing open weight models have needed eight or more high end GPUs to self host at full precision. That hardware bar pushed teams toward cloud APIs whether they wanted to be there or not, especially for production workloads.

Mistral Medium 3.5 changes that. A 128B dense model running on four GPUs is a multi GPU server build, not a hyperscaler only deployment. Teams running their own infrastructure can now host a model that scores 77.6% on SWE Bench Verified, ahead of bigger competitors like Qwen3.5 397B A17B.

What this means in practice:

• Teams worried about data privacy can keep AI workloads on premise
• Production inference costs become a hardware decision instead of a per token expense
• Fine tuning and customization stay fully under your control
• Latency drops because requests don’t leave the local network

Hardware Requirements: What You Actually Need

Mistral says Medium 3.5 can self host on “as few as four GPUs,” but they don’t say which GPUs in the announcement. Based on the model size and standard precision options, here’s what that practically means.

At FP8 Precision (Recommended for Production Inference)

A 128B dense model at FP8 needs around 128GB of VRAM for the weights, plus more headroom for KV cache and context. Four GPUs at 80GB each (320GB total) gives you room for production deployment with reasonable batch sizes and the full 256k context.

Suitable configurations:

• 4x NVIDIA H100 80GB
• 4x NVIDIA H200 141GB
• 4x NVIDIA RTX PRO 6000 Blackwell 96GB

At BF16 Precision (Higher Accuracy, More VRAM)

At BF16, the model needs roughly 256GB of VRAM for the weights alone. That usually means 4x H200 (141GB each, 564GB total) or moving to a bigger configuration.

Server Considerations Beyond GPUs

Running Medium 3.5 in production isn’t just about GPU count. The full stack matters:

• CPU: Multi core server CPU (Intel Xeon, AMD EPYC, or Threadripper PRO) for data preprocessing, request handling, and tool orchestration
• RAM: 256GB or more system memory for large context handling and concurrent users
• Storage: NVMe SSDs for model loading, KV cache offloading, and dataset access
• Networking: 10GbE minimum if the model is serving multiple users or feeding downstream systems
• Power and Cooling: 4x H100 or H200 systems can pull 2,500W+ under sustained load and need proper thermal design

Mistral Medium 3.5 vs Other Open Weight Models

Medium 3.5 sits in an interesting middle ground. Smaller than Large 3, but higher performing on coding benchmarks (77.6% on SWE Bench Verified ahead of competitors).

What’s New in This Release

The Medium 3.5 announcement included more than just model weights:

• Mistral Vibe Remote Agents: Coding agents that run async in the cloud, can be spawned from CLI or Le Chat, and produce finished pull requests on GitHub.
• Le Chat Work Mode: A new agentic mode for cross tool workflows including email triage, research synthesis, and multi step project execution.
• Configurable Reasoning: Reasoning effort can be set per request, so the same model handles both quick replies and extended chain of thought tasks without separate deployments.
• Vision Support: The model includes a vision encoder trained from scratch to handle variable image sizes and aspect ratios.

How the Vibe Agentic Runtime Connects

Diagram concept based on Mistral AI’s announcement.

Frequently Asked Questions

What This Means for Teams Running On Prem AI

Medium 3.5’s hardware requirements put it in reach of organizations that have invested in real AI infrastructure but aren’t operating at hyperscaler scale. A four GPU server build is achievable for:

• Research labs running internal tooling and experiments
• Engineering teams deploying AI assisted coding agents
• Companies with data privacy requirements that prevent cloud deployment
• Studios and creative teams running AI workflows alongside traditional production

The math has shifted. A 128B parameter model with 256k context and competitive coding performance is now deployable on a single multi GPU server.

Source: Mistral AI’s official announcement, “Remote agents in Vibe. Powered by Mistral Medium 3.5.” Published April 29, 2026. Read the original announcement.