Decision in brief
Start with a smaller pinned checkpoint that matches your task, such as Ministral 3 8B for text or Devstral Small 2 for repository work. Use the hosted API as a quality or throughput reference when policy permits it. Consider larger weights after measuring the smallest deployment that can pass the task, concurrency and recovery checks.

Match Mistral weights and serving requirements to the task
Mistral's catalog lists Ministral 3 8B and Mistral Small 4 under Apache 2.0. Large 4 weights use the Mistral Ecosystem Open Weight License. Mistral says a separate commercial license applies to businesses that operate a Model-as-a-Service offering. Check the license for the exact checkpoint and intended use before deployment.
Model residency depends on total parameters, storage precision and serving overhead. Small 4 has 119 billion total parameters and 6.5 billion active parameters. An idealized four-bit representation of those weights alone is about 59.5 GB in decimal units before metadata, runtime allocation or KV cache. Devstral Small 2 has 24 billion dense parameters, giving a corresponding weights-only estimate of about 12 GB. Published model context limits do not establish how much context your machine can serve under load.
| Use case | Try first | Acceptance check |
|---|---|---|
| Private extraction or classification | Pinned Ministral 3 8B on a supported local runtime | Required fields and structured output pass on approved samples within memory and latency limits. |
| Repository assistance on a controlled host | Pinned Devstral Small 2 with its documented prompt format | A harmless test change passes repository checks with usable latency at your intended context. |
| Higher quality or mixed coding and reasoning workloads | Hosted baseline, then the pinned Mistral Small 4 checkpoint | Concurrent requests fit the machine after accounting for cache and practical response budgets. |
| Multi-tenant service on Large 4 | Managed offering or a measured cluster pilot | Operations and counsel confirm the specific serving terms. Process restart and load tests pass. |
Turn hosting anecdotes into checks
In the Large 4 launch discussion, nsbk said a different local model met their use needs but that they missed the speed and concurrency of hosted subscriptions. Use this as a prompt to benchmark burst throughput and queueing. It supplies no performance estimate for a Mistral checkpoint.
In the separate Large 4 weights thread, rcr-anti called attention to the separate license for offering the model as a service. Check the publisher's terms directly, then distinguish an internal deployment from paid third-party access before budgeting.
Run a bounded local deployment pilot
- Pick a model and pinned revision from its publisher. Write down intended users, license owner, runtime compatibility, data boundary and a budget for host memory and GPU memory.
- Run a small approved workload on the intended runtime. If allowed, run the same tasks on the hosted endpoint with comparable settings. Compare result quality and track queue time, tokens per second and peak memory.
- Repeat at the longest approved practical input and your planned concurrent load. Restart the local service on your own staging host and check that it comes back on the pinned model without fetching unapproved artifacts.
- Proceed when license scope is confirmed and the local result meets quality, privacy, latency and restart targets. Stop at out-of-memory, unexplained outbound access or missed hard performance targets. Reassess with shorter context, less concurrency or smaller weights.