Mixtral 8x22B Instruct v0.1

Permissive

Mistral AI · Mistral · Released 2024-04

8x22B (141B total, ~39B active)Text

A large permissively-licensed MoE that competes with dense 70B+ models while activating a fraction of its total parameters per token.

Strengths

  • +Apache 2.0 with no usage restrictions
  • +Strong math and coding scores for an open MoE
  • +64K context window

Limitations

  • -141B total parameters require multi-GPU or multi-node serving
  • -Fewer community fine-tunes than the smaller Mixtral

License

Apache 2.0

use commercially with attribution niceties

Hardware

needs multi-GPU or datacenter hardware to run well

Links

WeightsGitHub

Stats

mistralai/Mixtral-8x22B-Instruct-v0.1— downloads·— likes

via Hugging Face

Related models

Mistral 7B Instruct v0.3

Mistral AI · Mistral

Mixtral 8x7B Instruct v0.1

Mistral AI · Mistral

Mistral Small 24B Instruct 2501

Mistral AI · Mistral

Mistral Nemo 12B Instruct 2407

Mistral AI / NVIDIA · Mistral