Open Source AI

The Open Source AI Ecosystem

Open source AI has transformed from a distant follower of proprietary models into a genuine competitive force. In 2026, several labs release their flagship or near-flagship models as open weights, and in many practical applications, the gap to proprietary models is negligible. For organizations that need data sovereignty, cost control, customization, or low-latency inference, open source models are not just viable — they are often the best choice.

This module covers the leading open source model families, how to run models locally, the trade-offs between self-hosting and APIs, and how to evaluate what makes sense for your use case.

The Major Open-Weight Model Families

Meta Llama 4

Llama 4, released in April 2025, was one of the most significant open-weight releases of its time. It introduced two models that redefined what open models could achieve:

• Llama 4 Scout: A Mixture of Experts model with 109B total parameters but only 17B active per token (16 experts, routing to a subset per token). Its headline feature is a 10 million token context window, which Meta called industry-leading at launch. This makes it well suited for tasks requiring massive context: entire codebases, long document collections, or extended conversation histories.
• Llama 4 Maverick: A larger MoE model with approximately 400B total parameters and 17B active, using 128 routed experts plus a shared expert. Maverick targets frontier-level quality on reasoning, coding, and multilingual tasks, while remaining efficient enough to serve at reasonable cost.

Meta describes Llama 4 as the first open-weight natively multimodal model family: both Scout and Maverick process images and text jointly from pretraining, not through bolted-on vision adapters. This is a significant architectural advantage. Meta's newer open-weight models use the Muse name; Muse Spark Open, released on 15 October 2026, ships under Meta's custom Muse Model License.

Mistral AI

The Paris-based lab has established itself as a leading open-weight provider with a portfolio of models across the capability spectrum:

Qwen (Alibaba Cloud)

Alibaba's Qwen family (Qwen3.8 as of October 2026) has been particularly notable for consistently delivering strong performance across a wide range of sizes. Qwen models are popular for fine-tuning and deployment in production systems, with a broad size range from small efficient models to large capable ones. Licences vary by model: Qwen3.8-27B is Apache 2.0, while the largest Qwen3.8 model uses a custom licence. Their multilingual capabilities, particularly for Chinese and other Asian languages, are a recognised strength.

Running Models Locally

One of the most powerful aspects of open-weight models is the ability to run them on your own hardware — for privacy, cost control, offline access, or just the satisfaction of self-sufficiency.

Ollama

Ollama is the easiest way to get started with local models. It provides a simple command-line interface for downloading and running models, handling all the complexity of model loading, quantization, and GPU management behind a clean API. If you can install a program and type a command, you can run a local LLM with Ollama.

vLLM

For production deployments, vLLM is the standard inference engine. It implements PagedAttention and continuous batching to maximize throughput, achieving 2-4x higher throughput than naive implementations. vLLM supports tensor parallelism across multiple GPUs, making it the go-to choice for serving large models at scale.

Quantization: Making Models Smaller

Full-precision models are enormous. A 70B parameter model in FP16 (16-bit floating point) requires approximately 140 GB of memory — far more than most consumer GPUs. Quantization reduces the precision of model weights, dramatically shrinking memory requirements at the cost of some quality.

GGUF deserves special mention. Developed by the llama.cpp community, GGUF is a quantization format specifically designed for efficient CPU and mixed CPU/GPU inference. It uses mixed-precision quantization — keeping more important layers at higher precision while aggressively quantizing less critical ones. This makes it the best format for running models on consumer hardware with limited GPU memory.

Hardware Requirements

Understanding hardware requirements helps you plan what models you can realistically run:

When to Self-Host vs. Use APIs

The decision to self-host models or use proprietary APIs depends on your specific requirements:

The Open Source vs. Proprietary Debate

The capability gap between open and proprietary models continues to narrow. The pattern is consistent: proprietary labs push the frontier, and open-weight models catch up within months. In 2026, Moonshot, Z.ai, and Alibaba each released a flagship or top-tier model as open weights.

This narrowing gap has important implications:

• For most applications, open models are sufficient. If you don't need the absolute cutting edge, an open model likely meets your quality requirements.
• The proprietary advantage is in the last mile. Where proprietary models maintain an edge is on the hardest reasoning tasks, the newest capabilities, and the most polished user experiences.
• Hybrid strategies work best. Many production systems use open models for the majority of requests (cost efficiency) and route complex queries to proprietary APIs (capability).

Resources