Run a 744B AI model on your laptop with Colibri: what it needs and how fast it is

Short videos this week say you can run a 744B AI model on your laptop with a free tool called Colibri, with no GPU and no cloud. That part is true. To run an AI model on a laptop this size, Colibri keeps a small part of it in memory and reads the rest from your SSD while it answers.

What the videos leave out is the speed. Colibri's own benchmark page says a cold start on its maker's machine gives about 0.05 to 0.1 tokens a second. That is one token, roughly three quarters of a word, every 10 to 20 seconds.

How Colibri fits a giant model into a laptop

Big open models like GLM-5.2 are "mixture of experts" models. For each token only a small slice of the model does any work: Colibri's page says about 5.4% of GLM-5.2's parameters are active per token. Colibri keeps the parts that are always needed in RAM, puts the busiest experts on a GPU if you have one, and streams the rest from an NVMe SSD.

The project describes itself as "a hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe." It is written in C, it is open source under the Apache 2.0 license, and the README says "no GPU required." There is no account and no fee.

What your laptop needs for the 744B model

These numbers are from Colibri's GitHub page for GLM-5.2 (744B):

• a 429 GB download, so you need well over 400 GB of free space,
• 16 GB of RAM as the stated minimum and 24 GB recommended,
• a fast NVMe SSD, because the model reads experts from the disk on every token.

The benchmark page is stricter than the videos about memory. It says "16 GB is below the floor for the large models," so plan on 24 GB or more.

How fast it really runs

On the maker's own development machine (12 cores, 25 GB of RAM, an NVMe drive reading about 1 GB a second), the benchmark page lists "~0.05–0.1 tok/s cold." The page puts it plainly: "This is not fast."

It gets faster with a warm cache, pinned experts and a GPU. The same page reports 6.84 tokens a second on a machine with six RTX 5090 graphics cards, which is not a laptop. On a normal laptop, a one paragraph answer can take many minutes.

Is it as good as Claude Opus?

One viral clip put Colibri next to Claude Opus. Colibri's pages show no such comparison. The model also runs compressed to int4 to fit, and Colibri's own quality test found that compression cost about 8 points on standard benchmarks (40 questions each). We left the Opus claim out because nothing on the maker's pages supports it.

A smaller model to start with

If you want to try local AI on an ordinary laptop, start smaller. Colibri's list includes Qwen3.6 35B, a 23 GB download that needs about 10 GB of RAM (20 GB recommended).

1. Download Colibri from its GitHub page (Colibri).
2. Run the setup: `START-HERE.bat` on Windows, or `./start-here.sh` on Mac and Linux.
3. The setup checks your machine, lists the models that fit, and downloads the one you pick. Then chat with `./coli chat` or open the web page it starts on your computer.

FAQ

Can I run a 744B AI model on my laptop?

Yes, with Colibri, if you have about 430 GB of free SSD space and 24 GB of RAM. Its own page measured about 0.05 to 0.1 tokens a second on a cold start.

Does Colibri need a GPU?

No. Its README says "no GPU required." A GPU makes it faster because the busiest experts can live there.

Is Colibri free?

Yes. It is open source under the Apache 2.0 license, with no account or subscription.

What is the easiest model to try first?

A smaller one, like Qwen3.6 35B: a 23 GB download and about 10 GB of RAM.

Sources

• Colibri on GitHub (README)
• Colibri benchmarks