PrismML Bonsai 2 27B: What It Is and How to Run It Locally

A 27-billion-parameter language model normally needs around 54 GB of memory at full precision. In practice that means a workstation GPU, a rented cloud server or a very expensive Mac. Bonsai 2 27B, which PrismML released on September 17, 2026, fits the same model into 5.9 GB and keeps 98.2% of its average benchmark score (PrismML, 2026). A decent gaming PC or a recent MacBook can run that.

When we wrote about 1-bit models and the compression stack for local AI in May, the biggest Bonsai model had 8 billion parameters. Now there's a 27B model with image input, tool calling and a 262K-token context window. Below: what the benchmarks do and don't tell you, and how to install it on Windows, Mac or Linux without writing code.

Key Takeaways

Bonsai 2 27B is a compressed version of Alibaba's open-source Qwen3.8 27B. It needs 5.9 GB instead of about 54 GB and keeps 98.2% of the original's benchmark average (PrismML, 2026).

A standard 2-bit build of the same model takes 9.4 GB and keeps only 84.1%. Bonsai 2 is smaller and scores more than 12 points higher.

PrismML measured about 81 tokens per second on an RTX 4090, 130 on an RTX 5090 and 28 on an Apple M5 Pro laptop.

Ollama and LM Studio can't run it yet. PrismML's own setup scripts can, on all three operating systems.

What Is Bonsai 2 27B?

Bonsai 2 27B is PrismML's second-generation 27B model, built on Qwen3.8 27B from Alibaba (TechCrunch, 2026). PrismML doesn't train new models from scratch. It takes a strong open model, rebuilds its weights in a far smaller format and then works to recover the quality that compression usually destroys.

That format is called ternary. A normal model stores each of its billions of weights as a 16-bit number. In Bonsai 2, every weight is -1, 0 or +1, with one shared scale factor per group of 128 weights. Across the whole language model that averages out to about 1.7 bits per weight. PrismML applies it to the embeddings and output layer too, which many "2-bit" builds quietly keep at higher precision (Hugging Face model card).

The spec sheet in plain terms:

Size: 5.9 GB for the language model, plus an optional 0.63 GB file for image input

Context window: 262,144 tokens, enough for a long contract or a mid-sized codebase

Input: text and images, including photos, screenshots, charts and scanned pages

Reasoning: it's a "thinking" model that works through a problem before answering

Tool calling: native support for the function-calling format that AI agents use

License: Apache 2.0, which allows commercial use

PrismML came out of Caltech and is led by professor Babak Hassibi, a compression specialist. Its backers include Khosla Ventures, Cerberus and Google, with continuing support from Samsung (PrismML, 2026). The company's first 27B models, released in July, kept about 95% of the full model's score. Bonsai 2 moves to a newer base model and gets that to 98.2% at the same size.

How Good Is It? What the Benchmarks Show

Comparing Bonsai 2 with other models is less useful than comparing it with other ways to shrink the same model. PrismML's model card runs four versions of Qwen3.8 27B through the same 14 benchmarks in thinking mode:

Version of Qwen3.8 27B

Bits per weight

Size

Average score

vs. full model

Full precision (FP16)

16.0

54 GB

86.32

100%

Standard "4-bit" (UD-Q4_K_XL)

5.2

17.6 GB

85.18

98.7%

Standard "2-bit" (IQ2_XXS)

2.8

9.4 GB

72.59

84.1%

Bonsai 2 27B1.725.9 GB84.7898.2%

Source: Bonsai 2 27B model card, September 2026.

Bonsai 2 comes within half a point of the popular 4-bit build at a third of the size. The 2-bit row is the more interesting one. That build still scores 88.93 on MMLU-Redux, a general knowledge test, but collapses on long chains of reasoning: 57.5 on AIME 2026 math and 56.4 on LiveCodeBench coding. Bonsai 2 scores 95.83 and 90.07 on the same tests. That's why a quick trivia question can make a heavily compressed model look fine when it isn't.

The losses aren't spread evenly, and it's worth knowing where they fall:

Skill area

Full model

Bonsai 2 27B

Difference

Math

97.06

96.57

-0.49

Coding

89.07

89.42

+0.35

Instruction following

81.25

82.66

+1.41

Agentic tool calling

76.74

74.92

-1.82

Vision

71.36

66.19

-5.17

Knowledge and reasoning

85.55

79.86

-5.69

Source: Bonsai 2 27B model card.

Math and coding barely move. Knowledge-heavy questions and image understanding drop by five to six points each, so test the model on your own material first when your work depends on dense charts or obscure facts.

PrismML's launch post uses a slightly different benchmark suite. On that one, Bonsai 2 scores 83.9 overall, just above the full-size Qwen3.6 27B at 83.6, the previous generation of the same model family (PrismML, 2026). A 5.9 GB file edges out last generation's 54 GB model.

One caveat covers everything above: these are PrismML's own numbers. Independent tests take a few weeks to appear, so treat them as strong claims from a team with a good track record, not