Janas runs LLMs larger than RAM by streaming experts from SSD: how it works and what it needs

Can you run a language model that's bigger than your computer's memory? A new open-source project called Janas says yes, for one type of open model: mixture-of-experts (MoE) models. Janas runs LLMs larger than RAM by streaming experts from SSD, reading from disk only the parts of the model each token needs. Its developer says an 80-billion-parameter model "runs, and answers, on a laptop with 32 GB of RAM." On October 12, 2026, a Hacker News post titled "Janas: Run LLMs larger than RAM by streaming experts from SSD" drew 220 points and 94 comments.

This story is based on the project's GitHub repository, including its README, feature and hardware documentation, changelog and license files, checked October 16, 2026. All speed figures are the developer's own.

The short version

• What it is: Janas-LLM, a C inference engine with no dependencies beyond the C library. It ships with a terminal chat, an HTTP server that follows OpenAI's API, an MCP server and a C library.
• Who made it: Maurizio "camauri" Cammalleri, a system architect based in Italy. It is part of his "Prabanta" family of tools. The project started on September 18, 2026.
• License: GPL-3.0-or-later. Converted model files keep the license of the original model.
• The claim: Qwen3-Next-80B-A3B, a 48.4 GB file, generates 24 tokens per second on a 32 GB laptop, or 37 with draft tokens, according to the developer.
• The caveat: the author calls it "an experiment, not a product." It was built and measured on one machine.

How does streaming experts from SSD work?

A mixture-of-experts model splits most of its layers into many small sub-networks called experts. A router picks a handful of them for each token. Qwen3-Next-80B-A3B has about 80 billion parameters in total, but only about 3 billion are active for any one token. As our guide to laptop models explains, standard local runtimes still need all the weights in memory, even though each token uses only a few experts.

Janas uses the gap between total and active parameters. According to its README and feature notes:

• The weights stay on disk. The experts each token needs are read from an NVMe SSD with direct I/O (O_DIRECT), which bypasses the operating system's page cache.
• A cache keeps the frequent ones. The experts that "keep coming back" stay in RAM. The cache fills whatever memory is free when the model opens. In the developer's 80B test, it held about 19 GB of the most-used experts.
• A custom file layout. Models are converted from GGUF into Janas's own .jns format, with "one slot per (layer, expert)" and experts "laid out by co-activation" so that reads land near each other. The conversion reorders the weights without retraining or re-quantizing them. The README says converted models are available from the project's Hugging Face page, which listed 15 .jns repositories on October 16, while MODELS.md still says the project publishes "the recipe" for converting a model yourself, "not forty gigabytes."
• Read-ahead. When the cache is small, the engine runs the next layer's router early and loads the experts it will need while the current layer computes.
• Skipping light experts. When the cache holds less than a fifth of a model's experts, Janas skips experts that the router weights only lightly and that would need a disk read. The other experts the token picked take up their weight. The developer credits this "cache-conditional routing" to Skliar et al. (TMLR 2025). Unlike the other techniques, this one changes the output. In the developer's 400-token comparison of Qwen3-Next-80B against llama.cpp with a 4 GiB cache, the most likely token matched at 384 positions with skipping on and 393 with all experts computed.

Streaming weights from flash isn't a new idea. Apple researchers described a related approach in "LLM in a flash" (December 2023), and llama.cpp can memory-map weights from disk. What Janas adds is a cache and file layout designed around MoE experts, and an engine that tunes itself by measuring the machine it runs on.

What hardware does Janas need?

Last verified: October 16, 2026

The developer is asking for test reports from AMD CPUs, CPUs without AVX-VNNI, discrete GPUs and machines with 16 GB or less. A tester's Radeon 780M report is in the docs. On a Quadro RTX 5000, a discrete-GPU bug corrupted output until it was fixed on October 13.

How fast is it? (developer-reported)

The README's comparison was run by the developer on the laptop above between October 9 and 12, 2026. It used janas-bench --prompt 1024 --gen 16 and llama.cpp build 2b18470 with its Vulkan backend.

What these numbers mean, and their limits:

• One machine. The README warns that "on another machine the figures will be other ones."
• A workaround setup for llama.cpp. On the 80B model, llama.cpp ran out of memory with its normal build. The developer rebuilt it without weight repacking and ran it with memory-mapped weights. The 2.8 tokens per second is from the CPU alone in that setup (2.0 with the GPU, per the README). The README describes the llama.cpp figures overall as "llama.cpp's best configuration" on that laptop, chosen by the Janas developer.
• Different measurement methods. The README notes that llama-bench generates without a preceding prompt, while janas-bench generates after the 1,024-token prompt. The 80B Janas figures are from a single round.
• Drafts depend on the text. The 37 tokens per second "with drafts" uses speculative decoding. The README says drafts were accepted 74–89% of the time on an English chat prompt and gained "next to nothing" on Italian prose with one small model.
• Results on a busy machine. With a browser and editor open, the developer says the 80B model kept "about three quarters of its speed."

What are its limits?

The README's "Read this first" section is blunt:

• The interface keeps changing. The command line and API "change without notice, and have changed several times a week." The .jns file format (version 2) is promised to stay readable.
• It can strain the machine. The engine "fills the free memory with its expert cache and reads the disk at full speed." It can push a desktop into swap, and a GPU driver reset "once" took the desktop down. The --cache <GiB> option caps memory use, and JANAS_GPU=0 keeps it off the GPU.
• Long prompts are the weak spot. Hacker News commenters questioned whether disk streaming holds up for agentic work with long prompts, where prompt reading dominates. Some also raised SSD wear and suggested the code was largely AI-written. The project doesn't say whether AI tools were used.

If you only want an everyday local setup, Ollama remains the simpler route for models that fit in memory. Janas is aimed at the case where the model doesn't fit, which matters more as open MoE releases grow. Meta's Muse Spark Open, for example, has 816 billion total parameters and 42 billion active.

What's confirmed, reported and uncertain

• Confirmed from the repository: the author, the GPL-3.0-or-later license, the September 18 start date, the supported platforms and models, the design described in its docs, and the warnings.
• Developer-reported: every speed figure, including the comparisons with llama.cpp.
• Uncertain:
  • performance on any other machine, AMD CPUs, discrete GPUs or SATA disks
  • how output quality holds up when light experts are skipped on very small caches
  • whether the project will stabilize