Give your computer
intelligence.
CIU is a local intelligence layer. Any app can connect to it, from a spreadsheet to a game engine to something you build yourself.
- Runs locally
- Your data stays yours
- Works with any app
Free during beta. Your feedback shapes what ships next.
Optimized models
Built for machines people actually own.
CIU currently has validated NF4DQ builds for Qwen 3.5 9B and Qwen 3.8 27B. The 9B is the accessible everyday option; the 27B is there when your machine has more memory. More open-source models are coming soon.
Qwen 3.5 9B
4.346 GiBRecommended starting point for tighter memory budgets and mainstream local hardware.
Qwen 3.8 27B
13.23 GiBHigher-capability option for machines with more memory.
More open models
Additional open-source model families will be added as optimized CIU builds are validated.
| Build | Size | Perplexity |
|---|---|---|
| Q4_K_M | 5.280 GiB | 8.2055 |
| NF4DQ | 4.346 GiB | 8.4181 |
| Q3_K_M | 4.342 GiB | 9.2978 |
Lower perplexity is better. NF4DQ is effectively the same size as Q3_K_M and reduces perplexity by about 9.5% versus Q3_K_M; Q4_K_M lowers perplexity by about 2.5% but uses about 21.5% more weight memory.
THE CIU INTEGRATION LIBRARY
Give every app
its own intelligence.
Install an addon. Keep the model on your machine. Let the apps you already use tap into the same local intelligence.
CIU Forge
for Blender
Bring CIU directly into your 3D workflow. Ask about a scene, generate geometry, and iterate without leaving Blender.
Explore Forge & installation guide →ZIP · v0.3.0 · Install from Blender Preferences → Add-ons → Install from Disk
Before you download anything
Find out what your machine can actually run.
Working this out is most of the difficulty with local models, and getting it wrong costs you a long download and a failed load. Pick your card.
4.50 GiB of 24 GB
13.40 GiB of 24 GB
17.66 GiB of 24 GB, shown for comparison
These are the figures a model needs before a conversation starts. On Apple unified memory the same pool is shared with everything else you have open, so an 8 GB Mac holds the 9B only with other applications closed.
How an addon works
This is the entire integration.
Because the model is loaded once and shared, a harness carries no model, no key and no server of its own. It sends context, gets an answer, applies it.
const r = await fetch("http://127.0.0.1:8674/v1/chat/completions", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model: MODEL_ID,
messages: [{ role: "user", content: contextFromTheApp }],
}),
});
const { choices } = await r.json();
applyToTheApp(choices[0].message.content);Creative tools
A 3D suite, an image editor, a DAW. The app knows its own scene, layers or session. A harness hands that to CIU and applies what comes back.
Notes and writing
Obsidian, Logseq, a static site generator. Local notes never have to leave the machine to be summarised, linked or queried.
Spreadsheets and data
Formula explanation, column classification, cleaning messy input. Work that is tedious by hand and too sensitive to paste into a browser.
Anything with a plugin API
If the application can run your code and make an HTTP request, it can reason. That is the whole requirement.
Why a shared layer
Your GPU holds one model. The second app that wants one fails.
Every local AI tool ships its own runtime and loads its own copy of the weights. Open a chat client and an editor assistant on the same machine and the second one has nowhere to go.
Without CIU
With CIU
Using it
Anything that speaks the OpenAI API works.
There is no CIU client and no SDK to learn. Set the base URL, send any API key, carry on.
http://127.0.0.1:8674/v1any non-empty stringChat front ends
Open WebUI, LibreChat, AnythingLLM and similar. Add a custom OpenAI provider and paste the local address.
Editors
Continue, Cline, Zed and anything else with a provider field that takes a base URL.
Your own code
The official OpenAI SDKs work unchanged. Point base_url at CIU and send any API key.
Agent frameworks
LangChain, LlamaIndex, CrewAI and the rest all accept a custom base URL.
The numbers, briefly
A 4.16 bit format, built for llama.cpp.
NF4DQ stores weights in about 4.16 bits each. Against Q4_K_M, the usual 4-bit choice, it is a quarter smaller and generates a quarter faster. Prompt processing is slower, and that is on this page rather than left out of it.
| Build | Size | Prefill | Decode |
|---|---|---|---|
| NF4DQ | 13.23 GiB | 734.52 | 60.81 |
| Q4_K_M | 17.66 GiB | 1300.04 | 48.67 |
Before you start
What CIU cannot do yet.
- NVIDIA on Windows and Linux, Apple Silicon on Mac.
- There is no Vulkan kernel yet, so AMD and Intel GPUs fall back to a CPU path at well under one token per second. That is not usable and this page will not pretend otherwise.
- Windows builds are published but untested on real hardware.
- They compile and the installer works. Nobody has yet run one on a machine with an NVIDIA card in it. If you have one, try it and open an issue either way.
- The weights need the fork.
- NF4DQ is ggml type 43, which exists only in this fork of llama.cpp. The GGUFs will not load correctly in a stock build.
- Memory floor.
- About 4.5 GiB free for the 9B and about 13.4 GiB for the 27B, before the first message.
Ready to run CIU?
Join the beta to unlock CIU installation and the Blender addon.