Clarit.ai Home
Run modern AI models on the hardware you already own — laptop, workstation, server, or Raspberry Pi-class edge board. No cloud dependency, no data leaving the device, no API bills that scale with your usage.
Sometimes the cloud just isn't the right answer — the connection drops, the data can't leave, the latency is too high, or the API bill is eating your margin. The hardware to run AI locally already exists in your pocket and on your desk. The software to use it well is what's been missing.
Synapse packages the best open-source CPU inference speedups with modern support for neural-processing chips into one program that does the right thing on any hardware you point it at. You install it once; it figures out what to use — your CPU, your GPU, or a dedicated NPU — and runs the model accordingly. The same binary that runs on a workstation runs on a $60 ARM board with a built-in neural chip.
Where Synapse earns its keep
Optimized quantization kernels for x86 and ARM. IQ + Trellis formats deliver 1.7–1.9× speedup over standard k-quants on ARM NEON.
GPU acceleration for desktop and server. FlashMLA support for DeepSeek-class models, multi-GPU graph splitting, automatic VRAM offload sizing.
Dedicated neural accelerator support via RKNPU2. Modernized to current ggml standards — unlike competing Rockchip forks that lag 1,000+ commits behind.
Built-in HTTP server with chat completions, function calling, and vision endpoints. Drop-in replacement for cloud API calls.
IQK and Trellis quant families — smaller, faster models without quality loss. Runtime tensor repacking picks the best format for your specific hardware.
Multi-Latent Attention with Flash Attention for DeepSeek-class models — the fastest CPU and CUDA inference for MLA architectures available anywhere.
LLaMA, Qwen, DeepSeek, Gemma, Granite, Nemotron, GLM, Kimi, Mistral and more — new architectures added weekly as upstream lands them.
Multi-modal vision support and structured function calling out of the box. Speculative decoding, control vectors, and adaptive sampling included.
Synapse is a maintained merge of two upstream sources plus GGML core tracking. Automated sync keeps drift low; clean changes land without manual intervention.
| Component | Source | Cadence |
|---|---|---|
| CPU + quantization | ik_llama.cpp | Weekly |
| Rockchip NPU | KHA/rk-llama.cpp | As needed |
| GGML core | tracked via ik | Continuous |
Research and expansion priorities on the Synapse tree. No promises of timelines — signal of direction. Hardware-partner demand and open-source interest shape what ships first.
Extending the snapshot-persistence model from Engram to llama.cpp — persistent hidden-state for Mamba and hybrid SSM architectures on edge hardware. Requires R&D; no committed timeline.
Beyond Rockchip: scoping AMD XDNA (Ryzen AI), Qualcomm Hexagon, and other NPU silicon. Each vendor brings its own SDK; prioritization will follow hardware-partner demand.
Working with edge-device makers on pre-flashed Synapse builds for turnkey local-AI appliances. Olares is an early conversation; open to more.
Synapse is open source. We work with hardware makers, edge deployment teams, and anyone who thinks AI should run on what you already own — not behind someone else's API.