Toskar Core
An open-source, self-hosted AI server.
Toskar Core runs AI models on computers you already own, with llama.cpp as its inference engine, and serves them through OpenAI- and Anthropic-compatible APIs and a web interface. Pair a Mac, a PC with a graphics card, and a Linux box, and it spreads the work across them.
For self-hosters, home labs, and developers. Free, and licensed AGPL-3.0-or-later.

What it does
What you get
OpenAI- and Anthropic-compatible APIs
Apps, SDKs, scripts, and coding agents built for OpenAI or Anthropic work against your own computer instead. Each app gets its own key, with its own limits.

Models that fit
Core looks at your computer's memory and graphics, recommends models that fit, and installs GGUF models from its catalog, from Hugging Face, or from a URL.

Several computers as one
Pair the computers on your network, and each request runs where its model fits. Traffic between them is encrypted.

See what it's doing
Live memory, speed, and GPU use for each computer, a health check, and a list of everything that left your network.

Your documents
Connect folders, PDFs (scanned ones too), and spreadsheets as knowledge, and answers cite where they came from.
Tools and editors
Add tools from MCP servers, and use Toskar from Claude Desktop, Claude Code, Cursor, and VS Code.
Train your own AI
LoRA fine-tuning on your own examples, on Apple silicon or an NVIDIA GPU with PyTorch and PEFT, on this computer or a paired one. Export the result as one GGUF file.
Faster answers
Speculative decoding: the largest models come with a small draft model that writes ahead while the large one checks, where the model fits well.
Models you already have
Use the GGUF models LM Studio, Ollama, llama.cpp, and GPT4All already downloaded, where they are, with no second copy.
Automations
Tasks that run on a schedule or when something changes, and tell you the result.
People and sign-in
Accounts with roles, private chats per person, and sign-in with an OpenID Connect provider.
Runs as a service
The Linux packages install a system service, and it runs headless on a server with no desktop.
Your other devices
The Toskar app on your phone, tablet, watch, and TV connects to Core with a one-time code.
How it fits
One system, four parts
Your models run on your computer, with Toskar Core or Toskar Pro. Toskar on your other devices uses them: directly at home, and through a relay when it has to away from home.
Privacy
What stays, and what leaves
Stays with you
- Your chats, documents, memories, settings, and models, in Core's data folder on your computer.
- The answers: models run on your own computers, not on a hosted AI service.
- There's no account with us, and no analytics, ads, or crash reporting.
Leaves only when you use it
- A model you choose to install, downloaded from Hugging Face.
- Web searches and pages, when you let it search the web.
- What a connected service or tool needs, when you set one up.
- Settings lists everything that left the computer, under What left this computer.
Requirements
What you need
- macOS
- Apple silicon or Intel. Install with Homebrew, the install script, or the headless archive.
- Linux
- x86-64 or ARM64. The install script, the apt repository, or the deb and rpm packages, which run it as a service.
- Windows
- x86-64, from the headless archive. For an app with a window, get Toskar Pro.
- Graphics
- Core uses the GPU when it can: Apple silicon on a Mac, and NVIDIA, AMD, or Intel graphics through Vulkan on Linux and Windows. Without one, models run on the processor, more slowly.
- Disk and network
- A few gigabytes for the first model, and internet access to download it. After that, chats work offline.
| Memory | Models that fit well |
|---|---|
| 8 GB | Small models, up to about 3.5 GB, such as Llama 3.2 3B and Qwen 3.5 4B |
| 16 GB | Models up to about 9 GB, such as Qwen 3.5 9B |
| 24 to 32 GB | The 14B models, such as Qwen 2.5 14B |
| 64 GB or more | The 30B models, such as Qwen 3.8 27B and Qwen 2.5 32B |
Toskar checks your computer and labels every model by how well it fits. What can my computer run?
Price
Free and open source
Toskar Core costs nothing, and the source is on GitHub. Hardware reports, documentation, and code are all welcome; see Contributing.
Toskar is open source. Buying Toskar or Toskar Pro pays for our work on it, and keeps Toskar Core free for everyone. The apps are the easy way in, not a paywall.
If Core is useful to you, you can donate to Toskar, once or monthly, or sponsor on GitHub. Donations are optional.
Questions
- What's the difference between Toskar Core and Toskar Pro?
- Toskar Core is the open-source server, with a web interface you open in a browser. Toskar Pro is Core and a desktop app in one install, from the Mac App Store or the Microsoft Store. Core has no trial and nothing in it is locked.
- Is it really free?
- Yes. Toskar Core is free, open-source software under AGPL-3.0-or-later, and nothing in it is gated on a payment or a donation.
- Which systems does it run on?
- macOS on Apple silicon and Intel, Linux on x86-64 and ARM64, and Windows on x86-64. On Linux it installs with a script, from the apt repository, or as a deb or rpm package; on a Mac, with Homebrew.
- Does it work without the internet?
- Yes, once a model is downloaded. Without the internet, only web search and connected services stop working.
- Can apps built for OpenAI use it?
- Yes. Core serves OpenAI- and Anthropic-compatible APIs on your computer, so apps, SDKs, and scripts built for them can point at it instead, with a key per app.
Install it in one command
The download page has the install script, Homebrew, apt, and every release file. Then read the documentation or the HTTP API.