Skip to content

Toskar Core

An open-source, self-hosted AI server.

Toskar Core runs AI models on computers you already own, with llama.cpp as its inference engine, and serves them through OpenAI- and Anthropic-compatible APIs and a web interface. Pair a Mac, a PC with a graphics card, and a Linux box, and it spreads the work across them.

For self-hosters, home labs, and developers. Free, and licensed AGPL-3.0-or-later.

A chat in Toskar Core's web interface, with the model that answered

What it does

What you get

  • OpenAI- and Anthropic-compatible APIs

    Apps, SDKs, scripts, and coding agents built for OpenAI or Anthropic work against your own computer instead. Each app gets its own key, with its own limits.

    API Access in Toskar Core, listing the keys apps use and their limits
  • Models that fit

    Core looks at your computer's memory and graphics, recommends models that fit, and installs GGUF models from its catalog, from Hugging Face, or from a URL.

    The Models page, with each model labeled by how well it fits this computer
  • Several computers as one

    Pair the computers on your network, and each request runs where its model fits. Traffic between them is encrypted.

    The Computers page, showing paired computers and the models each one runs
  • See what it's doing

    Live memory, speed, and GPU use for each computer, a health check, and a list of everything that left your network.

    The Performance page, with live GPU use and memory for each computer
  • Your documents

    Connect folders, PDFs (scanned ones too), and spreadsheets as knowledge, and answers cite where they came from.

  • Tools and editors

    Add tools from MCP servers, and use Toskar from Claude Desktop, Claude Code, Cursor, and VS Code.

  • Train your own AI

    LoRA fine-tuning on your own examples, on Apple silicon or an NVIDIA GPU with PyTorch and PEFT, on this computer or a paired one. Export the result as one GGUF file.

  • Faster answers

    Speculative decoding: the largest models come with a small draft model that writes ahead while the large one checks, where the model fits well.

  • Models you already have

    Use the GGUF models LM Studio, Ollama, llama.cpp, and GPT4All already downloaded, where they are, with no second copy.

  • Automations

    Tasks that run on a schedule or when something changes, and tell you the result.

  • People and sign-in

    Accounts with roles, private chats per person, and sign-in with an OpenID Connect provider.

  • Runs as a service

    The Linux packages install a system service, and it runs headless on a server with no desktop.

  • Your other devices

    The Toskar app on your phone, tablet, watch, and TV connects to Core with a one-time code.

How it fits

One system, four parts

Your models run on your computer, with Toskar Core or Toskar Pro. Toskar on your other devices uses them: directly at home, and through a relay when it has to away from home.

Privacy

What stays, and what leaves

Stays with you

  • Your chats, documents, memories, settings, and models, in Core's data folder on your computer.
  • The answers: models run on your own computers, not on a hosted AI service.
  • There's no account with us, and no analytics, ads, or crash reporting.

Leaves only when you use it

  • A model you choose to install, downloaded from Hugging Face.
  • Web searches and pages, when you let it search the web.
  • What a connected service or tool needs, when you set one up.
  • Settings lists everything that left the computer, under What left this computer.

The full app privacy policy

Requirements

What you need

macOS
Apple silicon or Intel. Install with Homebrew, the install script, or the headless archive.
Linux
x86-64 or ARM64. The install script, the apt repository, or the deb and rpm packages, which run it as a service.
Windows
x86-64, from the headless archive. For an app with a window, get Toskar Pro.
Graphics
Core uses the GPU when it can: Apple silicon on a Mac, and NVIDIA, AMD, or Intel graphics through Vulkan on Linux and Windows. Without one, models run on the processor, more slowly.
Disk and network
A few gigabytes for the first model, and internet access to download it. After that, chats work offline.
What fits well, by memory
MemoryModels that fit well
8 GBSmall models, up to about 3.5 GB, such as Llama 3.2 3B and Qwen 3.5 4B
16 GBModels up to about 9 GB, such as Qwen 3.5 9B
24 to 32 GBThe 14B models, such as Qwen 2.5 14B
64 GB or moreThe 30B models, such as Qwen 3.8 27B and Qwen 2.5 32B

Toskar checks your computer and labels every model by how well it fits. What can my computer run?

Price

Free and open source

Toskar Core costs nothing, and the source is on GitHub. Hardware reports, documentation, and code are all welcome; see Contributing.

Toskar is open source. Buying Toskar or Toskar Pro pays for our work on it, and keeps Toskar Core free for everyone. The apps are the easy way in, not a paywall.

If Core is useful to you, you can donate to Toskar, once or monthly, or sponsor on GitHub. Donations are optional.

Questions

What's the difference between Toskar Core and Toskar Pro?
Toskar Core is the open-source server, with a web interface you open in a browser. Toskar Pro is Core and a desktop app in one install, from the Mac App Store or the Microsoft Store. Core has no trial and nothing in it is locked.
Is it really free?
Yes. Toskar Core is free, open-source software under AGPL-3.0-or-later, and nothing in it is gated on a payment or a donation.
Which systems does it run on?
macOS on Apple silicon and Intel, Linux on x86-64 and ARM64, and Windows on x86-64. On Linux it installs with a script, from the apt repository, or as a deb or rpm package; on a Mac, with Homebrew.
Does it work without the internet?
Yes, once a model is downloaded. Without the internet, only web search and connected services stop working.
Can apps built for OpenAI use it?
Yes. Core serves OpenAI- and Anthropic-compatible APIs on your computer, so apps, SDKs, and scripts built for them can point at it instead, with a key per app.

Install it in one command

The download page has the install script, Homebrew, apt, and every release file. Then read the documentation or the HTTP API.