Skip to content

Local AI

Your computers. Your models. Your AI.

Run private AI on the hardware you already own. Toskar recommends models that fit, connects your computers, and handles the infrastructure so you can just use it.

brew tap yeixio/toskar https://github.com/yeixio/toskar-core
brew install yeixio/toskar/toskar
toskar

Then open http://127.0.0.1:7331 in your browser. Toskar Core is free and open source. All downloads

Use it from other devices(optional)

At first only the computer running Core can reach it. To use it from a laptop, phone, or app elsewhere on your home or office network:

  1. On the computer running Core, open http://127.0.0.1:7331, go to API Access, and turn on Local network access. It has you create an API key. Copy it now: it is shown only once.
  2. Restart Core so it listens on the network.
    # macOS (Homebrew): press Ctrl+C where toskar is running, then
    toskar
    
    # Linux
    sudo systemctl restart toskar
    
    # Windows: close toskar.exe and start it again
  3. Find the computer's local address, such as 192.168.1.20.
    ipconfig getifaddr en0   # macOS
    hostname -I              # Linux
    ipconfig                 # Windows: the IPv4 Address line
  4. On the other device, open http://ADDRESS:7331 and enter the key when asked. Apps that speak the OpenAI API use http://ADDRESS:7331/v1 with the key.

A Linux server with no browser can do the same before it starts. Pick a key of at least 12 characters, for example from openssl rand -hex 24:

sudo systemctl edit toskar
# add these lines, then save:
[Service]
Environment=TOSKAR_API_HOST=0.0.0.0
Environment=TOSKAR_API_KEY=your-long-random-key

sudo systemctl restart toskar

The key keeps other people on the network out. Since Toskar 1.7 the same address also answers over https://, which encrypts what you send; a browser warns about Toskar's own certificate until you trust it or install your own. Use this on a network you trust, not public Wi-Fi. If a firewall is on, allow port 7331. Your Mac may ask once whether toskar can accept incoming connections.

Connecting the Toskar app, an OpenAI-compatible app, or Claude Desktop? Read the full guide.

Toskar Core

The open-source engine. Run it on servers, workstations, and home labs, or build your own client against its APIs.

Free · AGPL-3.0-or-later

Toskar Pro

Local AI, made simple, on your computer. It runs your models and serves your other devices.

Mac App Store · Microsoft Store · Free to try, then $24.99

Toskar

Your AI on your phone, tablet, watch, TV, and in the car: a model on the device, or Toskar Pro on your computer.

App Store · Google Play · Free, Unlock $9.99

Toskar Relay

Reach your computer from anywhere, encrypted end to end. Hosted by us, or run by your organization.

From $4.99 a month

Get the apps

Toskar Pro for your Mac or Windows PC · free to try, then $24.99

Download Toskar Pro on the Mac App StoreDownload Toskar Pro from the Microsoft Store

Toskar for your phone, tablet, watch, and TV · Free, Unlock $9.99

Download Toskar on the App StoreGet Toskar on Google Play

Toskar is open source. Buying Toskar or Toskar Pro pays for our work on it, and keeps Toskar Core free for everyone. The apps are the easy way in, not a paywall.

How it works

From your question to an answer

Every chat, API request, and automation takes the same path. It stays on your computers unless you let it search the web or use a service you connected, and Settings lists everything that left.

A request goes from your apps to Toskar, where Huginn, Muninn and Mimir, and Norn handle it, then to this computer or a paired computer over Bifrost, and the answer comes back.BIFROSTYOUR APPSChat · Toskar apps · your code · MCP appsTOSKAR ON THIS COMPUTERHuginnpicks the modelMuninn · Mimiradd contextNornplaces itThis computerruns models that fitits memoryA paired computermore memory, forbigger models
  1. 1You ask

    From chat, Toskar Pro, the Toskar app, your own code over the OpenAI API, or an app like Claude Desktop over MCP.

  2. 2Huginn picks the model

    It reads what kind of request it is and picks the model that suits it and fits your computer's memory.

  3. 3Muninn and Mimir add what it needs to know

    Memories that fit the question, and passages from the knowledge you connected. The web is looked up only if you allow it.

  4. 4Norn places it

    On this computer, or over Bifrost on a paired computer with room for a bigger model. Anything marked “This computer only” stays here.

  5. 5The answer comes back, checked

    Figures are checked against the sources, and the answer shows where it came from.

Before you install

What can my computer run?

Pick your computer and its memory to see which of Toskar's models fit, with the same estimate the app uses. How memory decides what runs · Every model and computer

Your computer

Memory (unified)

Find it under Apple menu > About This Mac. On Apple silicon the graphics share this memory.

21 of 24 models run on this computer, 19 of them comfortably.

Excellent fitPlenty of memory to spare.

  • Qwen 2.5 14Breasoning≈ 10.2 GB
  • Qwen 2.5 Coder 14Bcoding≈ 9.7 GB
  • Qwen 3.5 9Bvision≈ 7.2 GB
  • Gemma 2 9B≈ 6.5 GB
  • Qwen 2.5 VL 7Bvision≈ 6.1 GB
  • Qwen 2.5 Coder 7Bcoding≈ 5.3 GB
  • Qwen 2.5 7B≈ 5.3 GB
  • DeepSeek R1 Distill 7Breasoning≈ 5.3 GB
  • Mistral 7B Instruct≈ 5.0 GB
  • Qwen 3.5 4Bvisionfast≈ 3.9 GB
  • Gemma 3 4Bvision≈ 3.9 GB
  • Phi 3.5 Minireasoning≈ 3.0 GB
  • Llama 3.2 3B≈ 2.6 GB
  • Qwen 2.5 3B≈ 2.5 GB
  • Gemma 2 2Bfast≈ 2.2 GB
  • Qwen 2.5 Coder 1.5Bcodingfast≈ 1.6 GB
  • DeepSeek R1 Distill 1.5Breasoningfast≈ 1.6 GB
  • Llama 3.2 1Bfast≈ 1.3 GB

Good fitShould run comfortably.

  • gpt-oss 20Breasoningcoding≈ 13.1 GB

May run slowlyMay need swap or a shorter context.

  • Qwen 3.8 27Bvisioncoding≈ 20.4 GB
  • Muse Glimmer 30Breasoningvision≈ 17.1 GB

Too big for this computer: gpt-oss 120B, Qwen 3.6 35B-A3B, Qwen 2.5 32B. A paired computer with more memory can run them for you.

Estimates use Toskar 1.14.1's own sizing: the model file plus working memory at an 8K context. Once a model has run, Toskar measures it on your computer and uses that instead.

What's new

New in 1.14

Everything in every release →
  • Ratatoskr

    Swarms: several agents on one job

    Ask for "a swarm" or "3 agents" and several agents work at once, each in a chat of its own, across your computers and models. In a split each takes a part of the job; in a race, models answer the same question and the best answer wins.

  • Muninn

    Projects keep related chats together

    A project holds instructions, files, a folder, a default profile and model, and a memory of its own. Its chats follow its instructions, search its files, and remember what they learn for the project's other chats, not your others.

  • Ymir

    gpt-oss and Qwen 3.6 in the catalog

    OpenAI's gpt-oss 20B and 120B and Qwen 3.6 35B-A3B are in the model catalog. They're mixtures of experts, so how well they fit and how fast they run is estimated from the part each word uses. Effort sets how long gpt-oss thinks.

  • Norn

    Keep a model running

    Pin a chat model as Always running and it loads when Toskar starts, comes back if it stops, and isn't unloaded when idle. With paired computers, a model that doesn't fit beside it runs on another computer; otherwise an Admin can stop the pinned one for that answer.

  • Huginn

    Honest answers offline, and links that open

    Offline, a question that needs the internet says this computer isn't connected instead of making up an answer. Turn on Check links in answers, and Toskar asks each linked site whether its page opens and crosses out the links that don't.

  • Heimdall

    Changes ask first again

    In the built-in profiles, writing files, running terminal commands, and Git add and commit ask first, and Allow for session still covers the rest of a chat. Git push asks every time, since it sends your code off the computer.

Works with

Bring it into your tools

Your local AI inside the apps you use

These apps can ask your local AI and search your connected knowledge over MCP. Toskar gives you the settings to paste.

  • Claude Desktop
  • Claude Code
  • Cursor
  • VS Code
  • any MCP app
How to connect them

Anything that speaks the OpenAI API

Point any app or SDK that lets you set an OpenAI base URL at Toskar. Use auto as the model and Toskar picks one for each request. No key is needed on this computer.

API reference
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:7331/v1", api_key="local")
reply = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(reply.choices[0].message.content)

With Toskar running, this prints a reply from a model on your computer.

The Toskar mark

The World Tree

Every part has a name

The mark draws Yggdrasil, the World Tree, as circuit traces, with roots and branches ending in nodes, like the computers Toskar connects into one system. Inside the app, each page carries a name from the old stories and an Elder Futhark rune. Pick one to see it.

Chat: A normal chat

Ratatoskr

The messenger squirrel of the World Tree carries your messages to the models on your computers.

A normal chat

Ask in plain language. Answers come back as Markdown, and a web search shows up as a short status instead of raw tool syntax.

Also in the app

  • BrokkrTrain

    The smith who forged Mjölnir. Here you forge an AI of your own.

  • MimirKnowledge

    Mímir guards the well of wisdom. Mimir keeps your documents searchable.

  • MuninnMemory

    Odin's raven of memory keeps what you ask Toskar to remember.

  • GungnirTools

    Odin's spear never misses. Tools are how the assistant acts: the web, files, and the apps you connect.

  • OdinProfiles

    Odin sends out his ravens. Profiles decide how the assistant thinks and what it may do.

The mascot

Meet Ratatoskr

In the myths, Ratatoskr is the squirrel who runs up and down the World Tree, carrying messages between the eagle at the top and the dragon Níðhöggr at the roots. Toskar is named after him, and he does the same job here, carrying your requests between your apps and the models on your computers. He thinks while a reply is written, celebrates when something finishes, and naps when no model is loaded.

Toskar is running and nothing is waiting.

Walkthrough

Chat, models, computers, performance, diagnostics, and API access. This recording uses demo data and does not start a model.

Open source

Toskar Core is licensed AGPL-3.0-or-later. Community contributions are welcome: hardware reports, runtime notes, documentation, and code. Toskar Pro and the Toskar app are separate products and are not developed in that repository.

Open-source Core on GitHub