You’re reading the docs for 1.9.1, an older release. Read the latest →
Documentation / 1.9.1
Choose and manage models
Models separates discovery, installed models, and running models. Choose the target computer before installing or managing models. A downloaded model uses disk space; a running model also needs memory.
In this release, fit labels are Excellent, Good, Tight, May run slowly, and Unsupported. Runtime memory estimates account for the model file, quantization, and configured context. Tight memory is a warning; installation is blocked only for an unsupported machine or backend. Close other applications if free memory is low.
Model details distinguish Estimated speed from Measured performance. A model card is a planning estimate until that model has actually run on your computer. Start with a smaller model if loading fails or responses are too slow.
Browse all models searches Hugging Face for GGUF models, and a model can also be installed from a URL. Embedding and reranker models are supporting models: they help Knowledge search and never answer chats, so the chat menus leave them out. A loaded model is unloaded after 15 minutes without use, unless Settings sets the model lifecycle to manual or another time. Settings also sets the most disk space models may use.
Add a model on the Models page adds one that isn't in the list. From a file sends a GGUF file from this browser, or takes the location of one on this computer and either copies it into Toskar or uses it where it is, with no copy. From another app lists the models LM Studio, Ollama, llama.cpp, and GPT4All already downloaded, and adds them where they are, so they take no more space. From a link downloads a GGUF file from its address. Every file is checked before it's added, and its name, size, and context length come from the file. Deleting a model that's used where it is leaves the file alone.
These pages describe Toskar 1.9.1. Download 1.9.1 and its release notes ↗ · Need help? Report a bug or request a feature ↗