Choose and manage models

Models separates discovery, installed models, and running models. Choose the target computer before installing or managing models. A downloaded model uses disk space; a running model also needs memory.

In this release, fit labels are Excellent, Good, Tight, May run slowly, and Unsupported. Runtime memory estimates account for the model file, quantization, and configured context. Tight memory is a warning; installation is blocked only for an unsupported machine or backend. Close other applications if free memory is low.

Model details distinguish Estimated speed from Measured performance. A model card is a planning estimate until that model has actually run on your computer. Start with a smaller model if loading fails or responses are too slow.

Browse all models searches Hugging Face for GGUF models, and a model can also be installed from a URL. Embedding and reranker models are supporting models: they help Knowledge search and never answer chats, so the chat menus leave them out. A loaded model is unloaded after 15 minutes without use, unless Settings sets the model lifecycle to manual or another time. Settings also sets the most disk space models may use.

Add a model on the Models page adds one that isn't in the list. From a file sends a GGUF file from this browser, or takes the location of one on this computer and either copies it into Toskar or uses it where it is, with no copy. From another app lists the models LM Studio, Ollama, llama.cpp, and GPT4All already downloaded, and adds them where they are, so they take no more space. From a link downloads a GGUF file from its address. Every file is checked before it's added, and its name, size, and context length come from the file. Deleting a model that's used where it is leaves the file alone.

After an answer, Toskar may offer a model that would have answered it better: one that writes code better, reads your files more reliably than a small model, writes your language better, thinks through math and logic, or reads a longer chat at once. It says why, how big the download is, and how well it fits this computer. Install and answer again installs it and answers the same question with it, as a new version you can switch with the first answer. Each kind is offered once in a chat, only to Admins and the Owner, and only for a model this computer runs well. Settings › History › Suggest better models turns it off for you.