Local inference basics
What gives me a chat window on top?
The app layer is the part you see and click. Desktop apps like LM Studio and Jan give you a chat window, a model browser, and a runtime bundled inside, so one download gets you running without the command line.
If you would rather not touch a terminal, the app layer is for you. These are desktop programs with a chat window and a built-in browser for finding and downloading models. Most of them bundle a runtime inside, so a single install gives you both layers at once.
LM Studio is a desktop app whose tagline is to run AI models locally and privately, and it is free for home and work use.[1] You search for a model, download it, and chat with it in a point-and-click window, with versions for Windows, Mac, and Linux.[1] It can also run a small local server that other programs on your computer can connect to, which matters once you want to wire AI into other tools.
Jan takes a similar shape and leans hard on privacy. It calls itself an open source alternative to ChatGPT that runs fully offline on your computer, and it is released under the Apache 2.0 license.[2, 3] You can download open models from a library and run them locally, and it also lets you plug in cloud models if you want a mix.[2] Like LM Studio, it can expose a local server (at the address localhost:1337) so other apps can use the model it is running.[3]
Under the hood these apps lean on the same kind of engine described on the previous page; Jan, for instance, lists llama.cpp among the projects it is built on.[3] For most people that detail can stay hidden. You install the app, pick a model, and start typing.
References
- LM Studio — LM Studio
- Jan — Jan / Menlo Research
- Jan (source) — Menlo Research (GitHub)