Self-Hosted AI & LLMs

Local model runners

★ Ollama — single binary; pull and run open-weight models with one command. The default for new self-host AI in 2026.
- https://github.com/ollama/ollama
LocalAI — drop-in OpenAI-compatible API; many backends (llama.cpp, transformers, Whisper, image gen).
- https://github.com/mudler/LocalAI
llama.cpp — the underlying runtime; minimal CLI + server.
- https://github.com/ggerganov/llama.cpp
vLLM — high-throughput inference server; optimized for GPUs; production-grade.
- https://github.com/vllm-project/vllm
Text Generation Inference (TGI) — Hugging Face inference server; production.
- https://github.com/huggingface/text-generation-inference
MLX-LM (Apple) — Apple-Silicon-native inference; very efficient on Macs.
- https://github.com/ml-explore/mlx-lm

★ Open WebUI — ChatGPT-shape UI for Ollama / OpenAI / Anthropic / others. Plugins, RAG, web search. The default front-end.
- https://github.com/open-webui/open-webui
★ LibreChat — multi-provider chat UI; agents; group chats; very feature-rich.
- https://github.com/danny-avila/LibreChat
AnythingLLM — chat + document RAG + agents in one app.
- https://github.com/Mintplex-Labs/anything-llm
Jan — desktop ChatGPT alternative; runs models locally.
- https://github.com/janhq/jan
OpenChat UI — minimal alternative.
Hollama — minimalist Ollama UI.
- https://github.com/fmaclen/hollama
Page Assist — browser extension for local LLMs.
- https://github.com/n4ze3m/page-assist

★ Continue.dev — VS Code / JetBrains extension; uses any local or cloud model.
- https://github.com/continuedev/continue
TabbyML — self-hosted Copilot alternative.
- https://github.com/TabbyML/tabby
Cody / Sourcegraph — has self-host tier.
Cline (formerly Claude Dev) — Claude-focused VS Code agent.
- https://github.com/cline/cline

★ ComfyUI — node-graph Stable Diffusion / Flux UI; the de-facto standard.
- https://github.com/comfyanonymous/ComfyUI
Automatic1111 webui — the original Stable Diffusion web UI.
- https://github.com/AUTOMATIC1111/stable-diffusion-webui
Invoke AI — alternative; cleaner UI for non-technical users.
- https://github.com/invoke-ai/InvokeAI
Fooocus — opinionated, simpler than Automatic1111.
- https://github.com/lllyasviel/Fooocus

Open Interpreter — local LLM-driven shell.
- https://github.com/OpenInterpreter/open-interpreter
AutoGPT — autonomous task agents.
- https://github.com/Significant-Gravitas/AutoGPT
Mastra / LangGraph / agent frameworks — see AI Agent Frameworks.

pgvector — Postgres extension; the simplest vector store.
Qdrant — Rust vector DB; fast.
- https://github.com/qdrant/qdrant
Weaviate — vector DB with ML modules.
- https://github.com/weaviate/weaviate
Chroma — Python-friendly.
Milvus — high-scale.
Vespa — search + vector hybrid.

★ Ollama + Open WebUI is the canonical "I want a chat UI on my GPU box" combo.
Pull models you'll actually use, not 50. Disk fills fast.
Pin model versions (llama3:8b-instruct-q4_K_M vs. llama3 tag).
GPU passthrough matters — Docker + NVIDIA / AMD container toolkit.
Mac Studios are remarkable inference boxes for the size; M-series unified memory loads larger models than equivalent NVIDIA cards.
Quantization — q4 / q5 quantized models run on consumer GPUs without quality cliff for chat.
Cache embeddings — re-computing on every query is wasteful.