Serve GGUF Models with llama-server on a VPS
Run llama-server as a long-lived CPU process, bind GGUF weights to localhost, and expose chat routes on an Intel Xeon Platinum VPS without a GPU.
AI Infrastructure8 min read
NetBay journal
AI Infrastructure
34 practical AI Infrastructure articles from Netbay Engineering — written for operators running Linux and Windows VPS in India.
34 articles in this topic
AllLinuxLinux AdministrationWindowsLinux SecurityLinux NetworkingContainersContainers & OrchestrationKubernetesCI/CDGitHubSelf-HostingBackups & DataDatabasesApp DeploymentDNS & TLSObservabilityEmailAI AgentsAI InfrastructureAPI & AutomationProxiesCloud ArchitectureHardware & PerformanceDevOps Practices
Chunk Markdown and Code for RAG on a VPS
Split markdown by headings and code by functions so RAG retrieves complete units instead of random 512-token windows on your private VPS corpus.
AI Infrastructure8 min read
Stream LLM Tokens Through nginx Without Buffering
Turn off nginx proxy buffering and gzip for Server-Sent Events so tokens from a remote model API reach the browser as they generate live on Ubuntu.
AI Infrastructure8 min read
An OpenAI-Compatible Local API for llama.cpp
Point existing OpenAI clients at llama-server on localhost so a small GGUF model serves chat completions on a CPU VPS in Lucknow without a GPU.
AI Infrastructure8 min read
Postgres and SQLite Vector Store Without a Vendor
Store float32 embeddings in SQLite or Postgres on one VPS, skip a hosted vector database, and run honest nearest-neighbor search you can backup.
AI Infrastructure8 min read
Build a Multi-Model Router Cheap Model First
Send easy prompts to a cheap model and escalate only when a classifier, token budget, or failed JSON parse says the expensive model is worth it.
AI Infrastructure8 min read
Ollama on Ubuntu: Install, Pull, and systemd
Install Ollama on Ubuntu 24.04, pull a small 3B GGUF-backed model, and run it under systemd on an Intel Xeon Platinum CPU VPS without a GPU.
AI Infrastructure8 min read
CPU Embeddings with Sentence Transformers
Encode docs on CPU with MiniLM or llama.cpp, normalize 384-d vectors, and keep embeddings on one Lucknow VPS instead of a billed embedding API.
AI Infrastructure8 min read
Store Model API Keys as Secrets on a Linux VPS
Keep OpenAI and Anthropic keys in a root-owned systemd EnvironmentFile on Ubuntu so a Lucknow VPS never leaks model credentials through git or logs.
AI Infrastructure8 min read
Run a Small LLM on a CPU VPS with llama.cpp
Compile llama.cpp, load a Q4 GGUF 3B to 8B model, and generate text on an Intel Xeon Platinum CPU VPS without a GPU on High-Speed SSD in Lucknow.
AI Infrastructure8 min read