Reranking RAG Results on a Small CPU VPS
Rerank a short hybrid shortlist with a MiniLM cross-encoder on CPU so the generator sees the right source chunk without calling a vendor rank API.
AI Infrastructure8 min read
NetBay journal
AI Infrastructure
30 practical AI Infrastructure articles from Netbay Engineering — written for operators running Linux and Windows VPS in India.
30 articles in this topic
AllLinuxLinux AdministrationWindowsLinux SecurityLinux NetworkingContainersContainers & OrchestrationKubernetesCI/CDGitHubSelf-HostingBackups & DataDatabasesApp DeploymentDNS & TLSObservabilityEmailAI AgentsAI InfrastructureAPI & AutomationProxiesCloud ArchitectureHardware & PerformanceDevOps Practices
Host Private LLM Inference on a Single VPS
Bind a local 7B to loopback, cap RAM and threads with systemd, and let only your app call it so private prompts never leave a Lucknow VPS host.
AI Infrastructure8 min read
RAG Ingest Pipeline: Git Pull, Embed, Upsert
Pull a docs repo, chunk the diff, embed new blobs, and upsert by path so your local VPS RAG index tracks git instead of a full nightly rebuild.
AI Infrastructure9 min read
Context Length vs RAM vs Tokens per Second
See how context length grows KV cache RAM and slows prompt eval on a CPU VPS so you can pick 2k, 4k, or 8k windows without an OOM on 8 GB RAM.
AI Infrastructure8 min read
Fine-Tuning Is Usually the Wrong First LLM Step
Evaluate prompts, retrieval, and routing on a VPS before you collect a fine-tune set, because most quality gaps are instructions and data, not weights.
AI Infrastructure8 min read
Q4 vs Q5 vs Q8 GGUF on a RAM-Limited CPU VPS
Compare Q4, Q5, and Q8 GGUF sizes on an 8 GB Intel Xeon Platinum VPS and pick the quantization that fits RAM without killing tokens per second.
AI Infrastructure8 min read
Hybrid Search with Keyword Plus Embeddings
Combine FTS5 or Postgres tsvector with cosine top-k, fuse ranks with RRF, and retrieve identifiers that pure embeddings miss on a single VPS.
AI Infrastructure8 min read
Queue LLM Inference Jobs with Redis on Ubuntu
Move model calls off the HTTP request path into a Redis list plus a worker so a Lucknow VPS can absorb bursts without dropping chats or overrunning spend.
AI Infrastructure8 min read
Serve GGUF Models with llama-server on a VPS
Run llama-server as a long-lived CPU process, bind GGUF weights to localhost, and expose chat routes on an Intel Xeon Platinum VPS without a GPU.
AI Infrastructure8 min read
Chunk Markdown and Code for RAG on a VPS
Split markdown by headings and code by functions so RAG retrieves complete units instead of random 512-token windows on your private VPS corpus.
AI Infrastructure8 min read
Stream LLM Tokens Through nginx Without Buffering
Turn off nginx proxy buffering and gzip for Server-Sent Events so tokens from a remote model API reach the browser as they generate live on Ubuntu.
AI Infrastructure8 min read
An OpenAI-Compatible Local API for llama.cpp
Point existing OpenAI clients at llama-server on localhost so a small GGUF model serves chat completions on a CPU VPS in Lucknow without a GPU.
AI Infrastructure8 min read