Disk Layout for Models, Caches, and Indexes
Put models, embedding caches, git clones, and SQLite indexes on separate paths so a full cache cannot bury the OS on a single Lucknow VPS disk.
AI Infrastructure8 min read
NetBay journal
AI Infrastructure
30 practical AI Infrastructure articles from Netbay Engineering — written for operators running Linux and Windows VPS in India.
30 articles in this topic
AllLinuxLinux AdministrationWindowsLinux SecurityLinux NetworkingContainersContainers & OrchestrationKubernetesCI/CDGitHubSelf-HostingBackups & DataDatabasesApp DeploymentDNS & TLSObservabilityEmailAI AgentsAI InfrastructureAPI & AutomationProxiesCloud ArchitectureHardware & PerformanceDevOps Practices
Agent Architecture: Local 7B Plus API Fallback
Put nginx, a tool-gated agent, a loopback 7B, Redis, and a cheap remote API on one Lucknow VPS so private prompts stay local and hard tasks can escalate.
AI Infrastructure9 min read
Picking 3B vs 7B vs 8B for an 8 GB CPU VPS
Choose a 3B, 7B, or 8B Q4 GGUF for an 8 GB Intel Xeon Platinum VPS using RAM, context, and tokens per second — not the Hugging Face model card.
AI Infrastructure8 min read
Observe Tokens, Latency, and OOM on CPU Inference
Log tokens, encode_ms, generate_ms, and RSS on every CPU inference request so you see saturation and OOM risk before the OOM killer ever fires.
AI Infrastructure9 min read
Health Checks and Draining a Local Model Process
Probe llama-server with /health, stop nginx from sending new work, and SIGTERM the GGUF process only after in-flight CPU generates finish safely.
AI Infrastructure8 min read
Back Up Vector Databases and Model Configs
Snapshot vector indexes, prompt packs, and router YAML on a Lucknow VPS, skip re-downloadable weights, and rehearse a restore before the disk fills.
AI Infrastructure8 min read
Rate-Limit Your Own Inference API on a VPS
Put token buckets and nginx limit_req in front of a local CPU inference API so one client cannot OOM the box or stall every other RAG query.
AI Infrastructure8 min read
Put nginx TLS in Front of a Local Model API
Terminate TLS with nginx, keep llama-server on loopback, and proxy OpenAI-shaped routes without exposing a CPU model port to the public internet.
AI Infrastructure8 min read
Caching Prompts and Embeddings on a Linux VPS
Hash the model name plus UTF-8 bytes into a local SQLite cache so repeat search queries skip the encoder and the generator on a Lucknow VPS.
AI Infrastructure8 min read
Plan LLM Capacity as Tokens Per Second on Xeon
Measure prefill and decode tokens per second on Intel Xeon Platinum CPU so a Lucknow VPS queue depth matches real 7B throughput, not marketing.
AI Infrastructure8 min read
LLM Concurrency: One User at a Time vs a Queue
Run one CPU generation at a time, queue the rest, and avoid llama.cpp parallel slots that multiply KV cache RAM on an 8 GB CPU VPS in Lucknow.
AI Infrastructure8 min read
Stop Prompt Injection Before Untrusted Tools Run
Treat user text as hostile, gate every tool with an allowlist and confirmation, and never let a model-chosen URL or shell string reach your Lucknow VPS.
AI Infrastructure8 min read