Distributed LoRA Fine-Tuning Pipelines with Ray Train and DeepSpeed ZeRO-2
Train custom foundation model adaptors efficiently on multi-GPU server clusters with automated checkpoint validation and memory-balanced sharding.
AI Infrastructure8 min read
NetBay journal
AI Infrastructure
34 practical AI Infrastructure articles from Netbay Engineering — written for operators running Linux and Windows VPS in India.
34 articles in this topic
AllLinuxLinux AdministrationWindowsLinux SecurityLinux NetworkingContainersContainers & OrchestrationKubernetesCI/CDGitHubSelf-HostingBackups & DataDatabasesApp DeploymentDNS & TLSObservabilityEmailAI AgentsAI InfrastructureAPI & AutomationProxiesCloud ArchitectureHardware & PerformanceDevOps Practices
Scaling Milvus Vector Search on Enterprise NVMe SSDs for Billion-Scale RAG
Tune HNSW index graphs, memory-mapped vector segments, and disk-ann partitions for million-query-per-minute semantic retrieval architectures.
AI Infrastructure6 min read
Low-Latency Model Serving with NVIDIA Triton and TensorRT-LLM on Linux
Build production-grade dynamic model pipelines that multiplex embedding, classification, and generation models across unified GPU memory pools.
AI Infrastructure7 min read
Deploying High-Throughput vLLM Inference Clusters on Dedicated GPU VPS
Optimize PagedAttention, continuous batching, and chunked prefill for serving open weights models at sub-20ms first-token latency in production.
AI Infrastructure8 min read
Disk Layout for Models, Caches, and Indexes
Put models, embedding caches, git clones, and SQLite indexes on separate paths so a full cache cannot bury the OS on a single Lucknow VPS disk.
AI Infrastructure8 min read
Agent Architecture: Local 7B Plus API Fallback
Put nginx, a tool-gated agent, a loopback 7B, Redis, and a cheap remote API on one Lucknow VPS so private prompts stay local and hard tasks can escalate.
AI Infrastructure9 min read
Picking 3B vs 7B vs 8B for an 8 GB CPU VPS
Choose a 3B, 7B, or 8B Q4 GGUF for an 8 GB Intel Xeon Platinum VPS using RAM, context, and tokens per second — not the Hugging Face model card.
AI Infrastructure8 min read
Observe Tokens, Latency, and OOM on CPU Inference
Log tokens, encode_ms, generate_ms, and RSS on every CPU inference request so you see saturation and OOM risk before the OOM killer ever fires.
AI Infrastructure9 min read
Health Checks and Draining a Local Model Process
Probe llama-server with /health, stop nginx from sending new work, and SIGTERM the GGUF process only after in-flight CPU generates finish safely.
AI Infrastructure8 min read
Back Up Vector Databases and Model Configs
Snapshot vector indexes, prompt packs, and router YAML on a Lucknow VPS, skip re-downloadable weights, and rehearse a restore before the disk fills.
AI Infrastructure8 min read
Rate-Limit Your Own Inference API on a VPS
Put token buckets and nginx limit_req in front of a local CPU inference API so one client cannot OOM the box or stall every other RAG query.
AI Infrastructure8 min read
Put nginx TLS in Front of a Local Model API
Terminate TLS with nginx, keep llama-server on loopback, and proxy OpenAI-shaped routes without exposing a CPU model port to the public internet.
AI Infrastructure8 min read