Topic · 29 stories · page 1 of 2
29 stories tagged kubernetes on AI Tamer, newest first.
Microsoft patches the AKS app routing add-on's NGINX ingress only through November 2026. Its Gateway API successor gets a new IP, so plan a DNS cutover.
EKS clusters created since 2018 have 10-year CAs nearing rotation. AWS rotates them, but nodes you manage and kubeconfigs must trust the new CA first.
Dynamo 1.6.0 adds pluggable router queues, multi-replica Rust EPP and coding-agent session routing. One default change touches Claude Code traffic.
llm-d 1.0, a CNCF Sandbox project started by Red Hat, IBM, Google Cloud, CoreWeave and NVIDIA, adds flow control, router HA and DisaggregatedSet.
NVIDIA's 13 October post says Run:ai is open source under DSX. The platform and its component repositories are under Apache 2.0.
Use an Istio VirtualService to copy live requests to a candidate model deployment, discard its responses, and compare its errors and latency with the stable version.
Liveness decides whether Kubernetes restarts a container. Readiness decides whether it gets traffic. Wiring one health check to both turns a slow dependency into a restart storm.
Use py-spy dump, top and record to see what a stuck or slow Python process is doing without restarting it, and grant the ptrace access it needs on Linux, Docker and Kubernetes.
Kubernetes refreshes ordinary Secret volume files eventually, but a subPath mount stays on its old content. Credential rotation also needs an application reload or a Pod rollout.
EndpointSlice conditions distinguish an endpoint that is serving from one accepting normal new traffic. Use ready, serving, and terminating to design a drain policy for long voice or AI sessions.
A Deployment manifest and an autoscaler can both set the replica count. Kubernetes field ownership explains the conflict and how to hand over scaling safely.
Memory requests place a Pod. Limits constrain its containers. Learn how to tell a container out-of-memory kill from node-pressure eviction and size inference workloads accordingly.
A replica count alone does not spread Pods across failure zones. Kubernetes topology spread constraints let you define the placement you need and the tradeoff when a zone is unavailable.
A Kubernetes scratch volume can hold files a model server prepares at startup. Its lifetime makes it a poor home for the only copy of a model or a result.
An ephemeral container gives you a way to inspect a running Kubernetes pod when its application image lacks a shell or diagnostic tools.
Use a Kubernetes init container and a shared volume to put model files in place before an inference server starts. Plan for retries, storage use, and model loading.
A PodDisruptionBudget can limit how many inference pods a node drain evicts at once. Its allowed-disruptions count shows when the workload has room for another eviction.
Kubernetes can give urgent Pods earlier scheduling attempts without removing running work. Non-preempting priority still depends on suitable capacity.
CPU requests guide Pod placement. CPU limits cap a running container's CPU time.
Inference requests can wait while CPU utilization stays below its scaling target. Measure pending work and queue delay to decide when more workers are useful.
A sudden inference surge meets several gates before new Pods serve traffic. Follow the autoscaler's measurement loop, replica rules, Node capacity, and readiness.
Creating a Kubernetes NetworkPolicy does not mean its rules are already enforced. Learn what can happen during the handoff and how to check the result.
A Deployment can exist even when its Pods cannot be created. Check namespace quota usage and resource declarations before investigating node placement.
Device plugins advertise GPUs as countable resources. That changes how Pods request them and how many workloads a node can schedule.
