Install
Inference & Serving
Serving runtimes, Triton, vLLM, ONNX Runtime, and scalable endpoints.
- 3 Tracked terms
- Last 30 days Feed window
What this topic collects on
An article joins this feed when it matches these terms. Each one is also a search of its own.
Related topics
Latest in Inference & Serving
A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x
11+ hour, 4+ min ago (1209+ words) This article provides a step by step guide to measuring a laptop GPU against the CPU in the same chassis for serving Gemma 4 E2B through llama.cpp, with both builds on a fresh Debian sid toolchain and the passes run in…...
Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It
1+ day, 10+ hour ago (1553+ words) Part 1 began with the GPU already working. This part begins with a Google Cloud project and nothing in it. The work splits in two. Everything before first boot is a single gcloud command whose choices are GPU choices: which zone…...
How to run Gemma 4 locally: a setup that survives
1+ day, 15+ hour ago (482+ words) Start with text, add GPU offload and vision carefully, then pin the command and runtime that keep Gemma 4…. Tagged with technology, gemma4, localllms, homelab....
Blender 5.x and the VRM add-on: 5.2 LTS needs 4.4.0+, 5.3 alpha blocks even 4.7.1 (blender_version_max is exclusive)
1+ day, 21+ hour ago (306+ words) On Blender 5.2 LTS, VRM Addon for Blender only enables from 4.4.0 up, and on 5.3 alpha even the... Tagged with blender, python, vrm, tooling....
Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16
5+ day, 6+ hour ago (1298+ words) The GPU is already there. A T4 attached to a Compute Engine VM needs no queued resource, no instance launch and no image, so this rig has no provisioning tools at all. Everything it ships is about the software on the…...
Serving Gemma 4 on an AMD MI300X: What $1.99 an Hour Buys
6+ day, 6+ hour ago (1104+ words) The card is a DigitalOcean GPU droplet reached through AMD Developer Cloud (devcloud.amd.com) — same v2 API, same droplet ids, token from the My AMD Team account. Creating and destroying it are console actions, deliberately: both are dollar-per-hour decisions and…...
2B Gemma 4 Deployment with Cloud Run, NVIDIA L4, MCP SDK 2.x, and Claude Code
1+ week, 6+ day ago (1508+ words) This article provides a step by step deployment guide for Gemma 4 E2B to a Cloud Run hosted GPU enabled system. A suite of Python MCP tools is built to simplify management of the vLLM hosted deployment with Claude Code. This project…...
External Rollout Engines Roadmap??? Vime
2+ week, 2+ day ago (685+ words) An external rollout engine is a vLLM engine that is not launched by the vime training job. Another system deploys and owns the engine lifecycle; vime connects to those engines during training, registers a router, and syncs updated actor weights…...
Rustls 0.23.44 Released With ML-DSA Certificates Enabled By Default
2+ week, 2+ day ago (200+ words) Rustls as the modern TLS library implementation written in the Rust programming language is out with a new feature release. This morning's Rustls 0.23.44 release enables post-quantum secure ML-DSA certificates by default... - Categories Computers Display Drivers Graphics Cards Linux Gaming Memory…...
Three Gemma 4 Deployments on One T4G for Under $3: What the Runtime Changes, and What It Doesn't
3+ week, 1+ day ago (1818+ words) This article provides a step by step comparison of three Gemma 4 deployments on a single AWS hosted GPU enabled system. A suite of Python MCP tools is built to simplify management of each deployment, and one benchmark harness is shared…...