Install
Frameworks & Training
PyTorch, TensorFlow, JAX, datasets, and accelerator workflows.
- 6 Tracked terms
- Last 30 days Feed window
What this topic collects on
An article joins this feed when it matches these terms. Each one is also a search of its own.
Related topics
Latest in Frameworks & Training
700 TPS on Kimi K3: A Case for TPU Megakernels
7+ hour, 39+ min ago (773+ words) Today we released inferact/tpu-megakernels, a collection of megakernels for TPU v7. Our Kimi K3 implementation delivers over 700 tokens/s with speculative decoding, compared with 452 tokens/s on GB200. Without speculative decoding, our megakernels for K3 and Qwen 3.8 27B deliver roughly 1.4 to 2× the decode throughput…...
A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x
11+ hour, 4+ min ago (1209+ words) This article provides a step by step guide to measuring a laptop GPU against the CPU in the same chassis for serving Gemma 4 E2B through llama.cpp, with both builds on a fresh Debian sid toolchain and the passes run in…...
Can vLLM Run GGUF? Yes — on GPU Only
11+ hour, 33+ min ago (357+ words) Two steps: install the plugin, then point vLLM at the model. GGUF support no longer ships inside core vLLM — the docs note it "has migrated to OOT vllm-gguf-plugin", so a plain pip install vllm is not enough: The --tokenizer flag…...
How Much VRAM Do You Really Need to Run a 70B LLM?
21+ hour, 41+ min ago (1186+ words) Running a large language model locally sounds simple until you start looking at GPU memory. A model has 70 billion parameters. Your GPU has 24 GB, 32 GB, 48 GB, or maybe 80 GB of VRAM. So will it fit? Unfortunately, parameter count alone does…...
The Laya-MLX model can run on 1GB of local memory, enabling fast decision-making.
1+ day, 15+ hour ago (922+ words) No, Jev has only been popular for a week—how did they already create a local version under 1GB?? A developer has ported the open-source alternative Laya native to Apple MLX, creating Laya-MLX— The 421M parameter version peaks at only 943.6 MiB of…...
Same Source, Two Cost Curves: Local Preview vs GPU Final
23+ hour, 10+ min ago (632+ words) Every video tool eventually grows two buttons that mean different economic things: Preview and Export. In timeline editors those buttons often hide divergent engines. Preview cheats. Export reinterprets. Someone notices during brand review when the fade is denser, the font…...
How to run Gemma 4 locally: a setup that survives
1+ day, 15+ hour ago (482+ words) Start with text, add GPU offload and vision carefully, then pin the command and runtime that keep Gemma 4…. Tagged with technology, gemma4, localllms, homelab....
Why GPU Availability Is Still the Biggest Bottleneck in ML Infra
1+ day, 19+ hour ago (513+ words) A few things are going on at once: Regional fragmentation. A specific GPU SKU (say, an A100 or H100) might be well-stocked in one region and completely unavailable in another. Teams end up either waiting, or provisioning in a region that adds…...
Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton
2+ day, 6+ hour ago (494+ words) The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability that enables a single TensorRT network to execute across multiple GPUs using NCCL-backed distributed collectives while…...
register_spark_column_namespace -
3+ day, 8+ hour ago (41+ words) Laktory Decorator for registering a custom namespace on PySpark's Column class... note:: This namespace is NATIVE-only. A Narwhals Expr bridge is not provided because Narwhals expressions and PySpark Columns have incompatible semantics. Use in a pipeline YAML:...