Skip to content

Scaling LLM Batch Inference: Ray Data & vLLM for High Throughput

InfoQ234K subscribers

About this video

Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by video

Covers LLM batch inference optimization using Ray Data and vLLM. This teaches specific tools and techniques for scaling inference workloads.

Video by InfoQ, played through YouTube's embedded player.

Part of QCon San Francisco