Scaling LLM Batch Inference: Ray Data & vLLM for High Throughput
About this video
Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by videoCovers LLM batch inference optimization using Ray Data and vLLM. This teaches specific tools and techniques for scaling inference workloads.
Video by InfoQ, played through YouTube's embedded player.
Part of QCon San Francisco