Hi guys, Welcome to the Advanced Video Search Engine, and you are watching Fast And Portable LLM Inference With WebAssembly And Rust by Michael Yuan. That video is uploaded by LambdaConf at 2024-07-08T02:19:06-07:00. We are promoting this video only for entertainment and educational purpose only. So, I hope you like & enjoy it. [uqYj7auwOzk]
| Name | Fast And Portable LLM Inference With WebAssembly And Rust by Michael Yuan |
| Video Uploader | Video From LambdaConf |
| Upload Date | This Video Uploaded At 08-07-2024 09:19:06 |
| Video Discription | Fast And Portable LLM Inference With WebAssembly And Rust by Michael Yuan at #LambdaConf2024 Get your ticket for #LambdaConf2025 here: https://www.eventbrite.com/e/lambdaconf-2025-tickets-903567092497 It is estimated that inference consumes over 90% of computing resources for AI workloads. The heavy resource consumption is exacerbated with LLMs, as we see companies struggle to meet customers’ inference demands. Traditional AI inference apps are written in Python and then wrapped in a container or VM for cloud deployment. Those apps are heavyweight (10GB+) and slow with Python-based data processing. Wasm has emerged as a strong alternative runtime for AI inference workloads. Developers write inference functions in Rust / JS / Python and then run them in Wasm sandboxes. Wasm functions are lightweight, fast, safe for the cloud, and portable. In this talk, Michael will talk about the way to zero python dependency in LLM inference and how to create Llama2 inference functions and extensions in Rust. Michael will also discuss how the WasmEdge community has leveraged and built up the Wasm container infra for LLM plugins. uqYj7auwOzk |
| Category | Nonprofits & Activism |
| Tags | LambdaConf | Conference | Functional Programming |
