WebLLM
WebLLM is a high-performance, in-browser language model inference engine that leverages WebGPU for hardware acceleration, enabling powerful LLM operations directly within web browsers without server-side processing. It provides full OpenAI API compatibility including JSON-mode and function-calling, supports diverse models like Llama and Mistral, and allows custom model integration. Features include Web Worker support, streaming, and npm integration.
Checked on Nov 2, 2026.
Alternative resources
Run open-source LLMs directly in your browser using WebGPU for client-side local inference without external server dependencies.
Test open source machine learning models directly in the browser using WebGPU without installation or accounts.
Run open-source large language models locally or via cloud infrastructure with API integration for coding agents and development workflows.