Skip to content

vLLM TPU

How it works

vLLM TPU

Home
Getting Started
Getting Started
Basic Usage (I use HuggingFace model)
Basic Usage (I use HuggingFace model)
- Overview & TPU
- Recommended Models
- Deploying model on GCE
  Deploying model on GCE
  - v7x setup (Ironwood)
  - v6e setup (Trillium)
- Code Examples
  Code Examples
  - Multi Modal Inference
  - Offline LoRA
Advanced Usage (I need my custom model)
Advanced Usage (I need my custom model)
- JAX
  JAX
  - How it works
  - Model Development
- Torch
  Torch
  - How it works
  - Model Development
- Your custom model on GCE
- Recommended Features
See Also
See Also

A mental model how JAX-based vLLM model runs on TPU¶

...