智能体工作流

serving-llms-vllm

Name: serving-llms-vllm
Author: orchestra-research

orchestra-research/ai-research-skills

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

1078 次安装
1.4万星标
收录于 2026.01.21

安装命令

$ npx skills add https://github.com/orchestra-research/ai-research-skills --skill serving-llms-vllm

SKILL.md

vLLM - High-Performance LLM Serving

Quick start

vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill/decode requests).

Installation:

pip install vllm

Basic offline inference:

from vllm import LLM, SamplingParams

llm = LLM(model="meta-llama/Llama-3-8B-Instruct")
sampling = SamplingParams(temperature=0.7, max_tokens=256)

SKILL.md

vLLM - High-Performance LLM Serving

Quick start

相关 Skills