Performance

Tokens per second

TPS stands for Tokens per second. How fast a model streams out its answer once it starts. Higher means the text appears more quickly.

TPS

How fast a model streams out its answer once it starts. Higher means the text appears more quickly.

In practice

At 80 tokens per second, a 400-word answer finishes in about six seconds.

Throughput Time to first token Streaming

Track usage and spend across every model and platform, free.

Start tracking free See the AI Cost Index

Image: Mathias Reding on Pexels. Definition free to reuse under CC BY 4.0.