Glossaire des coûts LLM

A hard limit on token or dollar spend that blocks LLM requests once the threshold is reached.

The maximum number of tokens an LLM can process in a single request, including both input and output.

The per-request cost of running a prompt through an LLM, calculated from input and output token counts.

The balance between response speed and API cost when selecting an LLM for a given task.

Strategies and tools that reduce the total spend on LLM API calls without degrading application quality.

A proxy layer between your application and LLM providers that adds routing, caching, and cost controls.

The ability to monitor, trace, and analyze LLM API calls including tokens, costs, latency, and errors.

Automatically selecting the most cost-effective LLM for each request based on complexity and requirements.

Automatically switching to an alternative LLM when the primary model is unavailable or over budget.

Storing LLM responses for reuse when identical or similar prompts are submitted again.

A technique that shortens prompts by removing redundant tokens while preserving semantic meaning.

Controlling the frequency of LLM API calls to prevent abuse, manage costs, and stay within provider limits.

A caching strategy that reuses LLM responses for prompts that are semantically similar, not just identical.

The process of measuring how many tokens a text string will consume when sent to an LLM.

The process of reducing the number of tokens sent to an LLM without degrading output quality.