DeepSeek V4 Flash across 14 providers: cost, speed and caching

The measurements

Choose the request shape and cache condition. Input sizes are targets and output sizes are budgets; actual token usage is retained in the downloadable records.

Timings describe successful calls. n is the number of observations behind each metric. Failure intervals are descriptive 95% Wilson intervals. “Not reported” means no measurement, not zero.

Which host wins on your prompts?

A chat agent, a document pipeline and a batch job stress different parts of this comparison. Use Compound to measure the hosts you are considering with your own messages.

1. Install Compound from the repository and set your provider API key.
2. Save your messages in shapes.json. Start with this structure and replace the sample prompt.
3. Preview the experiment, then add --go to execute it.

{
  "my-workload": {
    "messages": [{"role": "user", "content": "Replace this with a representative prompt."}],
    "max_tokens": 100
  }
}

uv sync --extra dev
# Discover available hosts for your chosen model
compound-bench providers YOUR_MODEL
# Compare automatic routing with a host from that list
compound-bench serving --model-or YOUR_MODEL \
  --providers openrouter/auto,openrouter/HOST \
  --shapes shapes.json --cache-mode both \
  --reasoning-modes off --temperature 0 --reps 5

The command previews the call count without spending. Replace YOUR_MODEL and HOST with discovery results, set OPENROUTER_API_KEY in .env, then add --go. Executed requests bill at provider rates; serving runs have no dollar cap.

Read the serving guide →, Get Compound on GitHub ↗

Explore the data

Download per-call timings, token usage, status codes and reported costs, or use the computed summaries. The manifest records the run windows and source provenance.

Per-call measurements ↓Summary data ↓Data manifest ↓