How to Test throughput and latency to API?

Testing API throughput and latency is about measuring how fast and how much your API can handle under different conditions. Below is a practical, production-style guide with tools, metrics, and examples—especially relevant if you’re working with .NET / web APIs.

1. Key Definitions (Clear Difference)

Latency

Time taken for one request → one response

Measured in:

• ms (milliseconds)
• P50, P90, P95, P99 (percentiles)

Example:

P50 = 120ms
P95 = 450ms

Throughput

Number of requests your API can handle per second

Measured in:

• RPS (Requests Per Second)
• TPS (Transactions Per Second)

Example:

Throughput = 2,500 RPS

2. What You Should Measure

3. Simple Local Testing (Quick Checks)

Using curl (Latency only)

curl -w "\nTime_Total: %{time_total}s\n" -o /dev/null -s https://api.example.com/users

Good for:

• One request
• Network + server latency

4. Apache Benchmark (ab)

Install

sudo apt install apache2-utils

Test example

ab -n 10000 -c 100 https://api.example.com/users

Where:

• -n = total requests
• -c = concurrent users

Output to focus on

Requests per second: 2300.15 [#/sec]
Time per request: 43.2 ms

5. k6 (Highly Recommended – Production Grade)

Best for throughput + latency + load patterns

Install

choco install k6       # Windows
brew install k6        # macOS

k6 Latency Test

import http from 'k6/http';
import { sleep } from 'k6';

export let options = {
  vus: 50,
  duration: '30s',
};

export default function () {
  http.get('https://api.example.com/users');
  sleep(1);
}

Run:

k6 run test.js

Metrics you get:

• avg latency
• p90 / p95 / p99
• RPS
• failure %

k6 Throughput Stress Test

export let options = {
  stages: [
    { duration: '30s', target: 100 },
    { duration: '1m', target: 500 },
    { duration: '30s', target: 0 },
  ],
};

Shows:

• breaking point
• max sustainable throughput

6. JMeter (GUI + Heavy Testing)

Best when:

• QA teams
• Complex workflows
• Auth flows

Steps:

• Thread Group → Users
• HTTP Request → API
• Summary Report
• Aggregate Report

Key Metrics:

• Throughput
• 90% Line
• Error %

7. Postman (Beginner-Friendly)

Collection Runner

• Set iterations
• Set concurrency
• View avg response time

Not accurate for high throughput testing

8. Production-Like Testing (Best Practice)

Run tests from:

• Same region as users
• Separate machine (not API server)

Monitor during test:

• API CPU
• Memory
• GC pauses (.NET)
• SQL query time
• Thread pool starvation

9. Example Targets (Reference Benchmarks)

10. Common Mistakes to Avoid

• Testing from localhost
• Ignoring P95/P99
• Not warming up API
• Forgetting DB limits
• Running load test on production without rate limits

11. Recommended Tool Stack (Professional)

12. Want This Tailored to You?

I can:

• Create k6 scripts for your specific API
• Help interpret test results
• Recommend rate limits & scaling strategy
• Tune ASP.NET MVC / Web API performance