🚀 Build a Redis-Powered API with Atomic Operations, Rate Limiting, and Real-Time Metrics — Complete Docker Tutorial

Deep Dive: Production-Ready API Control & Observability

• Running the stack
• Using the Redis CLI to validate atomic features
• Hitting every API endpoint with curl (with realistic scenarios)
• Watching keys update in Redis Commander (or via redis-cli)
• Verifying Prometheus scraping & running PromQL queries
• Exploring the Grafana dashboard and reading the graphs intelligently

Use this as your live demo script or QA checklist. Everything is self-contained.

0) Start & Check the Stack

# from the project root
docker compose up --build

Wait for:

• api: “Uvicorn running on http://0.0.0.0:8001”
• redis-commander: “listening on 0.0.0.0:8081”
• prometheus: “Server is ready” on 9090
• grafana: starting on 3000

Open:

• API docs: http://localhost:8001/docs
• Redis Commander: http://localhost:8081
• Prometheus: http://localhost:9090
• Grafana: http://localhost:3000 (login admin/admin)

1) Redis CLI: Validate Core Features (Atomicity, TTL, Data Types)

Enter Redis CLI (inside the container):

docker compose exec redis redis-cli

You’ll see 127.0.0.1:6379>.

1.1 Basic Key-Value

SET greeting "Hello Redis"
GET greeting
DEL greeting
EXISTS greeting   # → (integer) 0 after DEL

• Validates storage, retrieval, deletion, and existence.

1.2 Atomic Counters

DEL page_views
INCR page_views           # -> 1
INCRBY page_views 5       # -> 6
DECR page_views           # -> 5
GET page_views            # -> "5"

• Atomic guarantee: no race conditions even under concurrency.

1.3 TTL / Expiry

SET temp "bye"
EXPIRE temp 5
TTL temp                  # shows countdown (e.g., 5,4,3…)
# wait ~5s
GET temp                  # -> (nil)

• Essential for caches, sessions, fixed window counters.

1.4 Lists (Queue vs Stack)

DEL tasks
LPUSH tasks "job1"
LPUSH tasks "job2"
LRANGE tasks 0 -1          # -> "job2","job1"
RPOP tasks                 # FIFO behavior with LPUSH/RPOP
LRANGE tasks 0 -1

1.5 Sets (Uniqueness)

DEL users
SADD users "sree"
SADD users "abhi"
SADD users "sree"          # ignored duplicate
SMEMBERS users
SCARD users                # count stays 2

1.6 Sorted Sets (Sliding Window Base)

DEL visits
ZADD visits 1001 "req1"
ZADD visits 1002 "req2"
ZRANGE visits 0 -1 WITHSCORES
ZREMRANGEBYSCORE visits 0 1001
ZRANGE visits 0 -1 WITHSCORES

1.7 Hashes (Structured)

DEL user:100
HSET user:100 name "Sree" plan "Gold" credits 10
HGETALL user:100
HINCRBY user:100 credits 5
HGET user:100 credits      # -> "15"

1.8 Transactions (WATCH / MULTI / EXEC)

Terminal A:

SET stockA 10
SET stockB 5
WATCH stockA stockB
GET stockA                 # make your check in app code in real life
MULTI
DECRBY stockA 3
INCRBY stockB 3
EXEC                       # succeeds unless someone changed the watched keys

Terminal B (to force abort, run between WATCH and EXEC in A):

INCR stockA

• In A, EXEC returns null → safe abort (no partial updates). Then retry.

1.9 Pub/Sub (Real-time)

Terminal 1:

SUBSCRIBE alerts

Terminal 2:

PUBLISH alerts "Redis is awesome!"

• Message appears immediately in Terminal 1.

1.10 Lua Scripting (Atomic Logic)

DEL balance
SET balance 100
EVAL "local bal=tonumber(redis.call('GET',KEYS[1]) or '0');local amt=tonumber(ARGV[1]);if bal>=amt then redis.call('DECRBY',KEYS[1],amt);return bal-amt else return -1 end" 1 balance 30
GET balance               # -> "70"

• Entire check-then-debit runs atomically server-side.

1.11 Safe Exploration (SCAN)

SCAN 0 MATCH * COUNT 10
TYPE stockA

• Prefer SCAN to KEYS * in real systems.

1.12 Monitoring & Insights

INFO memory
INFO stats
INFO commandstats
MONITOR                   # live stream of all commands (Ctrl+C to exit)

• Run this while you hit API endpoints to see the keys changing live.

1.13 Reset (for test only)

FLUSHALL                  # ⚠ Destroys ALL keys – dev/test only

2) Hit Every API Endpoint (with Explanations & Expected Behavior)

Use these curl tests in a separate terminal. Keep Redis Commander open to watch keys appear (or keep redis-cli MONITOR running).

2.1 Health

curl -s http://localhost:8001/health
# {"ok": true}

• Confirms API ↔ Redis connectivity.

2.2 Atomic Counter Endpoints

curl -s "http://localhost:8001/counter/incr?key=page_views"
curl -s "http://localhost:8001/counter/get?key=page_views"

• Uses Redis INCR atomically.
• Expect values to increase monotonically; no race conditions.

2.3 Stock Transfer (WATCH/MULTI/EXEC)

Seed:

curl -s -X POST "http://localhost:8001/stock/seed?item=a&qty=50"
curl -s -X POST "http://localhost:8001/stock/seed?item=b&qty=10"

Transfer:

curl -s -X POST "http://localhost:8001/stock/transfer" \
  -H "Content-Type: application/json" \
  -d '{"from_item":"a","to_item":"b","qty":7}'

Check:

curl -s "http://localhost:8001/stock/get?item=a"
curl -s "http://localhost:8001/stock/get?item=b"

• Ensures atomic multi-key move with optimistic locking.

2.4 Wallet Debit (Lua)

curl -s -X POST "http://localhost:8001/wallet/seed?user=sree&amount=100"
curl -s -X POST "http://localhost:8001/wallet/debit?user=sree&amount=30"
curl -s -X POST "http://localhost:8001/wallet/debit?user=sree&amount=100"

• First debit succeeds (balance 70), second fails if insufficient.
• All logic runs atomically in Redis via Lua.

2.5 Rate Limiting — Composite Keys (X-API-Key + client IP)

The service uses X-Forwarded-For if present; else it uses your client’s IP. All strategies generate keys per <apiKey>:<ip> to be fair.

a) Fixed Window (Defaults: 5 requests / 10s)

for i in {1..10}; do
  curl -s -H "X-API-Key: demo123" "http://localhost:8001/rate/fixed";
done

• Expect ~5 allowed, then 429s until TTL expires.
• In Redis Commander, filter rl:fixed:* → watch value & TTL.

b) Sliding Window (Defaults: 8 requests / 10s)

for i in {1..12}; do
  curl -s -H "X-API-Key: demo123" "http://localhost:8001/rate/sliding";
done

• Expect smoother fairness, no boundary burst.
• Inspect key rl:slide:<apiKey>:<ip> → ZSET of timestamps.

c) Token Bucket (Customizable)

for i in {1..15}; do
  curl -s -H "X-API-Key: gold" \
    "http://localhost:8001/rate/tokenbucket?capacity=10&refill_per_sec=2&cost=1";
done
sleep 3
for i in {1..5}; do
  curl -s -H "X-API-Key: gold" \
    "http://localhost:8001/rate/tokenbucket?capacity=10&refill_per_sec=2&cost=1";
done

• First ~10 pass (consumes tokens), next blocked; after 3s, ~6 tokens refill.
• Redis keys: rl:tb:tokens:<key> and rl:tb:ts:<key>.

d) Composite Identity Behavior

Simulate different IPs:

curl -s -H "X-API-Key: demo123" -H "X-Forwarded-For: 1.2.3.4" "http://localhost:8001/rate/fixed"
curl -s -H "X-API-Key: demo123" -H "X-Forwarded-For: 5.6.7.8" "http://localhost:8001/rate/fixed"

• Different composite keys → independent limits.

3) Watch Redis Keys Live (Two Choices)

3.1 Redis Commander (UI)

• Open http://localhost:8081

• If you don’t see a text filter, use Commands → Find Keys (SCAN) and enter:

  • rl:fixed:*
  • rl:slide:*
  • rl:tb:*

• Click a key to view: values, TTL, ZSET members, etc.

• Refresh to see TTL countdown and expirations.

3.2 redis-cli (Terminal)

In a separate terminal:

docker compose exec redis redis-cli MONITOR

• Now hit API endpoints → watch commands stream live:

  • INCR for fixed window
  • ZADD/ZREMRANGEBYSCORE for sliding
  • EVAL for token bucket Lua

4) Prometheus: Verify Scrapes & Query Metrics

4.1 Validate /metrics

curl -s http://localhost:8001/metrics | head -n 30

• You’ll see:

  • requests_total{endpoint="/rate/fixed",status="200",strategy="fixed"} ...
  • rate_limited_total{strategy="sliding"} ...
  • request_latency_seconds_bucket{le="0.005"} ...

4.2 Prometheus Targets

• Open http://localhost:9090 → Status → Targets You should see api:8001 UP.

4.3 Run PromQL Queries (Prometheus “Graph” tab)

Allowed RPS by endpoint (1m):

sum by (endpoint) (rate(requests_total{status="200"}[1m]))

Blocked RPS by strategy (1m):

sum by (strategy) (rate(rate_limited_total[1m]))

P90 and P99 latency:

histogram_quantile(0.90, sum(rate(request_latency_seconds_bucket[5m])) by (le))
histogram_quantile(0.99, sum(rate(request_latency_seconds_bucket[5m])) by (le))

Requests for a specific endpoint (e.g., token bucket):

sum(rate(requests_total{endpoint="/rate/tokenbucket"}[1m]))

• Now hammer the endpoints again and re-run the queries; you’ll see lines spike in real time.

5) Grafana: Read the Dashboard Like a Pro

5.1 Open Dashboard

• http://localhost:3000 → login admin/admin
• Browse to Redis Rate Limiter (API) dashboard.

5.2 Panels & What to Expect

• Total Requests (5m) Increases as you fire curl loops.

• Total Rate-Limited (5m) Jumps when you exceed limits (429s).

• Requests by Endpoint & Status (1m rate)

  • Green lines = allowed (200)
  • Red lines = blocked (429) Hammer /rate/fixed: you’ll see green plateau at your limit and red rise as blocks kick in.

• Rate-Limited by Strategy (1m rate) Compare fixed vs sliding vs token bucket.

  • Sliding shows fewer spikes at boundaries
  • Token bucket shows burst capacity then steady-rate enforcement

• Latency P50/P90/P99 Should remain very low (Redis ops are fast). Spikes hint at contention, network issues, or resource caps.

5.3 Time & Refresh

• Top right: set Last 15 minutes and Refresh every 5s to watch changes live.

6) Optional: Concurrency Blasts

6.1 Quick & Dirty Concurrency (macOS/Linux)

# 20 concurrent hits to fixed
seq 1 20 | xargs -n1 -P20 -I{} curl -s -H "X-API-Key: demo123" http://localhost:8001/rate/fixed >/dev/null

# 20 concurrent hits to sliding
seq 1 20 | xargs -n1 -P20 -I{} curl -s -H "X-API-Key: demo123" http://localhost:8001/rate/sliding >/dev/null

6.2 Built-in Load Script (if included)

python api/app/scripts/load_test.py "http://localhost:8001/rate/fixed" 25
python api/app/scripts/load_test.py "http://localhost:8001/rate/sliding" 30
python api/app/scripts/load_test.py "http://localhost:8001/rate/tokenbucket?capacity=10&refill_per_sec=2&cost=1" 30

• Watch Grafana and Prometheus refresh; observe differences among strategies.

7) Troubleshooting Quick Guide

• No keys in Redis Commander

  • Use Commands → Find Keys (SCAN) with rl:* or switch DB to 0. Click Refresh.

• Prometheus shows DOWN

  • Check API logs and that /metrics is reachable: curl -s http://localhost:8001/metrics | head

• Grafana panels empty

  • Prometheus data source must be connected (auto-provisioned).
  • Time range too short? Set to Last 15m and refresh.

• All 429s

  • You may be reusing the same composite key; try a different X-API-Key or simulate another IP with X-Forwarded-For.

• Token bucket not refilling

  • Wait long enough (refill is per second). Check rl:tb:tokens:* value in Redis Commander; ensure capacity and refill_per_sec query params are set.

✅ What you’ve demonstrated

• Redis atomic operations (INCR, Lua)
• Optimistic transactions with WATCH/MULTI/EXEC
• Three rate limiting strategies with composite identity
• Live observability via Prometheus & Grafana (RPS, 429s, latency)
• Real-time Redis state via Redis Commander or MONITOR

If you want, I can package this into a single “Test Playbook” PDF with checkboxes and a one-page Grafana legend for workshops and demos.