Skip to content

Fix: PrometheusMiddleware exports metrics from only one worker when --workers > 1#594

Merged
s3rius merged 1 commit intotaskiq-python:masterfrom
sergey-chek:fix/prometheus-multiprocess-collector
Feb 19, 2026
Merged

Fix: PrometheusMiddleware exports metrics from only one worker when --workers > 1#594
s3rius merged 1 commit intotaskiq-python:masterfrom
sergey-chek:fix/prometheus-multiprocess-collector

Conversation

@sergey-chek
Copy link
Contributor

Fixes #590

Problem

start_http_server() in startup() uses the default REGISTRY, which only contains metrics from the current process. With --workers 2, one worker binds port 9000 and serves only its own metrics, the other workers' mmap files are never read.

Root Cause

Each worker process writes its metrics to a separate mmap file (counter_{pid}.db) in PROMETHEUS_MULTIPROC_DIR - this part works correctly. However, start_http_server() without the registry= parameter serves from the default REGISTRY, which only calls Counter.collect() / Histogram.collect() on the current process's values (source).

Per prometheus_client multiprocess docs, a separate CollectorRegistry with MultiProcessCollector is required to aggregate mmap files from all processes.

Fix

  • Pass CollectorRegistry + MultiProcessCollector to start_http_server() in startup()
  • Remove duplicate os.environ assignment (line 42)

Testing

Tested with --workers 2 on taskiq 0.12.1:

  • Before fix: 5 tasks executed (PID 10: 3, PID 11: 2) -> received_tasks_total = 3.0 (one worker only)
  • After fix: 12 tasks executed (PID 10: 7, PID 11: 5) -> received_tasks_total = 12.0 (all workers aggregated)

@s3rius s3rius merged commit d1fa98f into taskiq-python:master Feb 19, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PrometheusMiddleware /metrics endpoint only serves metrics from one worker process

2 participants

Comments