Apache Superset Async with Celery: 2026 Guide

Celery is the async engine that lets Apache Superset execute long queries without blocking the UI, send scheduled reports, and trigger alerts. This guide details the architecture, configuration, and tuning of Celery for 2026.

1. What is Celery for in Superset?

Four major features depend on Celery:

1. Async queries: SQL Lab and long charts execute without blocking the browser;
2. Scheduled reports: periodic dashboard email sending;
3. Alerts: conditional notifications;
4. Cache warming: pre-warming the cache after ETL.

Without Celery configured, these features are unavailable or very limited. TVL Managed Superset enables Celery by default on all instances.

2. Architecture

Component	Role	Replicas
Web (gunicorn)	Receives the request, pushes to queue	2-3
Broker (Redis)	Task queue	1 (HA via Sentinel)
Worker (celery worker)	Executes tasks	2-N depending on load
Beat (celery beat)	Periodic scheduler	1 (singleton)

3. Basic configuration

# superset_config.py
from datetime import timedelta

class CeleryConfig:
    broker_url = "redis://redis:6379/0"
    result_backend = "redis://redis:6379/0"
    imports = (
        "superset.sql_lab",
        "superset.tasks.scheduler",
        "superset.tasks.thumbnails",
    )
    worker_prefetch_multiplier = 1
    task_acks_late = True
    task_annotations = {
        "sql_lab.get_sql_results": {"rate_limit": "100/s"},
    }
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": timedelta(minutes=1),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": timedelta(days=1),
        },
    }

CELERY_CONFIG = CeleryConfig

# Enable async queries
GLOBAL_ASYNC_QUERIES_JWT_SECRET = os.environ["GAQ_SECRET"]
FEATURE_FLAGS = {
    "GLOBAL_ASYNC_QUERIES": True,
    "ALERT_REPORTS": True,
    "SCHEDULED_QUERIES": True,
}

4. Launch worker and beat

# worker
celery --app=superset.tasks.celery_app:app worker --loglevel=INFO --concurrency=4

# beat (singleton, do not duplicate)
celery --app=superset.tasks.celery_app:app beat --loglevel=INFO

On Kubernetes, two distinct Deployments: one for workers (scalable replicas), one for beat (replica fixed at 1).

5. Worker sizing

Load	Replicas	Concurrency
50 occasional users	1	4
200 active users	2	4-8
500+ users	4-6	8 (and HPA autoscaling)
Multi-tenant SaaS	Variable	HPA based on queue length

This configuration is applied by default on TVL Managed Superset, which follows community best practices.

6. The singleton beat trap

Beat must run in single instance, otherwise jobs are triggered twice. Three options:

• StatefulSet 1 replica + auto-restart (acceptable);
• celery-redbeat with Redis leader election: allows standby;
• RedBeat on Postgres: less common, more complex.

7. Async queries (GLOBAL_ASYNC_QUERIES)

With this feature flag, long queries from SQL Lab and charts no longer block the browser. The client polls the status regularly. Essential beyond 50 concurrent users.

8. Scheduled reports and alerts

Scheduled reports generate dashboard PNGs/PDFs and send them by email (SMTP) or Slack. Configuration:

EMAIL_REPORTS_WEBDRIVER = "chrome"
WEBDRIVER_BASEURL = "https://superset.example.com/"
WEBDRIVER_BASEURL_USER_FRIENDLY = WEBDRIVER_BASEURL

# SMTP
SMTP_HOST = "smtp.example.com"
SMTP_PORT = 587
SMTP_STARTTLS = True
SMTP_USER = "alerts@example.com"
SMTP_PASSWORD = os.environ["SMTP_PASSWORD"]
SMTP_MAIL_FROM = "alerts@example.com"

# Slack (webhook)
SLACK_API_TOKEN = os.environ.get("SLACK_API_TOKEN")

Celery workers must include headless Chrome for PDF generation (or use Playwright as alternative).

9. Celery monitoring

Metrics to watch:

• Queue length (alert if >100);
• Task duration p95;
• Task failure rate;
• Active workers.

Tools: Flower (Celery UI), celery-exporter + Prometheus, centralized logs.

10. Common pitfalls

• No result_backend: async results are lost;
• Duplicated beat: alerts sent N times;
• Worker timeout too short: long tasks killed;
• Without SQL Lab rate limit: a user can saturate the broker;
• Missing Chrome in worker: PDF reports silently fail.

11. Conclusion

Celery is the component that turns Superset into a productive platform: without it, no scheduled reports, no alerts, no long queries. Its configuration is straightforward but requires following a few rules (singleton beat, robust broker, dedicated monitoring).

Want the benefits of Apache Superset without the friction of installation and maintenance? Deploy your instance in 3 clicks with TVL Managed Superset, hosted in Europe (OVHcloud, Roubaix, France), pre-configured Celery included.

For more: Redis cache, scheduled reports, Superset alerts.