Skip to content

Conversation

@trianglesis
Copy link

@trianglesis trianglesis commented Jun 10, 2025

Description

Trying the variant where all queues and exchanges are durable by default.

Discussed at issue 2237
RabbitMQ recommendation: DOC
Related to the celery PR.

Conf: cat /etc/rabbitmq/rabbitmq.conf

# https://www.rabbitmq.com/docs/deprecated-features
deprecated_features.permit.transient_nonexcl_queues = false

A pretty simple fix just changed booleans to durable=True

Tested in my DEV environment.
Now running at prod version.

No issues with tasks, queues, managing and pinging/inspecting queues and workers.

I didn't run bundled tests, however.
Only relied upon my envs.

My celery conf
def celery_setup():
    # Get creds and hosts
    setup_celery_env()
    # Setup django project
    django.setup()
    #  The backend is specified via the backend argument to Celery
    global app
    if app is None:
        app = Celery('server',
                     broker=RabbitMQCreds.BROKER,
                     backend=backend,
                     )

    app.conf.timezone = 'UTC'
    app.conf.enable_utc = True
    # General config:
    app.conf.update(
        result_backend=result_backend,
        accept_content=['pickle'],
        task_serializer='pickle',
        result_serializer='pickle',
        result_extended=True,
        task_track_started=True,
        beat_scheduler='django_celery_beat.schedulers:DatabaseScheduler',
        database_engine_options={'pool_timeout': 90},
        worker_prefetch_multiplier=1,
        worker_concurrency=1,
        worker_timer_precision=1.0,
        broker_heartbeat=10.0,
        broker_heartbeat_checkrate=2.0,
    )

    app.autodiscover_tasks()
    app.conf.update(
        worker_max_memory_per_child=1024 * 100,
        worker_max_tasks_per_child=100,
        worker_proc_alive_timeout=5.0,
        task_acks_late=False,
        task_acks_on_failure_or_timeout=True,
        worker_cancel_long_running_tasks_on_connection_loss=True,
        task_reject_on_worker_lost=False,
        worker_pool_restarts=True,
        worker_enable_remote_control=True,
        worker_lost_wait=20,
        broker_connection_retry_on_startup=True,
        broker_connection_retry=True,
        broker_connection_max_retries=0,
        broker_connection_timeout=4.0,
        broker_channel_error_retry=True,
        broker_pool_limit=100,
        task_send_sent_event=True,
        worker_send_task_events=True,
        worker_disable_rate_limits=True,
        task_default_queue='[email protected]',
        task_default_exchange='someserver',
        task_default_routing_key='[email protected]',
        worker_direct=True,
        result_persistent=True,
        task_default_delivery_mode='persistent',
    )

@auvipy
Copy link
Member

auvipy commented Jun 10, 2025

Thanks

@auvipy
Copy link
Member

auvipy commented Sep 8, 2025

handled in another pr

@auvipy auvipy closed this Sep 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants