Skip to content

[Bug]: Docker server does not support headed crawling #1485

@Sjoeborg

Description

@Sjoeborg

crawl4ai version

0.7.4

Expected Behavior

Docker server is able to accept and perform crawls with headless: True

Current Behavior

The process crashes with "RuntimeError: Event loop is closed" after playwright says "Looks like you launched a headed browser without having a XServer running."

Is this reproducible?

Yes

Inputs Causing the Bug

- crwl https://example.com -b "headless=false"
- or by submitting a crawl via the REST API with headless: False

Steps to Reproduce

docker run --rm -it \
  --shm-size=1g \
  unclecode/crawl4ai:0.7.4 \
  crwl https://example.com -b 'headless=false'

Code snippets

OS

Linux

Python version

3.12

Browser

chromium

Browser version

N/A

Error logs & Screenshots (if applicable)

appuser@1f7af2dfdfb8:/app$ crwl https://example.com -b "headless=false"
Error: BrowserType.launch: Target page, context or browser has been closed
Browser logs:

╔════════════════════════════════════════════════════════════════════════════════════════════════╗
║ Looks like you launched a headed browser without having a XServer running.                     ║
║ Set either 'headless: true' or use 'xvfb-run <your-playwright-app>' before running Playwright. ║
║                                                                                                ║
║ <3 Playwright Team                                                                             ║
╚════════════════════════════════════════════════════════════════════════════════════════════════╝
Call log:
  - <launching> /home/appuser/.cache/ms-playwright/chromium-1181/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,ExtensionManifestV2Disabled,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,LensOverlay,MediaRouter,PaintHolding,ThirdPartyStoragePartitioning,Translate --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --force-color-profile=srgb --metrics-recording-only --no-first-run --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --edge-skip-compat-layer-relaunch --enable-automation --no-sandbox --disable-gpu --disable-gpu-compositing --disable-software-rasterizer --no-sandbox --disable-dev-shm-usage --no-first-run --no-default-browser-check --disable-infobars --window-position=0,0 --ignore-certificate-errors --ignore-certificate-errors-spki-list --disable-blink-features=AutomationControlled --window-position=400,0 --disable-renderer-backgrounding --disable-ipc-flooding-protection --force-color-profile=srgb --mute-audio --disable-background-timer-throttling --window-size=1080,600 --user-data-dir=/tmp/playwright_chromiumdev_profile-KUi5Uq --remote-debugging-pipe --no-startup-window
  - <launched> pid=128
  - [pid=128][err] [128:143:0916/072200.548997:ERROR:dbus/bus.cc:408] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
  - [pid=128][err] [128:128:0916/072200.550930:ERROR:ui/ozone/platform/x11/ozone_platform_x11.cc:249] Missing X server or $DISPLAY
  - [pid=128][err] [128:128:0916/072200.550950:ERROR:ui/aura/env.cc:257] The platform failed to initialize.  Exiting.

Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x798bbf97fa60>
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/base_subprocess.py", line 126, in __del__
    self.close()
  File "/usr/local/lib/python3.12/asyncio/base_subprocess.py", line 104, in close
    proto.pipe.close()
  File "/usr/local/lib/python3.12/asyncio/unix_events.py", line 767, in close
    self.write_eof()
  File "/usr/local/lib/python3.12/asyncio/unix_events.py", line 753, in write_eof
    self._loop.call_soon(self._call_connection_lost, None)
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 799, in call_soon
    self._check_closed()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 545, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
appuser@1f7af2dfdfb8:/app$

Metadata

Metadata

Assignees

No one assigned

    Labels

    🐞 BugSomething isn't working🩺 Needs TriageNeeds attention of maintainers

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions