[Bug]: user_data_dir playwright launch_persistent_context not work

[cw1427]

crawl4ai version

v0.7.4

Expected Behavior

BrowserConfig user_data_dir worn't work, the crwal4ai launch a chrome browser without an existing context

Current Behavior

BrowserConfig user_data_dir worn't work, the crwal4ai launch a chrome browser without an existing context

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

linux

Python version

3.11

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

[SohamKukreti]

Hey, can you share the code for this?

[cw1427]

Hi yeah,

I already used playwright launch_persistent_context to finish authentication with user_dir path. And then try to use crawl4ai to reuse this user_dir path to access the web site which ask for login. But it seems crawl4ai backend used playwright didn't known the user_dir session lead to ask for login again.

from playwright.sync_api import sync_playwright with sync_playwright() as p: # Channel can be "chrome", "msedge", "chrome-beta", "msedge-beta" or "msedge-dev". browser = p.chromium.launch_persistent_context('D:/eclipsepython/workspace/pytest/playwright_ud', channel="chrome", proxy={'server':'***'}, headless=False) page = browser.new_page() page.goto("https://docs.partner.android.com/partners/guides/general/aap-connection") print(page.title()) html_content = page.content() # Get the HTML content as a string

Below crawl4ai faield.
async def main(): #browser, playwright_instance = setup_custom_playwright_browser() try: # --- Instantiate crawl4ai adapter with your browser --- run_config = CrawlerRunConfig() # Default crawl run configuration # Create an instance of AsyncWebCrawler browser_config = {"user_data_dir": 'D:/eclipsepython/workspace/pytest/playwright_ud', 'proxy': PROXY_ADDRESS, 'headless': False, 'use_persistent_context': True} #crawlerStrategy = AsyncPlaywrightCrawlerStrategy() bc = BrowserConfig(**browser_config) async with AsyncWebCrawler(config=bc, **browser_config) as crawler: print(crawler.crawler_strategy.browser_manager) # Run the crawler on a URL result = await crawler.arun(url="https://docs.partner.android.com/partners/guides/general/aap-connection", config=run_config) print(result.markdown)