Skip to content

CloakBrowser 0.5.8 — False-negative visible actionability check on dropdown menu items #512

Description

@thesob

Bug Report: CloakBrowser 0.5.8 — False-negative visible actionability check on dropdown menu items


Environment

cloakbrowser (wrapper) 0.5.8 (regression) — confirmed working on 0.4.11
Browser binary v146 (free tier)
Playwright 1.61.0
Python 3.12.13
OS Linux 6.8.0-137-generic x86_64 (Docker container, glibc 2.41)
Launch mode launch_persistent_context(..., headless=False, humanize=True, viewport={"width": 1920, "height": 1080})

Summary

After upgrading only the cloakbrowser wrapper from 0.4.110.5.8 (base image + pip package, no other code changes), a previously-reliable page.click("text=Hoy") call started failing 100% of the time with ElementNotVisibleError, even though the target element is demonstrably visible and rendered on screen at the exact moment of the check.

Steps to reproduce (site-specific, described since it's on a private/authenticated site)

Open an open, absolutely-positioned dropdown menu (.daterangepicker.dropdown-menu, display: block) containing a list of <li> options, e.g.:

<div class="daterangepicker dropdown-menu opensright" style="display: block; ...">
  <div class="ranges">
    <ul>
      <li class="active">Hoy</li>
      <li>Últimos 7 Días</li>
      <li>Mes Actual</li>
      ...
    </ul>
  </div>
  <div class="calendar left">...</div>
</div>

Call page.click("text=Hoy", timeout=10000) (via the humanized wrapper, humanize=True).
3. Observe ElementNotVisibleError: Element 'text=Hoy' failed visible check: element is not visible after the full timeout elapses.
4. Retry with force=True → this skips ensure_actionable() entirely but still calls scroll_to_element(), which now raises RuntimeError: Element not found while scrolling into view because get_box() returns None.

Expected behavior

The click should succeed (as it reliably did on 0.4.11), since the element:

  • Is attached to the DOM
  • Has non-zero size (visually rendered, orange-highlighted active state)
  • Is within the viewport
  • Is not covered by any other element

Actual behavior

_stealth_actionable() in cloakbrowser/human/actionability.py reports visible: false for the entire 10-second retry window (backoff loop [100, 250, 500, 1000]ms), even though:

  • A debug screenshot taken by our own error handler at the exact moment of the ElementNotVisibleError shows the "Hoy" <li> fully rendered, highlighted, and in the same visual state as a manual browser inspection of the identical dropdown (screenshots available, see "Attachments" below).
  • The manual (non-automated) inspection of the same page/dropdown, in a separate browser session, shows the element in an ordinary, clearly visible state (class=&quot;active&quot;, orange background) — no unusual CSS (opacity, transform, visibility: hidden, display: none) is present.

This strongly suggests a false negative in the in-world visibility computation added/changed in 0.5.x, rather than a real rendering issue.

Comparison across versions (from our production logs)

  • Last run on 0.4.11: Clicking filter option 'Hoy'... → succeeded in < 1 second.
  • First run on 0.5.8 (same code path, no other changes deployed yet): Clicking filter option 'Hoy'... → failed after ~30s (default click timeout) with the identical ElementNotVisibleError.
  • Reproduced identically across 4 separate runs on 0.5.8 (different sessions, different persistent profile states), always at the same step.
  • Reverting the wrapper + base image back to 0.4.11 (no other code changes) immediately restored the previous reliable behavior.

Suspected root cause / area to investigate

  • cloakbrowser/human/actionability.py::_stealth_actionable() — the in-world (page._stealth_world) evaluation of build_actionable_js(selector) for the &quot;visible&quot; check.
  • Possibly related to the 0.5.x changelog items "safer humanized pre-click checks" (0.5.7) and "safer humanized behavior after in-page navigation ... scroll refinements" (0.5.8) — a change here may have altered how visibility/geometry is computed for elements inside dropdowns that are:
    • Absolutely/fixed-positioned relative to a trigger element rather than in normal document flow,
    • Rendered via a toggled display: block (Bootstrap-style dropdown-menu) rather than being always in the DOM,
    • Possibly cached/stale in the isolated-world context (page._stealth_world) across an earlier navigation or DOM mutation in the same page session (we also removed an unrelated banner via page.evaluate() shortly before this click, in case that's a relevant trigger for isolated-world staleness — see stealth.invalidate() calls tied to framenavigated in __init__.py, which wouldn't fire here since there's no navigation, only a DOM mutation).

Suggested fix / workaround for the CloakBrowser team

  1. Add a regression test with a Bootstrap-style toggled dropdown-menu (display: nonedisplay: block, no navigation involved) to the actionability test suite, asserting the in-world visible check agrees with Playwright's own locator.is_visible()/bounding_box() for such elements.
  2. Consider falling back to Playwright's own is_visible()/bounding_box() as a cross-check when the in-world read reports not visible but the element has a non-null, non-zero bounding box — i.e., only trust "not visible" from the isolated world when geometry is also absent/zero, to avoid false negatives like this one.
  3. If the isolated world (page._stealth_world) can go stale after in-page DOM mutations that aren't full navigations (e.g. page.evaluate() removing an unrelated banner elsewhere on the page), consider invalidating/re-syncing it on such mutations too, not just on framenavigated.

Suggested workaround for our own use (already applied)

Pinned back to cloakbrowser==0.4.11 / cloakhq/cloakbrowser:0.4.11 until this is fixed upstream.

Attachments

CloakBrowser 0.5.8 — false-negative 'visible' actionability check
====================================================================

Environment
-----------
cloakbrowser (wrapper):  0.5.8   (regression)
Known-good version:      0.4.11 (confirmed working, same code, same site)
Browser binary:          v146 (free tier)
Playwright:              1.61.0
Python:                  3.12.13
OS:                      Linux 6.8.0-137-generic x86_64 (Docker, glibc 2.41)
Launch options:          launch_persistent_context(
                             PROFILE_DIR,
                             headless=False,
                             humanize=True,
                             viewport={"width": 1920, "height": 1080},
                             accept_downloads=True,
                             args=['--start-maximized'],
                         )

Failing call
------------
page.click("text=Hoy", timeout=10000)

... where "Hoy" is a <li class="active">Hoy</li> inside an open
Bootstrap-style dropdown menu (.daterangepicker.dropdown-menu.opensright,
style="display: block; ..."), toggled open by an earlier click on its
trigger element (not a page navigation).

A screenshot taken via page.screenshot() at the exact moment of each
failure below shows the "Hoy" element fully rendered, orange-highlighted,
and unobstructed -- visually identical to a manual (non-automated)
inspection of the same dropdown in a separate, unrelated browser session.

====================================================================
Occurrence 1 -- job started 2026-08-20T21:02:24.358196Z
====================================================================
2026-08-20 21:02:xx [INFO] Clicking filter option 'Hoy'...
2026-08-20 21:02:xx [WARNING] Click failed on 'Hoy': Element 'text=Hoy'
    failed visible check: element is not visible, retrying force click
    after overlay check...
2026-08-20 21:03:04.397 [ERROR] Error during scraping: Element not found
    while scrolling into view

Traceback (most recent call last):
  File "/app/scraper.py", line 463, in run_scraper
    page.click("text=Hoy", timeout=10000)
  File "/app/cloakbrowser/human/__init__.py", line 1081, in _human_click
    ensure_actionable(page, selector, CHECKS_CLICK, timeout=_remaining_ms(), force=force)
  File "/app/cloakbrowser/human/actionability.py", line 163, in ensure_actionable
    raise last_error
  File "/app/cloakbrowser/human/actionability.py", line 169, in ensure_actionable
    if not _stealth_actionable(page, selector, checks):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/actionability.py", line 109, in _stealth_actionable
    raise ElementNotVisibleError(selector)
cloakbrowser.human.actionability.ElementNotVisibleError: Element 'text=Hoy' failed visible check: element is not visible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/scraper.py", line 467, in run_scraper
    page.click("text=Hoy", force=True)
  File "/app/cloakbrowser/human/__init__.py", line 1084, in _human_click
    box, cx, cy, did_scroll = scroll_to_element(
                              ^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/scroll.py", line 231, in scroll_to_element
    return human_scroll_into_view(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/scroll.py", line 131, in human_scroll_into_view
    raise RuntimeError("Element not found while scrolling into view")
RuntimeError: Element not found while scrolling into view

2026-08-20 21:03:04.400 [INFO] Screenshot saved to: /app/downloads/error_20260820_210304.png
2026-08-20 21:03:04.397 [INFO] Scraper session ended. Total duration: 40.04 seconds
Job Failed Logic: {'status': 'error', 'message': 'Element not found while scrolling into view', ...,
'scrapedAt': '2026-08-20T21:02:24.358196Z'}

====================================================================
Occurrence 2 -- job started 2026-08-20T21:03:33.499457Z
====================================================================
2026-08-20 21:03:56.766 [INFO] Clicking filter option 'Hoy'...
2026-08-20 21:04:06.809 [WARNING] Click failed on 'Hoy': Element 'text=Hoy'
    failed visible check: element is not visible, retrying force click
    after overlay check...
2026-08-20 21:04:08.930 [ERROR] Error during scraping: Element not found
    while scrolling into view

Traceback (most recent call last):
  File "/app/scraper.py", line 463, in run_scraper
    page.click("text=Hoy", timeout=10000)
  File "/app/cloakbrowser/human/__init__.py", line 1081, in _human_click
    ensure_actionable(page, selector, CHECKS_CLICK, timeout=_remaining_ms(), force=force)
  File "/app/cloakbrowser/human/actionability.py", line 163, in ensure_actionable
    raise last_error
  File "/app/cloakbrowser/human/actionability.py", line 169, in ensure_actionable
    if not _stealth_actionable(page, selector, checks):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/actionability.py", line 109, in _stealth_actionable
    raise ElementNotVisibleError(selector)
cloakbrowser.human.actionability.ElementNotVisibleError: Element 'text=Hoy' failed visible check: element is not visible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/scraper.py", line 467, in run_scraper
    page.click("text=Hoy", force=True)
  File "/app/cloakbrowser/human/__init__.py", line 1084, in _human_click
    box, cx, cy, did_scroll = scroll_to_element(
                              ^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/scroll.py", line 231, in scroll_to_element
    return human_scroll_into_view(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/scroll.py", line 131, in human_scroll_into_view
    raise RuntimeError("Element not found while scrolling into view")
RuntimeError: Element not found while scrolling into view

2026-08-20 21:04:09.112 [INFO] Screenshot saved to: /app/downloads/error_20260820_210408.png
2026-08-20 21:04:09.112 [INFO] Scraper session ended. Total duration: 35.61 seconds
Job Failed Logic: {'status': 'error', 'message': 'Element not found while scrolling into view', ...,
'scrapedAt': '2026-08-20T21:03:33.499457Z'}

====================================================================
Occurrence 3 -- job started 2026-08-20T21:04:47.601244Z
====================================================================
2026-08-20 21:05:08.880 [INFO] Clicking filter option 'Hoy'...
2026-08-20 21:05:18.918 [WARNING] Click failed on 'Hoy': Element 'text=Hoy'
    failed visible check: element is not visible, retrying force click
    after overlay check...
2026-08-20 21:05:21.014 [ERROR] Error during scraping: Element not found
    while scrolling into view

Traceback (most recent call last):
  File "/app/scraper.py", line 463, in run_scraper
    page.click("text=Hoy", timeout=10000)
  File "/app/cloakbrowser/human/__init__.py", line 1081, in _human_click
    ensure_actionable(page, selector, CHECKS_CLICK, timeout=_remaining_ms(), force=force)
  File "/app/cloakbrowser/human/actionability.py", line 163, in ensure_actionable
    raise last_error
  File "/app/cloakbrowser/human/actionability.py", line 169, in ensure_actionable
    if not _stealth_actionable(page, selector, checks):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/actionability.py", line 109, in _stealth_actionable
    raise ElementNotVisibleError(selector)
cloakbrowser.human.actionability.ElementNotVisibleError: Element 'text=Hoy' failed visible check: element is not visible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/scraper.py", line 467, in run_scraper
    page.click("text=Hoy", force=True)
  File "/app/cloakbrowser/human/__init__.py", line 1084, in _human_click
    box, cx, cy, did_scroll = scroll_to_element(
                              ^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/scroll.py", line 231, in scroll_to_element
    return human_scroll_into_view(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/cloakbrowser/human/scroll.py", line 131, in human_scroll_into_view
    raise RuntimeError("Element not found while scrolling into view")
RuntimeError: Element not found while scrolling into view

2026-08-20 21:05:21.196 [INFO] Screenshot saved to: /app/downloads/error_20260820_210521.png
2026-08-20 21:05:21.196 [INFO] Scraper session ended. Total duration: 33.59 seconds
Job Failed Logic: {'status': 'error', 'message': 'Element not found while scrolling into view', ...,
'scrapedAt': '2026-08-20T21:04:47.601244Z'}

====================================================================
Screenshots taken at the moment of each failure (available on request,
files are on the reporter's machine, not attached to this text file):
====================================================================
- downloads/error_20260820_210304.png  (Occurrence 1)
- downloads/error_20260820_210408.png  (Occurrence 2)
- downloads/error_20260820_210521.png  (Occurrence 3)

All three show the "Hoy" <li> fully visible, orange-highlighted
(class="active"), inside the open .daterangepicker dropdown -- no
overlay, modal, or covering element on top of it.

====================================================================
Resolution
====================================================================
Reverting cloakbrowser (wrapper + cloakhq/cloakbrowser base image) from
0.5.8 back to 0.4.11, with no other code changes, immediately restored
the previous reliable behavior: the identical page.click("text=Hoy")
call succeeded in under a second on the next run.
Image Image Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions