Fix dropdown interaction bug with DOM manipulation

[leejamesss]

Problem

Dropdown menus in Magento admin don't work, clicking them either redirects to Dashboard or does nothing.

This is related to #89, #121, and PR #112. The issue is that Playwright's accessibility tree updates before Magento's event handlers finish binding. When we click based on the tree saying an element is ready, the handlers aren't actually ready yet, so Magento's error handler kicks in and redirects.

Playwright also has a known popup menu rendering issue (microsoft/playwright#27142) that won't be fixed.

Solution

Check the element's role before clicking and handle dropdowns specially:

• For <option> elements: Use JavaScript to set .selected and dispatch a change event directly, bypassing the accessibility tree.
• For custom menuitems: Click the parent first, wait for initialization, then force-click the item.

Changes

Added TreeNode infrastructure to track accessibility tree relationships, then updated the CLICK handler to detect and handle dropdown elements.

Code:

node = obseration_processor.get_node_info_by_element_id(int(element_id))

if node and node.role == "option":
    page.get_by_role("option", name=node.name).evaluate(
        """(el) => {
            if (el.parentElement && el.parentElement.tagName === 'SELECT') {
                el.selected = true;
                el.parentElement.dispatchEvent(new Event('change', { bubbles: true }));
            }
        }"""
    )
elif node and node.role == "menuitem":
    if node.parent and node.parent.role in ["combobox", "button"]:
        page.get_by_role(node.parent.role, name=node.parent.name).click()
        page.wait_for_timeout(500)
    page.wait_for_timeout(300)
    page.get_by_role(node.role, name=node.name).click(force=True)

Testing

Tested in my modified environment, it fixes the dropdown issues. Can't guarantee it works in vanilla upstream though, would appreciate if you could test on the official setup, especially Magento dropdowns

References

• Environment fail to capture drop down menu #89
• Pop-up windows and Combobox can not be viewed #121
• Feature/fix pop menu #112
• [BUG] Dropdown displayed abnormally (out of the browser window) microsoft/playwright#27142

[QMMMS]

Thanks for the work, but after applying these changes, when generating a report in shopping_admin and selecting the Period, the dropdown menu is still not visible in the accessibility tree. Did I miss something?

action: click [2332] where [2332] is combobox 'Period' hasPopup: menu expanded: False
accessibility tree:

Tab 0 (current): Orders Report / Sales / Reports / Magento Admin

[1188] RootWebArea 'Orders Report / Sales / Reports / Magento Admin' focused: True
	[1388] link 'Magento Admin Panel'
		[1562] img 'Magento Admin Panel'
	[1290] menubar '' orientation: horizontal
		[1292] link '\ue604 DASHBOARD'
		[1295] link '\ue60b SALES'
		[1301] link '\ue608 CATALOG'
		[1307] link '\ue603 CUSTOMERS'
		[1313] link '\ue609 MARKETING'
		[1319] link '\ue602 CONTENT'
		[1325] link '\ue60a REPORTS'
		[1343] link '\ue60d STORES'
		[1349] link '\ue610 SYSTEM'
		[1355] link '\ue612 FIND PARTNERS & EXTENSIONS'
	[1615] heading 'Orders Report'
	[1617] link '\ue600 admin'
	[1619] link '\ue607'
	[2263] textbox '\ue60c' required: False
	[1235] main ''
		[1595] StaticText 'Scope:'
		[1625] button 'All Websites' hasPopup: menu
		[1628] link '\ue633 What is this?'
		[1600] button 'Show Report'
		[2247] StaticText 'For accurate reporting, be sure to refresh lifetime statistics whenever you change the time zone.'
		[2249] StaticText "Last updated: Jun 17, 2023, 12:00:03 AM. To refresh last day's "
		[2250] link 'statistics'
		[2251] StaticText ', click '
		[2252] link 'here'
		[2253] StaticText '.'
		[1262] group 'Filter'
			[2323] StaticText 'Date Used'
			[2324] combobox 'Date Used' hasPopup: menu expanded: False
			[2329] StaticText 'The Order Updated report is created in real time and does not require a refresh.'
			[2331] StaticText 'Period'
			[2332] combobox 'Period' focused: True hasPopup: menu expanded: True
			[2377] StaticText 'From'
			[1378] textbox 'From *' required: True
				[2401] StaticText '05/01/2022'
			[2333] button 'undefined \ue627'
			[2385] StaticText 'To'
			[1380] textbox 'To *' required: True
				[2552] StaticText '12/31/2022'
			[2334] button 'undefined \ue627'
			[2337] combobox 'Order Status' hasPopup: menu expanded: False

screenshot:

[QMMMS]

#229 (comment)

Root Cause:

• HTML Specification Behavior: Native <select> elements render <option> children via browser-native controls, not through normal page layout. As a result, getBoundingClientRect() on <option> elements returns {x: 0, y: 0, width: 0, height: 0}. This is expected per the HTML spec, not a bug.
• Filtering Logic Issue: In fetch_page_accessibility_tree(), when current_viewport_only=True, nodes with union_bound where width == 0 or height == 0 are removed by remove_node_in_graph(). This filters out expanded combobox children (option/menuitem) even when they are visible.
• Accessibility Tree vs DOM Mismatch: Some UI frameworks or browser versions may expose <option> elements as menuitem in the accessibility tree while the DOM still contains <option>. This requires handling both roles.

Modification Plan:

1. fetch_page_accessibility_tree() in browser_env/processors.py:
   • Collect all combobox nodes with expanded: True
   • Recursively collect their children
   • Skip viewport filtering for these children (preserve them even if union_bound is [0, 0, 0, 0])
2. execute_action() (CLICK case) in browser_env/actions.py:
   • Detect option/menuitem elements with invalid bounds
   • For option: Use JavaScript to manipulate the parent <select> element
   • For menuitem: First check if DOM node is actually <option>, if so use option handling

After modification, the dropdown menu will be visible and can be clicked:

[2321] combobox 'Period' focused: True hasPopup: menu expanded: True
        [2362] menuitem 'Day' selected: False
        [2363] menuitem 'Month' selected: True
        [2364] menuitem 'Year' selected: False