feat: headless UI tree dump via app_process (no instrumentation)#28
Conversation
Adds a standalone app_process entry point that dumps the UI hierarchy without an Instrumentation host, plus the bootstrap that makes it possible. - UiAutomationFactory: builds and connects a UiAutomation by reflecting the @hide UiAutomation(Looper, IUiAutomationConnection) constructor and connect() method (the same path AOSP's uiautomator shell tool uses). Must run as shell/root. Also exposes configureForWindowRetrieval(), now shared by the Instrumentation entry points instead of duplicated in each. - UiDump: app_process main() that prints the hierarchy JSON to stdout: app_process / com.mobilenext.devicekit.UiDump [waitUntilIdleMs] - UiTreeSerializer: dropped the Instrumentation/UiDevice parameter; idle waiting now uses UiAutomation.waitForIdle directly, so the dump no longer depends on the instrumentation framework. - Removed the androidx.test.uiautomator dependency (UiDevice was its only use), further decoupling from instrumentation and shrinking the dex. DeviceKitServer and ViewTreeDump are unchanged in behavior; they now reuse the shared a11y configuration helper.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
3 issues found across 6 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="app/src/main/java/com/mobilenext/devicekit/UiTreeSerializer.kt">
<violation number="1" location="app/src/main/java/com/mobilenext/devicekit/UiTreeSerializer.kt:18">
P1: uiAutomation.waitForIdle() throws TimeoutException on timeout; the old UiDevice.waitForIdle() wrapper swallowed it, allowing dump to proceed with current state. Now an uncaught TimeoutException propagates to callers, causing dump to fail entirely instead of returning best-effort hierarchy.</violation>
</file>
<file name="app/src/main/java/com/mobilenext/devicekit/UiAutomationFactory.kt">
<violation number="1" location="app/src/main/java/com/mobilenext/devicekit/UiAutomationFactory.kt:35">
P2: No disconnect/cleanup path — HandlerThread and UiAutomation connection are never torn down. The thread leaks and the accessibility registration persists unless the process exits. Adding a disconnect() (or making the factory AutoCloseable) would let callers clean up.</violation>
<violation number="2" location="app/src/main/java/com/mobilenext/devicekit/UiAutomationFactory.kt:58">
P2: latch.await() return value ignored — timeout is treated as success. If the accessibility connection isn't live within 2 s, callers will proceed with a UiAutomation that may return empty windows or throw.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
|
||
| fun dump(uiAutomation: UiAutomation, waitUntilIdle: Long = 0L): String { | ||
| if (waitUntilIdle > 0) { | ||
| uiAutomation.waitForIdle(IDLE_WINDOW_MS, waitUntilIdle) |
There was a problem hiding this comment.
P1: uiAutomation.waitForIdle() throws TimeoutException on timeout; the old UiDevice.waitForIdle() wrapper swallowed it, allowing dump to proceed with current state. Now an uncaught TimeoutException propagates to callers, causing dump to fail entirely instead of returning best-effort hierarchy.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At app/src/main/java/com/mobilenext/devicekit/UiTreeSerializer.kt, line 18:
<comment>uiAutomation.waitForIdle() throws TimeoutException on timeout; the old UiDevice.waitForIdle() wrapper swallowed it, allowing dump to proceed with current state. Now an uncaught TimeoutException propagates to callers, causing dump to fail entirely instead of returning best-effort hierarchy.</comment>
<file context>
@@ -1,19 +1,21 @@
+
+ fun dump(uiAutomation: UiAutomation, waitUntilIdle: Long = 0L): String {
+ if (waitUntilIdle > 0) {
+ uiAutomation.waitForIdle(IDLE_WINDOW_MS, waitUntilIdle)
}
</file context>
| uiAutomation.waitForIdle(IDLE_WINDOW_MS, waitUntilIdle) | |
| try { | |
| uiAutomation.waitForIdle(IDLE_WINDOW_MS, waitUntilIdle) | |
| } catch (_: java.util.concurrent.TimeoutException) { | |
| // Proceed with current UI state, mirroring UiDevice.waitForIdle behavior | |
| } |
| * standalone app_process entry points. | ||
| */ | ||
| fun createAndConnect(): UiAutomation { | ||
| val thread = HandlerThread("devicekit-uiautomation").apply { start() } |
There was a problem hiding this comment.
P2: No disconnect/cleanup path — HandlerThread and UiAutomation connection are never torn down. The thread leaks and the accessibility registration persists unless the process exits. Adding a disconnect() (or making the factory AutoCloseable) would let callers clean up.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At app/src/main/java/com/mobilenext/devicekit/UiAutomationFactory.kt, line 35:
<comment>No disconnect/cleanup path — HandlerThread and UiAutomation connection are never torn down. The thread leaks and the accessibility registration persists unless the process exits. Adding a disconnect() (or making the factory AutoCloseable) would let callers clean up.</comment>
<file context>
@@ -0,0 +1,81 @@
+ * standalone app_process entry points.
+ */
+ fun createAndConnect(): UiAutomation {
+ val thread = HandlerThread("devicekit-uiautomation").apply { start() }
+ val uiAutomation = construct(thread.looper)
+ connect(uiAutomation)
</file context>
| AccessibilityServiceInfo.FLAG_REPORT_VIEW_IDS or | ||
| AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS | ||
| } | ||
| latch.await(CONNECT_TIMEOUT_SECONDS, TimeUnit.SECONDS) |
There was a problem hiding this comment.
P2: latch.await() return value ignored — timeout is treated as success. If the accessibility connection isn't live within 2 s, callers will proceed with a UiAutomation that may return empty windows or throw.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At app/src/main/java/com/mobilenext/devicekit/UiAutomationFactory.kt, line 58:
<comment>latch.await() return value ignored — timeout is treated as success. If the accessibility connection isn't live within 2 s, callers will proceed with a UiAutomation that may return empty windows or throw.</comment>
<file context>
@@ -0,0 +1,81 @@
+ AccessibilityServiceInfo.FLAG_REPORT_VIEW_IDS or
+ AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS
+ }
+ latch.await(CONNECT_TIMEOUT_SECONDS, TimeUnit.SECONDS)
+ uiAutomation.setOnAccessibilityEventListener(null)
+ }
</file context>
A bare app_process has no main looper (a normal app gets one from ActivityThread). The accessibility framework builds a Handler on the main looper during connect, so without one the headless dump crashed with: NullPointerException: ... 'android.os.Looper.mQueue' on a null object reference (AccessibilityInteractionClient.<init>) Prepare the main looper in UiDump.main and run the dump on a worker thread while the main thread services looper callbacks. Verified end to end on an emulator (API 37): UiDump now emits the full UI hierarchy as JSON.
|
Warning Review limit reached
More reviews will be available in 20 minutes and 55 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
✅ Verified on emulator (API 37)Tested both paths end to end on a running emulator. Headless
|
1 participant
Summary
Introduces a standalone
app_processpath for dumping the UI hierarchy, so the tree dump can run headless — withoutam instrumentand without anInstrumentationhost. This is the first step toward running the whole toolkit underapp_process.What's new
UiAutomationFactory— builds and connects aUiAutomationwithout instrumentation, by reflecting the@hideUiAutomation(Looper, IUiAutomationConnection)constructor andconnect()method. This is the same approach AOSP's legacyuiautomatorshell tool (UiAutomationShellWrapper) uses. It also exposesconfigureForWindowRetrieval(), which is now shared by the existing instrumentation entry points instead of being duplicated in each.UiDump— a standalonemain()that prints the hierarchy JSON to stdout:@file:JvmName("UiDump")keeps the class name clean for the command line.)Refactors (behavior-preserving)
UiTreeSerializer.dumpno longer takes anInstrumentation. Idle-waiting now usesUiAutomation.waitForIdle(...)directly, so the serializer is fully decoupled from the instrumentation framework.DeviceKitServerandViewTreeDumpnow reuseUiAutomationFactory.configureForWindowRetrieval()and the newdumpsignature. No behavior change.androidx.test.uiautomatordependency —UiDevicewas its only consumer. Fully decouples from instrumentation and shrinks the dex (0uiautomatorrefs remain).Requirements / caveats
shell(uid 2000) or root.UiAutomationConnection.connect()registers withAccessibilityManagerService/WindowManager, which require those permissions.adb shell app_process …satisfies this; it would not work from inside a normal installed app.@hidereflection. The constructor/connect()signatures aren't SDK-stable and can shift across Android versions.minSdkis 29, which is within the supported range for this pattern.app_processinvocations (same reasonscrcpy's server can call hidden APIs).Verification
./gradlew assembleDebugand./gradlew detektboth pass.Lcom/mobilenext/devicekit/UiDump;withmain([Ljava/lang/String;)VandUiAutomationFactory; 0uiautomatorreferences remain.Test plan
app_process / com.mobilenext.devicekit.UiDump 2000against a foreground app; confirm the JSON hierarchy matches the instrumentationViewTreeDumpoutput.DeviceKitServer(device.dump.ui) and instrumentationViewTreeDumpstill work unchanged.@hidereflection differences.Summary by cubic
Adds a headless UI tree dump that runs via
app_processwithoutam instrument. Drops use ofandroidx.test.uiautomatorin code, decouples the serializer fromInstrumentation, and fixes a headless startup crash by preparing the main looper.New Features
UiAutomationFactory: builds and connectsUiAutomationwithout instrumentation; exposesconfigureForWindowRetrieval().UiDump: standalonemain()that prints hierarchy JSON; optionalwaitUntilIdleMs; prepares the main looper and runs on a worker thread; requiresshellor root.Refactors
UiTreeSerializer.dump(uiAutomation, waitUntilIdle)usesUiAutomation.waitForIdle; noInstrumentation.DeviceKitServerandViewTreeDumpnow reuseUiAutomationFactory.configureForWindowRetrieval(); behavior unchanged.Written for commit eb7d1f3. Summary will update on new commits.