Android native instrumentation testing: GTest + doctest cross-platform strategy, activity lifecycle, CI/CD one-command design

[e-gleba]

Summary

Define a production-grade, research-backed strategy for native C++ instrumented testing on Android within this CMake template. The core question: given this repo already uses doctest and has an android-project/ scaffold with Gradle Managed Devices + AndroidJUnitRunner, what is the correct path to one-command cross-platform test execution — covering Android, iOS (XCTest), and Desktop — without hacks, without hand-rolled ADB scripts, and with optimal Activity lifecycle management?

---

Architecture Decision: Three Execution Paths

graph TD
    subgraph Desktop
        A[ctest --preset=gcc-release] --> B[Native Executable]
        B --> C[doctest_discover_tests → CTest]
    end

    subgraph Android
        D[./gradlew connectedCheck] --> E[Instrumentation APK]
        E --> F{Test Type}
        F -->|Pure Logic| G[Raw Native Executable via ADB]
        F -->|SDL/Android APIs| H[Shared Library + JNI Bridge]
        H --> I[GTestRunner or doctest JNI runner]
    end

    subgraph iOS
        J[xcodebuild test] --> K[XCTest Bundle]
        K --> L[xctest_add_bundle → CTest]
    end

    style D fill:#3a3,stroke:#2a2
    style J fill:#33a,stroke:#22a

Loading

The recommendation: use one Activity, one instrumentation invocation, all tests inside — the Activity-reused batching model. This is the approach Chromium calls "Activity-reused" and is the standard outside Chromium for nearly all Android test suites.

---

1. Activity Lifecycle: Restart vs. Keep-Alive — The Data

Chromium maintains the world's largest Android-native C++ test suite. Their batching documentation is definitive Chromium Batching Guide, 2024:

"Outside of Chromium, it is most common to run all tests of a test suite using a single adb shell am instrument command (a single execution / OS process)."

Performance Hierarchy (empirically proven):

Model	Process	Activity	Speed (Debug)	Speed (Release)	Isolation
Process-per-test	New each test	New each test	Baseline	Baseline	Max
Activity-restarted batched	Same	Finished between tests	>50% faster	~25% faster	High
Activity-reused batched	Same	Kept alive	"Significant" gain over restart	"Huge" gain over restart	Requires state reset

Source: Chromium's internal benchmarks on 25,000+ test runs Chromium Android GTests Architecture. The per-test process restart cost is 2–10 seconds. For a suite of 100 tests, that's 3–17 minutes of wasted overhead.

The Critical Detail

Process-per-test uses am instrument per test case — Chromium's default because their browser tests are enormous and state-heavy. For a game engine or SDL application, where state is self-contained in C++, Activity-reused batching is the correct default:

1. am instrument → starts NativeTestInstrumentationTestRunner
2. Creates ONE Activity (NativeUnitTestActivity)
3. JNI bridge → native RunTests()
4. All GTest suites run inside that single process
5. Activity finishes once, results collected

This is exactly how Google's official junit-gtest library works NDK Unit Test Sample, and how this repo's current doctest_android_jni.cpp pattern maps: one shared library, one JNI call, all tests executed.

---

2. The Three Approaches — Ranked

🥇 Tier 1: junit-gtest + GtestRunner (Recommended for GTest users)

Google's officially blessed path. The androidx.test.ext:junit-gtest artifact bridges native GTest binaries into Android's JUnit instrumentation runner GoogleTest Platform Docs.

CMake side:

# prefab must be enabled in build.gradle (already done ✅)
find_package(googletest REQUIRED CONFIG)
find_package(junit-gtest REQUIRED CONFIG)

add_library(engine SHARED
    src/engine.cpp src/renderer.cpp)

add_library(engine_tests SHARED
    tests/test_engine.cpp
    tests/test_renderer.cpp)
target_link_libraries(engine_tests PRIVATE
    engine
    googletest::gtest
    junit-gtest::junit-gtest)

Gradle side (extends existing build.gradle):

dependencies {
    // ADD these to existing deps:
    androidTestImplementation("androidx.test.ext:junit-gtest:1.0.0-alpha01")
    androidTestImplementation("com.android.ndk.thirdparty:googletest:1.11.0-beta-1")
}

Kotlin wrapper (in androidTest/java/):

@RunWith(GtestRunner::class)
@TargetLibrary(libraryName = "engine_tests")
class NativeEngineTests

One command:

./gradlew connectedCheck
# → builds APK, installs on device/emulator, runs all tests, JUnit XML output

🥈 Tier 2: Enhanced doctest JNI (Current path, needs refinement)

This repo already has doctest_android_jni.cpp and compiles tests as a shared library on Android. The gap: no @RunWith bridge to the JUnit runner — tests likely require manual ADB invocation. Fix:

1. Create a doctest-flavored @RunWith runner (similar to GtestRunner) that:

   • Implements InstrumentationTestRunner callbacks
   • Maps doctest assertions to JUnit AssertionError
   • Reports results via Instrumentation.sendStatus()

2. Or, simpler: extend the existing AndroidJUnitRunner with a custom test class that calls the native doctest entry point via JNI and converts exit codes to pass/fail.

3. Alternative — raw executable path (pure logic only): for tests that don't touch Android APIs, build as native executable and run via adb shell:

   if(ANDROID AND CMAKE_ANDROID_ARCH_ABI)
       add_executable(engine_raw_tests tests/test_math.cpp)
       target_link_libraries(engine_raw_tests PRIVATE engine doctest::doctest)
   endif()

   Then in CI: adb push + adb shell + parse stdout. Chromium uses this for signal-handler tests Chromium Android GTests.

🥉 Tier 3: Bare adb shell am instrument (Avoid)

Manually constructing am instrument invocations with extras bundles, parsing stdout from FIFO files, no IDE integration. What Chromium's harness does internally — do not replicate this. Use Tier 1 or 2.

---

3. iOS Counterpart: XCTest via CMake

CMake natively supports XCTest since version 3.3 CMake FindXCTest Documentation. The ios-cmake toolchain (2K+ stars) is the de facto standard for iOS CMake builds ios-cmake GitHub.

CMake XCTest Integration

# Requires Xcode generator
if(IOS)
    find_package(XCTest REQUIRED)

    # The testee must be a Framework or App Bundle
    add_library(engine_framework SHARED src/engine.cpp)
    set_target_properties(engine_framework PROPERTIES
        FRAMEWORK TRUE
        MACOSX_FRAMEWORK_IDENTIFIER com.egleba.engine)

    xctest_add_bundle(EngineTests engine_framework
        tests/test_engine.mm
        tests/test_renderer.mm)
    xctest_add_test(EngineTestsBundle EngineTests)
endif()

Critical XCTest requirements CMake FindXCTest Troubleshooting:

• CMAKE_OSX_SYSROOT must be set (Xcode generator does this automatically)
• Testee must be a Framework or App Bundle (MACOSX_BUNDLE=TRUE)
• Swift module issues arise with non-Xcode generators (Ninja); use Xcode generator for iOS

One command:

cmake -G Xcode -B build/ios -DCMAKE_SYSTEM_NAME=iOS
xcodebuild -project build/ios/Engine.xcodeproj -scheme EngineTests test

---

4. Unified Cross-Platform Test Architecture

graph TB
    subgraph "One Command Per Platform"
        D[Desktop: ctest --preset=gcc-release]
        A[Android: ./gradlew connectedCheck]
        I[iOS: xcodebuild test -scheme EngineTests]
    end

    subgraph "Test Framework Layer"
        G[GTest / doctest C++ Test Code]
        B1[junit-gtest bridge]
        B2[doctest JNI bridge]
        B3[XCTest bridge]
    end

    subgraph "Runner / Harness"
        R1[AndroidJUnitRunner]
        R2[GtestRunner]
        R3[XCTest Runner]
        R4[CTest]
    end

    subgraph "Output"
        O[JUnit XML / CTest XML → CI Dashboard]
    end

    G --> B1 --> R2 --> R1 --> O
    G --> B2 --> R1 --> O
    G --> B3 --> R3 --> O
    G --> R4 --> O

Loading

---

5. CI/CD Integration

Android — Gradle Managed Devices (already configured ✅)

The repo already has a pixel_6_aosp_atd_30 GMD. This is the top-tier approach — AGP handles the full AVD lifecycle Android Instrumentation Testing in CI, arXiv 2024:

# GitHub Actions — extends existing workflow
- name: Android Instrumented Tests
  run: |
    cd android-project
    ./gradlew pixel_6_aosp_atd_30DebugAndroidTest

The ATD (Automated Test Device) system image has no bloat, no Play Services, headless-optimized — significantly faster CI boot times than standard emulator images Android Instrumented Tests Guide.

iOS — Xcode Simulator

- name: iOS Tests
  run: |
    cmake -G Xcode -B build/ios \
      -DCMAKE_SYSTEM_NAME=iOS \
      -DCMAKE_OSX_DEPLOYMENT_TARGET=16.0
    xcodebuild test \
      -project build/ios/cxx_project.xcodeproj \
      -scheme EngineTests \
      -destination 'platform=iOS Simulator,name=iPhone 16,OS=latest' \
      -resultBundlePath TestResults.xcresult

Desktop — CTest (already working ✅)

- name: Desktop Tests
  run: cmake --workflow --preset=gcc-full

---

6. The Activity Lifecycle Decision for This Template

Verdict: Keep Activity alive — Activity-reused batching.

For an SDL3/doctest-based project where:

• Tests are self-contained C++ logic (no browser process, no Content layer)
• The Activity is a thin shell (NativeActivity or custom app.Activity)
• Tests don't modify global Android framework state between cases

The junit-gtest and doctest JNI patterns both naturally implement Activity-reused batching: one instrumentation invocation = one process = one Activity = all tests.

When to use Activity-restart:

• Tests that exercise SDL_Init / SDL_Quit and need a clean engine state
• Tests that modify SharedPreferences or other persistent Android state
• Isolate these into a separate test class annotated with process-per-test

When to use raw executables:

• Pure math/data structure tests with zero Android dependency
• Signal handler tests
• These compile to a native binary, push via ADB, execute directly — no Activity at all, fastest possible path Chromium Android GTests.

---

7. Concrete Implementation Plan for This Repo

Phase 1: Stabilize doctest Android path (2–4 hours)

• Create NativeDoctestRunner.kt in androidTest/java/com/egleba/app/ — a JUnit4 @RunWith runner that calls the native doctest entry point via JNI, captures stdout, maps assertion failures to AssertionError
• Wire into existing AndroidJUnitRunner / connectedCheck flow
• Verify ./gradlew connectedCheck produces JUnit XML with individual test case names

Phase 2: Add GTest as alternative (1–2 hours)

• Add junit-gtest and googletest dependencies to build.gradle
• Create optional CMakeLists.txt path: BUILD_TESTING_WITH_GTEST=ON switches from doctest to GTest
• Add NativeGtestTests.kt wrapper with @RunWith(GtestRunner::class)

Phase 3: Raw executable fallback (1 hour)

• Add CMake option ANDROID_TEST_RAW_EXECUTABLE for pure-logic test suites
• CI step: build executable, push, run, parse stdout

Phase 4: iOS XCTest (2–3 hours)

• Add ios-cmake toolchain integration to cmake/toolchains/
• Create tests/CMakeLists.txt XCTest branch using FindXCTest
• Add Xcode test scheme to CI

Phase 5: Unified CI (1 hour)

• Single scripts/test.sh dispatcher: ./scripts/test.sh android|ios|desktop
• GitHub Actions matrix: os: [ubuntu-latest, macos-latest] + platform-specific test steps

---

8. Key References

1. Chromium Android GTest Architecture — APK vs raw executable design, NativeTestInstrumentationTestRunner lifecycle, stdout redirection via FIFO: source

2. Chromium Test Batching Guide — Activity-restarted vs Activity-reused performance data, @Batch annotations, state management: source

3. Android NDK Unit Test Sample (Google official) — junit-gtest integration, GtestRunner, Prefab setup: source

4. Android NDK CMake Guide — Toolchain file requirements, Prefab, compiler flag best practices, explicit warning against CMake's built-in NDK support: source

5. GoogleTest Platform Docs (AOSP) — BUILD_NATIVE_TEST, Atest, Trade Federation, adb shell manual execution: source

6. GoogleTest CMake Quickstart — FetchContent, gtest_discover_tests, enable_testing(): source

7. CMake FindXCTest Documentation (v4.3.1) — xctest_add_bundle, xctest_add_test, Framework/App Bundle requirements: source

8. ios-cmake Toolchain (2K+ stars) — De facto standard for iOS CMake, XCTest integration, platform detection: source

9. Android Instrumented Tests Guide — AndroidJUnitRunner, Gradle Managed Devices, test filtering, CI: source

10. Bazel Android Instrumentation Tests — Sandboxed emulators, clean-state caching, headless testing with Xvfb: source

11. Android Instrumentation Testing in CI (arXiv 2024) — Execution environment taxonomy, Community vs Custom vs Third Party performance, 25,000+ analyzed CI runs: source

12. Chromium Native Test Launcher Source — native_test_launcher.cc, JNI bridge implementation, AndroidLogPrinter gtest listener, stdout-to-file redirection: source

13. SDL3 iOS README — xcframework distribution, callback-based main loop (SDL_AppInit/SDL_AppIterate), iOS 11.0+ deployment: source

14. sdl3-sample (Ravbug, 341★) — Cross-platform SDL3 + CMake including iOS, tvOS, visionOS, Android, Web: source

15. SDL3 CI/CD Apple Platforms Issue (#11332) — CMAKE_OSX_DEPLOYMENT_TARGET handling, minimum OS versions for iOS/tvOS/macOS: source

---

Related Issues

• #20 — macOS/iOS Xcode presets (XCTest depends on this)
• #3 — vcpkg compatibility (googletest available via vcpkg)
• #9 — Sanitizers (ASan/UBSan critical for Android native test reliability)
• #10 — Codecov/CodeQL (test coverage tracking)