tests: Only run tests with the host and flatpak app info by default

[swick]

Every AppInfo kind multiplies the number of tests. Let's run the tests with the two which will give us the biggest coverage by default.

By setting XDP_TEST_RUN_LONG, all tests run with all AppInfo kinds.

In CI, we run the tests 3 times (test in the build dir, installed tests, dist). Let's run "main" tests with XDP_TEST_RUN_LONG=1 to get the full coverage while keeping the other tests short.

[whot]

code LGTM but for the CI at least it might be worth figuring out a way to run jobs in parallel. The tests are already correctly named so you could pass the test name into meson and parallelize it via strategy:matrix:.

[swick]

Sorry, I'm not following.

[whot]

The CI currently runs a single¹ invocation of meson test -C _build which currently takes too long. What you could do is to parallelize this into multiple jobs so you run each of the integration tests in a separate run, so e.g. meson test -C _build integration/account.

Push that into the CI's strategy:matrix and now you can run all tests in parallel, so something like (don't know the exact yaml syntax off hand):

  strategy:
    matrix:
      meson_testname:
        - "--no-suite=integration"   # runs the unit tests
        - "integration/account"
        - "integration/background"
        ...

   - name: Run xdg-desktop-portal tests
     run: timeout --signal=KILL -v ${TESTS_TIMEOUT}m meson test -C _build ${{matrix.meson_test_name}}

And for safety an extra job that compares the output of meson test --list with a yq invocation on .github/workflows/build-and-test.yml to make sure all integration tests are listed in the CI, similiar to this one.

Footnotes

1. well, once for gcc and once for clang ↩

[swick]

Ah, I see what you mean now. The problem isn't really the time a single meson test run requires, but the fact that we run it 4 times: gcc, clang, dist, and installed. The tests are already heavily parallelized in meson (is_parallel) and in pytest (xdist -n 5). So more parallelization doesn't help much; for the runs to get faster we have to cut down the amount of tests.