Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 8ea46cc

Browse files
committedJun 25, 2025
tests: Require run-fail ui tests to have an exit code (SIGABRT not ok)
Normally a `run-fail` ui test shall not be terminated by a signal like SIGABRT. So begin having that as a hard requirement. Some of our current tests do terminate by a signal however. Introduce and use `run-crash` for those tests.
1 parent 3129d37 commit 8ea46cc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+139
-91
lines changed
 

‎src/doc/rustc-dev-guide/src/tests/directives.md

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -69,19 +69,20 @@ not be exhaustive. Directives can generally be found by browsing the
6969
See [Controlling pass/fail
7070
expectations](ui.md#controlling-passfail-expectations).
7171

72-
| Directive | Explanation | Supported test suites | Possible values |
73-
|-----------------------------|---------------------------------------------|-------------------------------------------|-----------------|
74-
| `check-pass` | Building (no codegen) should pass | `ui`, `crashes`, `incremental` | N/A |
75-
| `check-fail` | Building (no codegen) should fail | `ui`, `crashes` | N/A |
76-
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen`, `incremental` | N/A |
77-
| `build-fail` | Building should fail | `ui`, `crashes` | N/A |
78-
| `run-pass` | Running the test binary should pass | `ui`, `crashes`, `incremental` | N/A |
79-
| `run-fail` | Running the test binary should fail | `ui`, `crashes` | N/A |
80-
| `ignore-pass` | Ignore `--pass` flag | `ui`, `crashes`, `codegen`, `incremental` | N/A |
81-
| `dont-check-failure-status` | Don't check exact failure status (i.e. `1`) | `ui`, `incremental` | N/A |
82-
| `failure-status` | Check | `ui`, `crashes` | Any `u16` |
83-
| `should-ice` | Check failure status is `101` | `coverage`, `incremental` | N/A |
84-
| `should-fail` | Compiletest self-test | All | N/A |
72+
| Directive | Explanation | Supported test suites | Possible values |
73+
|-----------------------------|---------------------------------------------------------------|-------------------------------------------|-----------------|
74+
| `check-pass` | Building (no codegen) should pass | `ui`, `crashes`, `incremental` | N/A |
75+
| `check-fail` | Building (no codegen) should fail | `ui`, `crashes` | N/A |
76+
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen`, `incremental` | N/A |
77+
| `build-fail` | Building should fail | `ui`, `crashes` | N/A |
78+
| `run-pass` | Running the test binary should make it exit with code 0 | `ui`, `crashes`, `incremental` | N/A |
79+
| `run-fail` | Running the test binary should make it exit with code 1 - 127 | `ui`, `crashes` | N/A |
80+
| `run-crash` | Running the test binary should make it crash | `ui`, | N/A |
81+
| `ignore-pass` | Ignore `--pass` flag | `ui`, `crashes`, `codegen`, `incremental` | N/A |
82+
| `dont-check-failure-status` | Don't check exact failure status (i.e. `1`) | `ui`, `incremental` | N/A |
83+
| `failure-status` | Check | `ui`, `crashes` | Any `u16` |
84+
| `should-ice` | Check failure status is `101` | `coverage`, `incremental` | N/A |
85+
| `should-fail` | Compiletest self-test | All | N/A |
8586

8687
### Controlling output snapshots and normalizations
8788

‎src/doc/rustc-dev-guide/src/tests/ui.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -443,7 +443,7 @@ even run the resulting program. Just add one of the following
443443
- `//@ build-pass` — compilation and linking should succeed but do
444444
not run the resulting binary.
445445
- `//@ run-pass` — compilation should succeed and running the resulting
446-
binary should also succeed.
446+
binary should make it exit with code 0 which indicates success.
447447
- Fail directives:
448448
- `//@ check-fail` — compilation should fail (the codegen phase is skipped).
449449
This is the default for UI tests.
@@ -452,7 +452,10 @@ even run the resulting program. Just add one of the following
452452
without the codegen phase, then a second time the full compile should
453453
fail.
454454
- `//@ run-fail` — compilation should succeed, but running the resulting
455-
binary should fail.
455+
binary should make it exit with a code in the range 1 - 127 which indicates
456+
regular failure.
457+
- `//@ run-crash` — compilation should succeed, but running the resulting
458+
binary should fail with a crash (e.g. `SIGABRT` or `SIGSEGV`).
456459

457460
For `run-pass` and `run-fail` tests, by default the output of the program itself
458461
is not checked.

0 commit comments

Comments
 (0)
Please sign in to comment.