Skip to content

Commit 2058746

Browse files
jgarciaojgarciao
authored
Slim down Sanity moving some tests to Tier2 (execution time > 15 mins) (#453)
* Move some tests from Sanity to Tier2 (execution time > 15 mins) Signed-off-by: Jorge Garcia Oncins <[email protected]> * Refactor test and move it to Tier2 Signed-off-by: Jorge Garcia Oncins <[email protected]> * Move all culler tests to Tier2 Signed-off-by: Jorge Garcia Oncins <[email protected]>
1 parent 7694c2d commit 2058746

File tree

5 files changed

+50
-40
lines changed

5 files changed

+50
-40
lines changed

‎tests/Tests/400__ods_dashboard/400__ods_dashboard.robot‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,8 +163,9 @@ Verify "Notebook Images Are Building" Is Not Shown When No Images Are Building
163163

164164
Verify Notifications Appears When Notebook Builds Finish And Atleast One Failed
165165
[Documentation] Verifies that Notifications are shown when Notebook Builds are finished and atleast one fails
166-
[Tags] Sanity
166+
[Tags] Tier2
167167
... ODS-470 ODS-718
168+
... Execution-Time-Over-30m
168169
... FlakyTest
169170
Clear Dashboard Notifications
170171
${build_name}= Search Last Build namespace=redhat-ods-applications build_name_includes=pytorch

‎tests/Tests/500__jupyterhub/image-iteration.robot‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,13 +32,15 @@ Iterative Testing Classifiers
3232
END
3333

3434
Iterative Testing Clustering
35-
[Tags] Sanity POLARION-ID-Clustering
35+
[Tags] Tier2 POLARION-ID-Clustering
3636
... ODS-923 ODS-924
37+
... Execution-Time-Over-15m
3738
&{DICTIONARY} = Evaluate ${python_dict}
3839
FOR ${sublist} IN @{DICTIONARY}[clustering]
3940
Run Keyword And Continue On Failure Iterative Image Test ${sublist}[0] ${sublist}[1] ${sublist}[2]
4041
END
4142

43+
4244
*** Keywords ***
4345
Iterative Image Test
4446
[Arguments] ${image} ${REPO_URL} ${NOTEBOOK_TO_RUN}

‎tests/Tests/500__jupyterhub/long-running-test-generic-ds.robot‎

Lines changed: 32 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,37 @@
11
*** Settings ***
2-
Resource ../../Resources/ODS.robot
3-
Resource ../../Resources/Common.robot
4-
Resource ../../Resources/Page/ODH/JupyterHub/JupyterHubSpawner.robot
5-
Resource ../../Resources/Page/ODH/JupyterHub/JupyterLabLauncher.robot
6-
Library DebugLibrary
7-
Library JupyterLibrary
8-
Suite Setup Begin Web Test
9-
Suite Teardown End Web Test
10-
Force Tags JupyterHub
2+
Resource ../../Resources/ODS.robot
3+
Resource ../../Resources/Common.robot
4+
Resource ../../Resources/Page/ODH/JupyterHub/JupyterHubSpawner.robot
5+
Resource ../../Resources/Page/ODH/JupyterHub/JupyterLabLauncher.robot
6+
Library DebugLibrary
7+
Library JupyterLibrary
8+
9+
Suite Setup Begin Web Test
10+
Suite Teardown End Web Test
11+
12+
Force Tags JupyterHub
13+
1114

1215
*** Test Cases ***
16+
Long Running Test Case
17+
[Tags] Tier2
18+
... Execution-Time-Over-15m
19+
Launch JupyterLab
20+
Run Repo And Clean https://github.com/lugi0/minimal-nb-image-test minimal-nb-image-test/minimal-nb.ipynb
21+
Run Repo And Clean https://github.com/lugi0/clustering-notebook clustering-notebook/CCFraud-clustering.ipynb
22+
Run Repo And Clean https://github.com/lugi0/clustering-notebook clustering-notebook/customer-segmentation-k-means-analysis.ipynb
23+
Run Repo And Clean https://github.com/lugi0/clustering-notebook clustering-notebook/CCFraud-clustering-S3.ipynb
24+
Run Repo And Clean https://github.com/lugi0/notebook-benchmarks notebook-benchmarks/pytorch/PyTorch-MNIST-Minimal.ipynb
25+
Run Repo And Clean https://github.com/lugi0/notebook-benchmarks notebook-benchmarks/tensorflow/GPU-no-warnings.ipynb
26+
27+
28+
*** Keywords ***
1329
Launch JupyterLab
14-
[Tags] Sanity
15-
Login To RHODS Dashboard ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}
16-
Wait for RHODS Dashboard to Load
17-
${version-check} = Is RHODS Version Greater Or Equal Than 1.4.0
18-
IF ${version-check}==True
30+
Login To RHODS Dashboard ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}
31+
Wait For RHODS Dashboard To Load
1932
Launch JupyterHub From RHODS Dashboard Link
20-
ELSE
21-
Launch JupyterHub From RHODS Dashboard Dropdown
22-
END
23-
Login To Jupyterhub ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}
24-
${authorization_required} = Is Service Account Authorization Required
25-
Run Keyword If ${authorization_required} Authorize jupyterhub service account
26-
Fix Spawner Status
27-
Spawn Notebooks And Set S3 Credentials image=s2i-generic-data-science-notebook
28-
29-
Long Running Test Case
30-
Run Repo and Clean https://github.com/lugi0/minimal-nb-image-test minimal-nb-image-test/minimal-nb.ipynb
31-
Run Repo and Clean https://github.com/lugi0/clustering-notebook clustering-notebook/CCFraud-clustering.ipynb
32-
Run Repo and Clean https://github.com/lugi0/clustering-notebook clustering-notebook/customer-segmentation-k-means-analysis.ipynb
33-
Run Repo and Clean https://github.com/lugi0/clustering-notebook clustering-notebook/CCFraud-clustering-S3.ipynb
34-
Run Repo and Clean https://github.com/lugi0/notebook-benchmarks notebook-benchmarks/pytorch/PyTorch-MNIST-Minimal.ipynb
35-
Run Repo and Clean https://github.com/lugi0/notebook-benchmarks notebook-benchmarks/tensorflow/GPU-no-warnings.ipynb
33+
Login To Jupyterhub ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}
34+
${authorization_required} = Is Service Account Authorization Required
35+
Run Keyword If ${authorization_required} Authorize jupyterhub service account
36+
Fix Spawner Status
37+
Spawn Notebooks And Set S3 Credentials image=s2i-generic-data-science-notebook

‎tests/Tests/500__jupyterhub/notebook_size_verification.robot‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,11 @@ ${CUSTOME_SIZE} {"limits":{"cpu":"6","memory":"9gi"},"requests":{"cpu":"2","
3030

3131

3232
*** Test Cases ***
33-
Verify Spwaned Notebook Size
33+
Verify Spawned Notebook Size
3434
[Documentation] Check the available container size spec
35-
... with actual assign to spwaned notebook pod
36-
[Tags] Sanity
35+
... with actual assign to spawned notebook pod
36+
[Tags] Tier2
37+
... Execution-Time-Over-15m
3738
... FlakyTest
3839
... ODS-1072
3940
Launch JupyterHub Spawner From Dashboard

‎tests/Tests/800__culler/culler.robot‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ ${CUSTOM_CULLER_TIMEOUT} = 600
1919
*** Test Cases ***
2020
Verify Default Culler Timeout
2121
[Documentation] Checks default culler timeout
22-
[Tags] Sanity Tier1
22+
[Tags] Tier2
2323
... ODS-1255
2424
Disable Notebook Culler
2525
${current_timeout} = Get And Verify Notebook Culler Timeout
@@ -28,7 +28,7 @@ Verify Default Culler Timeout
2828

2929
Verify Culler Timeout Can Be Updated
3030
[Documentation] Verifies culler timeout can be updated
31-
[Tags] Sanity Tier1
31+
[Tags] Tier2
3232
... ODS-1231
3333
Modify Notebook Culler Timeout ${CUSTOM_CULLER_TIMEOUT}
3434
${current_timeout} = Get And Verify Notebook Culler Timeout
@@ -39,8 +39,9 @@ Verify Culler Timeout Can Be Updated
3939
Verify Culler Kills Inactive Server
4040
[Documentation] Verifies that the culler kills an inactive
4141
... server after timeout has passed.
42-
[Tags] Sanity Tier1
42+
[Tags] Tier2
4343
... ODS-1254
44+
... Execution-Time-Over-15m
4445
... ProductBug
4546
Spawn Minimal Image
4647
Clone Git Repository And Run https://github.com/redhat-rhods-qe/ods-ci-notebooks-main ods-ci-notebooks-main/notebooks/500__jupyterhub/notebook-culler/Inactive.ipynb
@@ -66,8 +67,9 @@ Verify Culler Kills Inactive Server
6667
Verify Culler Does Not Kill Active Server
6768
[Documentation] Verifies that the culler does not kill an active
6869
... server even after timeout has passed.
69-
[Tags] Sanity Tier1
70+
[Tags] Tier2
7071
... ODS-1253
72+
... Execution-Time-Over-15m
7173
Spawn Minimal Image
7274
Clone Git Repository And Open https://github.com/redhat-rhods-qe/ods-ci-notebooks-main ods-ci-notebooks-main/notebooks/500__jupyterhub/notebook-culler/Active.ipynb
7375
Open With JupyterLab Menu Run Run All Cells
@@ -79,8 +81,9 @@ Verify Culler Does Not Kill Active Server
7981

8082
Verify Do Not Stop Idle Notebooks
8183
[Documentation] Disables the culler (default configuration) and verifies nb is not culled
82-
[Tags] Sanity Tier1
84+
[Tags] Tier2
8385
... ODS-1230
86+
... Execution-Time-Over-15m
8487
Disable Notebook Culler
8588
Close Browser
8689
Spawn Minimal Image
@@ -107,6 +110,7 @@ Verify That "Stop Idle Notebook" Setting Is Not Overwritten After Restart Of Ope
107110
Fail Restart of operator pod causing 'Stop Idle Notebook' setting to change in RHODS dashboard
108111
END
109112

113+
110114
*** Keywords ***
111115
Spawn Minimal Image
112116
[Documentation] Spawn a minimal image

0 commit comments

Comments
 (0)