Test #2

kalvinkhuu · 2025-09-19T21:23:56Z

Verifying that the jpipe runner works for a paper

github-actions · 2025-09-19T21:25:02Z

Justification process failed!

Runner Output


ERROR - validate():317 - 2025-09-19 21:24:58,643 - [EvidenceDependencyValidator]
Pipeline validation error: evidence node does not produce any variables.
  • Evidence: 'headless_linux_distribution_used_for_ctf'
  • Problem: This evidence does not produce any output variables.
  • Impact: Strategies connected above this evidence will not receive required inputs.
  • Recommendation: Ensure that the function associated with evidence 'headless_linux_distribution_used_for_ctf' is correctly producing at least one variable.

[EvidenceDependencyValidator]
Pipeline validation error: evidence node does not produce any variables.
  • Evidence: 'curated_ctf_challenges_list'
  • Problem: This evidence does not produce any output variables.
  • Impact: Strategies connected above this evidence will not receive required inputs.
  • Recommendation: Ensure that the function associated with evidence 'curated_ctf_challenges_list' is correctly producing at least one variable.

[EvidenceDependencyValidator]
Pipeline validation error: evidence node does not produce any variables.
  • Evidence: 'ctf_challenges_evaluation_results'
  • Problem: This evidence does not produce any output variables.
  • Impact: Strategies connected above this evidence will not receive required inputs.
  • Recommendation: Ensure that the function associated with evidence 'ctf_challenges_evaluation_results' is correctly producing at least one variable.

[EvidenceDependencyValidator]
Pipeline validation error: evidence node does not produce any variables.
  • Evidence: 'threshold_for_ctf_model_performance'
  • Problem: This evidence does not produce any output variables.
  • Impact: Strategies connected above this evidence will not receive required inputs.
  • Recommendation: Ensure that the function associated with evidence 'threshold_for_ctf_model_performance' is correctly producing at least one variable.

github-actions · 2025-09-23T17:38:07Z

Justification process failed!

Download Diagram Artifact

Runner Output

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 40, in __init__
    self.load_files(libraries or [])
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 54, in load_files
    self._import_file(file_path)
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 73, in _import_file
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/jpipe-examples/jpipe-examples/llm_FATES/compiler/cyber_capability.py", line 6, in <module>
    @jpipe(consume=[], produce=[])
     ^^^^^
NameError: name 'jpipe' is not defined

github-actions · 2025-09-23T17:51:19Z

Justification process failed!

Download Diagram Artifact

Runner Output

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 40, in __init__
    self.load_files(libraries or [])
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 54, in load_files
    self._import_file(file_path)
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 73, in _import_file
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/jpipe-examples/jpipe-examples/llm_FATES/compiler/cyber_capability.py", line 6, in <module>
    @jpipe(consume=["test1", "test2", "test3", "test4"], produce=["test5"])
     ^^^^^
NameError: name 'jpipe' is not defined

github-actions · 2025-09-23T17:53:27Z

Justification process failed!

Download Diagram Artifact

Runner Output

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 40, in __init__
    self.load_files(libraries or [])
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 54, in load_files
    self._import_file(file_path)
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 73, in _import_file
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/jpipe-examples/jpipe-examples/llm_FATES/compiler/cyber_capability.py", line 4, in <module>
    import httpx
ModuleNotFoundError: No module named 'httpx'

github-actions · 2025-09-23T18:16:17Z

Justification process failed!

Download Diagram Artifact

Runner Output

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 40, in __init__
    self.load_files(libraries or [])
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 54, in load_files
    self._import_file(file_path)
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 73, in _import_file
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/jpipe-examples/jpipe-examples/llm_FATES/compiler/cyber_capability.py", line 4, in <module>
    import httpx
ModuleNotFoundError: No module named 'httpx'

github-actions · 2025-09-23T18:24:48Z

Justification process failed!

Download Diagram Artifact

Runner Output

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 40, in __init__
    self.load_files(libraries or [])
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 54, in load_files
    self._import_file(file_path)
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/jpipe_runner/runtime.py", line 73, in _import_file
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/jpipe-examples/jpipe-examples/llm_FATES/compiler/cyber_capability.py", line 4, in <module>
    import httpx
ModuleNotFoundError: No module named 'httpx'

github-actions · 2025-09-23T18:30:56Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-23 18:30:52,401 - TypeError: curated_ctf_challenges_list() got an unexpected keyword argument 'produce'

…ersfine_cap_cybersec_acceptable

github-actions · 2025-09-23T18:55:56Z

Justification process completed!

View Generated Diagram

Download Diagram Artifact

github-actions · 2025-09-23T19:00:35Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-23 19:00:29,432 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T18:22:50Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 18:22:46,160 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T18:31:04Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 18:30:57,618 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T18:34:33Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 18:34:30,193 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T18:40:21Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 18:40:17,029 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T18:46:18Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 18:46:12,664 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T18:50:24Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 18:50:20,380 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T19:29:01Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 19:28:57,549 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T19:40:42Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 19:40:38,244 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T20:08:02Z

Justification process completed!

View Generated Diagram

Download Diagram Artifact

github-actions · 2025-09-25T20:11:16Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 20:11:09,812 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T20:15:02Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 20:14:56,755 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T20:54:02Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - validate():315 - 2025-09-25 20:53:56,909 - [ProducedButNotConsumedValidator]
Pipeline validation error: produced variable not consumed.
  • Variable 'test7' is produced by function 'cyber_range_capability_is_acceptable' but is never consumed by any function.
  • This may indicate redundant computation or misconfiguration.
  • Consider removing the production of 'test7' if unused, or verify downstream usage.

ERROR - validate():317 - 2025-09-25 20:53:56,909 - [EvidenceDependencyValidator]
Pipeline validation error: evidence variables not consumed by strategies.
  • Evidence: 'cyber_range_capability_is_acceptable'
  • Produced Variables: ['test7']
  • Affected Strategies: ['aggregate_all_the_models_evaluations_in_terms_of_cybersecurity_capabilities']
  • Problem: These strategies do not consume all variables produced by the evidence directly below them.
  • Recommendation: Verify that the strategies listed above are configured to consume all outputs from evidence 'cyber_range_capability_is_acceptable'.

github-actions · 2025-09-25T20:59:38Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - validate():315 - 2025-09-25 20:59:35,351 - [DuplicateProducerValidator]
Pipeline validation error: duplicate producers detected.
  • Variable 'test5' is produced by multiple functions: ['verify_ctf_challenges_capability_is_acceptable', 'headless_linux_distribution_used_for_ctf']
  • Each variable must have exactly one producer to maintain a valid pipeline structure.
  • To fix:
    - Choose a single function to produce 'test5' and remove it from the others.
    - If multiple outputs are required, consider renaming or splitting the variables.

ERROR - validate():317 - 2025-09-25 20:59:35,351 - [SelfDependencyValidator]
Pipeline validation error: self-dependency detected.
  • Function 'verify_ctf_challenges_capability_is_acceptable' declares variable 'test5' as both consumed and produced by itself.
    This is likely a misconfiguration:
      - If 'test5' should come from outside, remove it from this function's produce list
        and ensure an external provider supplies it.
      - If this function is the sole producer for downstream use, remove 'test5' from its consume list.
      - If you truly need to consume an initial 'test5' and then produce an updated 'test5',
        ensure that initial 'test5' is provided in context or by another function under a distinct name,
        so the dependency graph does not treat the same function as its own producer.

github-actions · 2025-09-25T21:05:47Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - pretty_display():172 - 2025-09-25 21:05:43,328 - FunctionException: 
Function 'curated_ctf_challenges_list' returned False, indicating failure.
  - The function associated with node 'curated_ctf_challenges_list' (label: 'Curated CTF challenges list') executed but did not pass its check.
  - Please review the implementation and input data for this function.
  - Returned value: False
  - The function must return True to indicate a successful check.

github-actions · 2025-09-25T21:09:13Z

Justification process failed!

Download Diagram Artifact

Runner Output


WARNING - inject_arguments():97 - 2025-09-25 21:09:09,982 - Consumed variable 'test6' is declared but not used in function 'verify_ctf_challenges_capability_is_acceptable'.
WARNING - pretty_display():172 - 2025-09-25 21:09:09,983 - NameError: name 'test5' is not defined

github-actions · 2025-09-25T21:14:28Z

Justification process completed!

View Generated Diagram

Download Diagram Artifact

github-actions · 2025-09-25T21:23:48Z

Justification process completed!

View Generated Diagram

Download Diagram Artifact

kalvinkhuu added 3 commits September 19, 2025 17:07

test

bfb8dc5

changed branch name in yml

efa781c

changed jd file to json

eb6da34

added the consume and produce

b781994

kalvinkhuu added 2 commits September 23, 2025 13:49

added variables on the consume and the produce

35a4266

added imports

a8a25c4

created jd

d3300ef

fix: json justification and steps

9c043ac

changes

f9e8887

fix(cyber_capability): renaming function name for strategy verify_adv…

dd47dcd

…ersfine_cap_cybersec_acceptable

changes

ea6d98c

changed the diagram

12b4c87

restart pipeline

8a4c4aa

restart pipeline

3d13989

restart pipeline

135f65d

restart pipeline

c678acc

restart pipeline

306e529

test execution

e1a3b0d

tryign with everything pass

183de12

test without cyber evidence

94b53c2

comment cyber_range_capability_is_acceptable

32a74ea

added back the evidence

5d041f8

correctly connected the cyber range

4ebcb22

corrected error message

2fcd653

changed all to true

7cc3248

changed all to true

03efe3b

changed python version

2b9409f

kalvinkhuu closed this Sep 25, 2025

kalvinkhuu deleted the test branch September 25, 2025 21:34

Test #2

Test #2

Uh oh!

Conversation

kalvinkhuu commented Sep 19, 2025

Uh oh!

github-actions bot commented Sep 19, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

github-actions bot commented Sep 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants