Skip to content

After #395 lodestar seems unavailable. #400

@yoshidan

Description

@yoshidan

The following error was observed at 7bef190.

Error

Wait for L1 to start up - can take up to 2 minutes
Trouble with your devnet? Try Autofix!
Set AUTOFIX=true to automatically fix common configuration issues.
If that doesn't work, set AUTOFIX=nuke to start fresh with a clean slate.

2025/09/26 10:07:54 Error: error deploying environment: error deploying kurtosis package: execution error: An error occurred executing instruction (number 41) at github.com/ethpandaops/optimism-package/src/wait/wait_for_sync.star[28:16]:
run_sh(name="wait-for-l1-consensus-startup", run="while true; do sleep 5; echo 'L1 Chain is starting up'; SLOT=$(curl -fs $CL_RPC_URL/eth/v1/beacon/headers/ | jq -r '.data[0].header.message.slot // \"0\"'); if [ \"$SLOT\" -gt 0 ]; then echo 'L1 Chain has started!'; break; fi; done", image="mslipper/deployment-utils@sha256:4506b112e4261014329152b161997129e7ca577f39c85e59cfdfdcb47ab7b5cf", env_vars={"CL_RPC_URL": "http://{{kurtosis:aac2023c65cd410ab4a0432ad1e3dcf0:ip_address.runtime_value}}:4000", "L1_BLOCK_TIME": "6", "L1_CHAIN_ID": "3151908", "L1_RPC_KIND": "standard", "L1_RPC_URL": "http://{{kurtosis:9968ebd600644983b524719413084258:ip_address.runtime_value}}:8545", "L1_WS_URL": "ws://{{kurtosis:9968ebd600644983b524719413084258:ip_address.runtime_value}}:8546", "WEB3_RPC_URL": "http://{{kurtosis:9968ebd600644983b524719413084258:ip_address.runtime_value}}:8545"}, wait="300s", description="Wait for L1 to start up - can take up to 2 minutes")
 --- at /home/circleci/project/core/server/api_container/server/startosis_engine/startosis_executor.go:164 (sendErrorAndFail) ---
Caused by: error occurred while executing one time task command: while true; do sleep 5; echo 'L1 Chain is starting up'; SLOT=$(curl -fs $CL_RPC_URL/eth/v1/beacon/headers/ | jq -r '.data[0].header.message.slot // "0"'); if [ "$SLOT" -gt 0 ]; then echo 'L1 Chain has started!'; break; fi; done 
 --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/tasks/run_sh.go:352 (RunShCapabilities.Execute) ---
Caused by: The exec request timed out after 300 seconds
 --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/tasks/tasks_shared.go:239 (executeWithWait) ---
exit status 1

Environment

  • EL = ethpandaops/geth:master-6492751
  • CL = ethpandaops/lodestar:fusaka-devnet-5-5e2a800

Root Cause

Lodestar raises bad request for eth/v1/beacon/headers/. Correct endpoint is eth/v1/beacon/headers.

curl localhost:59151/eth/v1/beacon/headers

{"data":[{"root":"0xe8a7c9157fa6fcd8f005cc12aee6bef61f08f3daf352d2dd8888504eb9ca686f","canonical":true,"header":{"message":{"slot":"97","proposer_index":"52","parent_root":"0x6df41108cac6767914ad16ed392bfead22bac14ff8127be5736f1d2333b3d520","state_root":"0x349a500fed0f6b2b695a830e902896e8a93b778b8ecf702d867604359786853f","body_root":"0x83a4ceacde6ccf4aeb2a4af9231ccc538bc3fb71272c7b51386d35895ae410c2"},"signature":"0x8acfa7ccf8a9b19b91966b4a1d91cd4c7bdeac6eb43e207398682f73f205a36f0d5793f3f12a4e6c96c3938c1018ec4b07694c9088902142a5aeb687db52946702c5680c484c0d1f731aff6ccae6da8f90fc5a4724988e89df5bad31e8121072"}}],"execution_optimistic":false,"finalized":false}

curl localhost:59151/eth/v1/beacon/headers/

{"code":400,"message":"Invalid block id ''"}

In previous commits, even bad requests would return a non-zero value and pass through.
However, due to stricter checks introduced in this commit, it now results in an error.

https://github.com/ethpandaops/optimism-package/pull/395/files

Solution

We should consider using curl -fs $CL_RPC_URL/eth/v1/beacon/headers instead of curl -fs $CL_RPC_URL/eth/v1/beacon/headers/
After forking and making the necessary changes, it is now running successfully.
https://github.com/yoshidan/optimism-package/blob/ca2ca4fab42446c1a9ddc596f1b8036db4766194/src/wait/wait_for_sync.star#L33

(I have not been able to confirm this for other CL).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions