Merge branch 'main' into adjust-bal-for-spec-tests

Author: matkt

.github/workflows/reference-tests.yml

@@ -19,7 +19,7 @@ concurrency:
 
 jobs:
   referenceTestEthereum:
-    runs-on: ubuntu-22.04
+    runs-on: besu-research-ubuntu-8 # since eest-v5.1.0 this job needs more disk space
     permissions:
       statuses: write
       checks: write

CHANGELOG.md

@@ -15,9 +15,60 @@
   - Fast Sync
 
 ### Additions and Improvements
+
+### Bug fixes
+- Fix loss of colored output in terminal when using `--color-enabled=true` option [#8908](https://github.com/hyperledger/besu/issues/8908)
+
+## 25.10.0-RC2
+This RC is a pre-release, recommended update for Holesky and Sepolia users for the Fusaka hardfork.
+This updates a dependency used for PeerDAS.
+This RC is still pending burn in for mainnet.
+
+### Breaking Changes
+
+### Upcoming Breaking Changes
+- Deprecated CLI options
+  - `--Xbonsai-parallel-tx-processing-enabled` is deprecated since 25.7.0. Use `--bonsai-parallel-tx-processing-enabled` instead.
+  - `--Xsnapsync-server-enabled` is deprecated since 25.7.0. Use `--snapsync-server-enabled` instead.
+  - `--Xsnapsync-synchronizer-pre-merge-headers-only-enabled` is deprecated since 25.7.0. Use `--snapsync-synchronizer-pre-checkpoint-headers-only-enabled` instead.
+  - `--Xhistory-expiry-prune` is deprecated since 25.7.0. Use `--history-expiry-prune` instead.
+- Sunsetting features - for more context on the reasoning behind the deprecation of these features, including alternative options, read [this blog post](https://www.lfdecentralizedtrust.org/blog/sunsetting-tessera-and-simplifying-hyperledger-besu)
+  - Proof of Work consensus (PoW)
+  - Fast Sync
+
+### Additions and Improvements
+- Performance improvements to EQ opcode [#9229](https://github.com/hyperledger/besu/pull/9229)
+- Update jc-kzg dependency [#9261](https://github.com/hyperledger/besu/pull/9261)
+
+### Bug fixes
+
+## 25.10.0-RC1
+This RC is a pre-release for Holesky and Sepolia users.
+Only upgrade to this release if you're experiencing the eth_subscribe or ethstats known issue from 25.9.0.
+- fixes and more info [#9212](https://github.com/hyperledger/besu/pull/9212) and [#9220](https://github.com/hyperledger/besu/pull/9220)
+This RC is still pending burn in for mainnet.
+
+### Breaking Changes
+
+### Upcoming Breaking Changes
+- Deprecated CLI options
+  - `--Xbonsai-parallel-tx-processing-enabled` is deprecated since 25.7.0. Use `--bonsai-parallel-tx-processing-enabled` instead.
+  - `--Xsnapsync-server-enabled` is deprecated since 25.7.0. Use `--snapsync-server-enabled` instead.
+  - `--Xsnapsync-synchronizer-pre-merge-headers-only-enabled` is deprecated since 25.7.0. Use `--snapsync-synchronizer-pre-checkpoint-headers-only-enabled` instead.
+  - `--Xhistory-expiry-prune` is deprecated since 25.7.0. Use `--history-expiry-prune` instead.
+- Sunsetting features - for more context on the reasoning behind the deprecation of these features, including alternative options, read [this blog post](https://www.lfdecentralizedtrust.org/blog/sunsetting-tessera-and-simplifying-hyperledger-besu)
+  - Proof of Work consensus (PoW)
+  - Fast Sync
+
+### Additions and Improvements
+- Update spring security framework (toml parsing) [#9153](https://github.com/hyperledger/besu/pull/9153)
+- Update grpc and guava [#9150](https://github.com/hyperledger/besu/pull/9150)
 - Implement optional sender balance checks in the layered txpool [#9176](https://github.com/hyperledger/besu/pull/9176)
 - Add `--cache-last-block-headers` flag to cache the last n block headers persisted to the blockchain [#9223](https://github.com/hyperledger/besu/pull/9223)
 - Manage unexpected exceptions during block creation [#9208](https://github.com/hyperledger/besu/pull/9208)
+- Upgrade to execution-spec-tests v5.1.0 [#9226](https://github.com/hyperledger/besu/pull/9226)
+- Add `--cache-last-block-headers-preload-enabled` flag to enable preloading the block headers cache during startup time [#9248](https://github.com/hyperledger/besu/pull/9248)
+- Upgrade to execution-spec-tests v5.2.0 [#9226](https://github.com/hyperledger/besu/pull/9226), [#9242](https://github.com/hyperledger/besu/pull/9242)
 
 ### Bug fixes
 - Fix eth_subscribe RPC failing returning a block response [#9212](https://github.com/hyperledger/besu/pull/9212)
@@ -28,6 +79,12 @@
 ### Breaking Changes
 - Remove deprecated option `--bonsai-maximum-back-layers-to-load` (deprecated since 23.4.0). Use `--bonsai-historical-block-limit` instead 
 
+### Known issue 
+Affects users of eth_subscribe (WebSocket) eg SSV, and ethstats integration
+- symptom: `io.vertx.core.json.EncodeException: Failed to encode as JSON: Java 8 optional type`
+- fixes and more info [#9212](https://github.com/hyperledger/besu/pull/9212) and [#9220](https://github.com/hyperledger/besu/pull/9220)
+- if you experience this issue, you can downgrade to the previous released version of Besu, or build off main if that is an option for your environment.
+
 ### Upcoming Breaking Changes
 - Deprecated CLI options
   - `--Xbonsai-parallel-tx-processing-enabled` is deprecated since 25.7.0. Use `--bonsai-parallel-tx-processing-enabled` instead.
@@ -43,8 +100,8 @@
 - Update netty [#9156](https://github.com/hyperledger/besu/pull/9156)
 - Expose new method to query hardfork by block number Plugin API [#9115](https://github.com/hyperledger/besu/pull/9115)
 - Support loading multiple transaction selector plugins [#8743](https://github.com/hyperledger/besu/pull/9139)
-- Configurable limit for how much time plugins are allowed take, to propose transactions, during block creation [#9184](https://github.com/hyperledger/besu/pull/9184)
-- Add Osaka, BPO1 and BPO2 fork times for holesky, hoodi and sepolia [#9196](https://github.com/hyperledger/besu/pull/9196/files)
+- Configurable limit for how much time plugins are allowed to take, to propose transactions, during block creation [#9184](https://github.com/hyperledger/besu/pull/9184)
+- Add Osaka, BPO1 and BPO2 fork times for holesky, hoodi and sepolia [#9196](https://github.com/hyperledger/besu/pull/9196)
 
 #### Performance
 - Add jmh benchmarks for some compute-related opcodes [#9069](https://github.com/hyperledger/besu/pull/9069)

acceptance-tests/tests/src/test/java/org/hyperledger/besu/tests/acceptance/jsonrpc/ExecutionEngineOsakaAcceptanceTest.java

@@ -0,0 +1,48 @@
+/*
+ * Copyright contributors to Hyperledger Besu.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+package org.hyperledger.besu.tests.acceptance.jsonrpc;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.stream.Stream;
+
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.params.provider.Arguments;
+
+public class ExecutionEngineOsakaAcceptanceTest extends AbstractJsonRpcTest {
+  private static final String GENESIS_FILE = "/jsonrpc/engine/osaka/genesis.json";
+  private static final String TEST_CASE_PATH = "/jsonrpc/engine/osaka/test-cases/";
+
+  private static JsonRpcTestsContext testsContext;
+
+  public ExecutionEngineOsakaAcceptanceTest() {
+    super(testsContext);
+  }
+
+  @BeforeAll
+  public static void init() throws IOException {
+    testsContext = new JsonRpcTestsContext(GENESIS_FILE);
+  }
+
+  public static Stream<Arguments> testCases() throws URISyntaxException {
+    return testCasesFromPath(TEST_CASE_PATH);
+  }
+
+  @AfterAll
+  public static void tearDown() {
+    testsContext.cluster.close();
+  }
+}

acceptance-tests/tests/src/test/resources/jsonrpc/engine/osaka/genesis.json

@@ -0,0 +1,45 @@
+{
+  "request": {
+    "jsonrpc": "2.0",
+    "method": "eth_getBlockByNumber",
+    "params": [
+      "0x00",
+      true
+    ],
+    "id": 67
+  },
+  "response": {
+    "jsonrpc" : "2.0",
+    "id" : 67,
+    "result" : {
+      "number" : "0x0",
+      "hash" : "0x0d16454be2446b1f8e013562833b84da2e76536bef3d529cf2f8976ac1eb5905",
+      "mixHash" : "0x0000000000000000000000000000000000000000000000000000000000000000",
+      "parentHash" : "0x0000000000000000000000000000000000000000000000000000000000000000",
+      "nonce" : "0x0000000000000042",
+      "sha3Uncles" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
+      "logsBloom" : "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
+      "transactionsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
+      "stateRoot" : "0x1a4b82e6ca6153aeb948dcf6f4e8c061c3c86a77106a865821d00062c9c749e3",
+      "receiptsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
+      "miner" : "0x0000000000000000000000000000000000000000",
+      "difficulty" : "0x400000000",
+      "totalDifficulty" : "0x400000000",
+      "extraData" : "0x0000000000000000000000000000000000000000000000000000000000000000a94f5374fce5edbc8e2a8697c15331677e6ebf0b0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
+      "baseFeePerGas" : "0x7",
+      "size" : "0x2db",
+      "gasLimit" : "0x1c9c380",
+      "gasUsed" : "0x0",
+      "timestamp" : "0x0",
+      "uncles" : [ ],
+      "transactions" : [ ],
+      "withdrawalsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
+      "withdrawals" : [ ],
+      "requestsHash" : "0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "blobGasUsed" : "0x0",
+      "excessBlobGas" : "0x0",
+      "parentBeaconBlockRoot" : "0x0000000000000000000000000000000000000000000000000000000000000000"
+    }
+  },
+  "statusCode": 200
+}

acceptance-tests/tests/src/test/resources/jsonrpc/engine/osaka/test-cases/01_get_genesis_block_info.json

@@ -0,0 +1,22 @@
+{
+  "request": {
+    "jsonrpc": "2.0",
+    "method": "engine_getBlobsV2",
+    "params": [
+      [
+        "0x01ce3ae35678dd496bce46fd00a11180ebf30169d4fc3517eef3b753425b013c"
+      ]
+    ],
+    "id": 67
+  },
+  "response": {
+    "jsonrpc" : "2.0",
+    "id" : 67,
+    "error" : {
+      "code" : -38005,
+      "message" : "Unsupported fork",
+      "data" : "OSAKA configured to start at timestamp: 32"
+    }
+  },
+  "statusCode": 200
+}

acceptance-tests/tests/src/test/resources/jsonrpc/engine/osaka/test-cases/02_prague_send_raw_blob-v0.json

@@ -0,0 +1,34 @@
+{
+  "request": {
+    "jsonrpc": "2.0",
+    "method": "engine_forkchoiceUpdatedV3",
+    "params": [
+      {
+        "headBlockHash": "0x0d16454be2446b1f8e013562833b84da2e76536bef3d529cf2f8976ac1eb5905",
+        "safeBlockHash": "0x0d16454be2446b1f8e013562833b84da2e76536bef3d529cf2f8976ac1eb5905",
+        "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000"
+      },
+      {
+        "timestamp": "0x10",
+        "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000",
+        "suggestedFeeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b",
+        "withdrawals": [],
+        "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000"
+      }
+    ],
+    "id": 67
+  },
+  "response": {
+    "jsonrpc": "2.0",
+    "id": 67,
+    "result": {
+      "payloadStatus": {
+        "status": "VALID",
+        "latestValidHash": "0x0d16454be2446b1f8e013562833b84da2e76536bef3d529cf2f8976ac1eb5905",
+        "validationError": null
+      },
+      "payloadId": "0x282643d69a019d11"
+    }
+  },
+  "statusCode": 200
+}