Skip to content

Feat/rpc issue 67#636

Open
promisszn wants to merge 48 commits intodotandev:mainfrom
promisszn:feat/rpc-issue-67
Open

Feat/rpc issue 67#636
promisszn wants to merge 48 commits intodotandev:mainfrom
promisszn:feat/rpc-issue-67

Conversation

@promisszn
Copy link

Closes #383

- Add internal/trace/printer.go: PrintExecutionTrace and PrintTraceTree
  with tree-style ASCII connectors, color-coded icons per event type,
  CPU/MEM budget lines, and a summary footer (Steps / Errors / Status)
- Add --print and --no-color flags to 'erst trace' CLI command
- Use github.com/fatih/color v1.18.0 for ANSI styling with NoColor support
- Add printer_test.go: TestPrintTraceTree_NoColor, TestPrintExecutionTrace_NoColor, TestPrintOptions_Width
- Add internal/visualizer/ansi.go: SGR constants for visualizer package
- Fix pre-existing compilation/test errors:
  visualizer/color.go: bad merge + NO_COLOR/TERM=dumb env var support
  dwarf/parser.go: invalid stdlib DWARF API calls
  dwarf/parser_test.go: dwOpStackValue constant, duplicate struct fields
  trace/navigation.go: duplicate struct fields from bad merge
  trace/node.go: missing fmt import, unused variable
  trace/treeui.go: unused scrollPos variable
  trace/trap.go + trap_test.go: wrong SourceLocation type
  trace/viewer.go: redundant newline, add expand keyword to help
  trace/node_test.go + search_unicode_test.go: test compilation fixes
- Add TxStreamer interface with Stream(ctx, hash) channel API
- Implement wsStreamer: persistent WebSocket connection with periodic
  getTransaction JSON-RPC calls at 2s intervals
- Implement pollingStreamer: HTTPS JSON-RPC polling at 3s intervals
- NewTxStreamer factory probes for WebSocket support (3s timeout)
  and selects wsStreamer or pollingStreamer transparently
- WebSocket protocol implemented per RFC 6455 using stdlib only:
  wsDialUpgrade, wsWriteFrame (masked), wsReadFrame, wsGenKey,
  wsAcceptKey -- no new external dependencies
- Add telemetry spans: rpc_tx_stream_ws and rpc_tx_stream_poll
- TxStatus constants: PENDING, SUCCESS, FAILED, NOT_FOUND
- Add txstream_test.go with 15 tests covering unit + integration
- Fix pre-existing compilation errors in verification.go,
  verification_test.go, client_test.go, client_retry_test.go
Copilot AI review requested due to automatic review settings February 25, 2026 16:43
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements WebSocket-based transaction status streaming for Soroban RPC (closes #383), adds a --print flag to the trace command for non-interactive output, and includes several bug fixes and code cleanup tasks.

Changes:

  • Adds WebSocket streaming support for transaction status updates with automatic fallback to HTTP polling
  • Introduces --print and --no-color flags to the trace command for generating ASCII tree reports suitable for CI/logs
  • Improves color handling in the visualizer with environment variable checks (NO_COLOR, FORCE_COLOR, TERM=dumb)
  • Fixes XDR unmarshaling to use io.Reader interface
  • Removes duplicate struct fields and unused code
  • Adds dependencies for color output (fatih/color, mattn/go-colorable)

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
internal/rpc/txstream.go New WebSocket streaming implementation with fallback to HTTP polling
internal/rpc/txstream_test.go Comprehensive test coverage for streaming modes and edge cases
internal/trace/printer.go New printer implementation for generating rich ASCII trace trees
internal/trace/printer_test.go Tests for printer with color and width handling
internal/cmd/trace.go Adds --print and --no-color CLI flags
internal/visualizer/color.go Improved ColorEnabled() with env var checks, new Colorize() implementation
internal/visualizer/ansi.go ANSI SGR constants moved from terminal package
internal/rpc/verification.go Fixed XDR unmarshaling to use bytes.NewReader
internal/rpc/verification_test.go Fixed XDR pointer usage for ContractId and ScSymbol
internal/trace/navigation.go Removed duplicate struct fields
internal/dwarf/parser.go Cleaned up WASM parsing code, removed unused unit field
internal/trace/treeui.go Removed unused scrollbar calculation variable
internal/trace/search_unicode_test.go Removed duplicate EventData line
internal/trace/node_test.go Added missing closing brace for test function
internal/trace/node.go Added fmt import for similarityKey()
internal/rpc/client_retry_test.go Fixed test to use inline struct instead of undefined LedgerEntryResult
internal/rpc/client_test.go Added missing closing brace
go.mod, go.sum Added fatih/color, mattn/go-colorable, atotto/clipboard dependencies
Comments suppressed due to low confidence (1)

internal/dwarf/parser.go:193

  • The function parseWASMSections and helper readULEB128 are defined but never actually called (parseWASM just returns ErrNoDebugInfo and uses a blank assignment to suppress the unused warning). Consider either removing this dead code entirely, or if it's intended for future use, add a TODO comment explaining the planned implementation. The current approach of keeping unused code with a suppression comment is not ideal for maintainability.
	_ = parseWASMSections // suppress unused warning
	return nil, ErrNoDebugInfo
}

// parseWASMSections parses custom sections from a WASM binary
func parseWASMSections(data []byte) map[string][]byte {
	sections := make(map[string][]byte)

	i := 8 // Skip WASM magic + version
	for i < len(data) {
		sectionID := data[i]
		i++

		// Read section size (LEB128 unsigned)
		sectionSize, n := readULEB128(data[i:])
		if n == 0 {
			break
		}
		i += n

		sectionEnd := i + int(sectionSize)
		if sectionEnd > len(data) {
			break
		}

		if sectionID == 0 { // Custom section
			// Read name length (LEB128 unsigned)
			nameLen, nn := readULEB128(data[i:])
			if nn == 0 {
				i = sectionEnd
				continue
			}
			nameStart := i + nn
			nameEnd := nameStart + int(nameLen)
			if nameEnd > sectionEnd {
				i = sectionEnd
				continue
			}

			name := string(data[nameStart:nameEnd])
			sections[name] = data[nameEnd:sectionEnd]
		}

		i = sectionEnd
	}

	return sections
}

// readULEB128 decodes an unsigned LEB128 value from buf.
// Returns the value and the number of bytes consumed; 0 bytes means the buffer
// was too short.
func readULEB128(buf []byte) (uint64, int) {
	var val uint64
	var shift uint
	for i, b := range buf {
		val |= uint64(b&0x7f) << shift
		if b&0x80 == 0 {
			return val, i + 1
		}
		shift += 7
		if shift >= 64 {
			return 0, 0
		}
	}
	return 0, 0
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dotandev
Copy link
Owner

check ci, they're all failing

- Standardized formatting in `internal/trace/trap.go`, `internal/trace/trap_test.go`, and `internal/trace/treeui.go` for improved readability.
- Updated `internal/updater/checker.go` to enhance update checking logic and error handling.
- Modified `internal/visualizer/color.go` to adjust symbol representation.
- Cleaned up imports in `internal/watch/spinner.go` and ensured consistent code style.
- Added missing newline at the end of `main.go` for better file formatting.
- Updated licensing headers in various scripts and test files to reflect authorship and licensing information.
- Improved code structure and readability in `simulator/src/main.rs` and `simulator/src/runner.rs`.
- Enhanced error handling and logging in `simulator/src/stack_trace.rs`.
- Added licensing headers to TypeScript files in the audit and command directories.
- Ensured consistent formatting and licensing in test files across the project.
@promisszn
Copy link
Author

check now

# Conflicts:
#	internal/dwarf/parser.go
#	internal/rpc/client.go
#	internal/rpc/client_retry_test.go
#	internal/rpc/verification_test.go
#	internal/terminal/ansi.go
@gitguardian
Copy link

gitguardian bot commented Feb 26, 2026

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

Since your pull request originates from a forked repository, GitGuardian is not able to associate the secrets uncovered with secret incidents on your GitGuardian dashboard.
Skipping this check run and merging your pull request will create secret incidents on your GitGuardian dashboard.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
27626607 Triggered Generic Private Key 5dcaab8 integration/integration_test.go View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

ikemHood and others added 15 commits February 26, 2026 12:53
…imulator runs, update trace node fields, and refine source map resolution prompts.
When a compiler inlines a function the DWARF debug info records the
inlined body under a DW_TAG_inlined_subroutine child entry inside the
caller's DW_TAG_subprogram.  Without handling this, any fault that
occurs inside inlined code is attributed to the enclosing caller,
misleading the developer about where the crash originated.

Changes
-------
internal/dwarf/parser.go
  - Fix pre-existing compile errors: broken dwarf.New call (wrong arity),
    non-existent dwarf.Unit / dwarf.AttrParent / dwarf.LineProgram /
    dwarf.LocExprStackValue / dwarf.LocAddr / dwarf.LocEnd symbols.
  - Rewrite parseWASMSections with a correct, panic-safe LEB128 decoder.
  - Rewrite getLocalVariables to walk DWARF child entries sequentially
    instead of the bogus AttrParent reference approach.
  - Rewrite GetSourceLocation / findLineForAddr to use the standard
    data.LineReader API.
  - Replace raw opcode constants in formatLocation with documented values
    (DW_OP_addr=0x03, DW_OP_stack_value=0x9f).
  - Add InlinedSubroutineInfo struct carrying AbstractOrigin, Name,
    DemangledName, CallSite, InlinedLocation, LowPC/HighPC.
  - Add GetInlinedSubroutines: reads DW_TAG_inlined_subroutine children,
    resolves DW_AT_abstract_origin for the function name, and decodes
    DW_AT_call_file / DW_AT_call_line / DW_AT_call_column for the
    call-site coordinates inside the caller.
  - Add ResolveInlinedChain: finds the concrete subprogram containing a
    given address, then returns all inlined children that cover that
    address.
  - Add resolveAbstractOrigin, resolveFileIndex, findSubprogramOffset
    helpers.
  - Add readULEB128 helper.

internal/dwarf/parser_test.go
  - Fix pre-existing issues: LocExprStackValue constant reference, duplicate
    struct field in TestNameDemangle table.
  - Split the ELF magic table row into a dedicated TestNewParser_ShortELF
    to avoid asserting a specific error type from the elf package.
  - Add TestInlinedSubroutineInfo, TestGetInlinedSubroutines_NoDebugInfo,
    TestResolveInlinedChain_NoDebugInfo, TestResolveInlinedChain_MinimalWASM,
    TestFindSubprogramOffset_NoData, TestResolveFileIndex_ZeroIndex,
    TestInlinedSubroutineInfo_ZeroValues.

internal/trace/trap.go
  - Add SourceLocation type alias (= dwarf.SourceLocation) so package-
    internal code and tests can use the unqualified name.
  - Add InlinedFrame struct (Function, CallSite, InlinedAt).
  - Add InlinedChain []InlinedFrame field to TrapInfo.
  - Update DetectTrap to call resolveInlinedChain after subprogram lookup;
    remove the previous guard that required state.Arguments != nil.
  - Add resolveInlinedChain: populates TrapInfo.InlinedChain, updates
    SourceLocation to the innermost inlined fault site, updates Function
    to the innermost inlined function name.
  - Add displayName helper.
  - Update FormatTrapInfo to render the Inlined Call Chain section showing
    each frame with its call-site and inlined-at coordinates.

internal/trace/trap_test.go
  - Add TestInlinedFrame, TestFormatTrapInfo_WithInlinedChain,
    TestFormatTrapInfo_NoInlinedChain, TestResolveInlinedChain_NoDwarfParser,
    TestTrapInfo_InlinedChainFieldExists,
    TestDetectTrap_FunctionUpdatedFromInlined.

internal/trace/viewer.go
  - Fix pre-existing fmt.Println trailing-newline lint error.
  - Add expand entry to showHelp output (was missing, causing a pre-existing
    test failure).

internal/trace/treeui.go
  - Remove unused scrollPos variable (pre-existing).

internal/trace/search_unicode_test.go
  - Remove duplicate EventData field in TestSearchUnicode_Mixed (pre-existing).

Closes dotandev#376
Add CheckHashMismatch to detect when the local WASM hash does not match
the on-chain hash, and introduce HashMismatchError for structured error
reporting. Add tests covering match, mismatch, missing file, and error
message content.

Closes dotandev#373
Add var _ Interface = (*Struct)(nil) assertions to enforce that
concrete types satisfy their declared interfaces at compile time.

Covered interfaces and implementors:
- rpc.MethodTelemetry <- noopMethodTelemetry
- rpc.MethodTimer <- noopMethodTimer
- terminal.Renderer <- ANSIRenderer, MockRenderer
- authtrace.ContractAuthHandler <- MultiSigContractAuth, RecoveryAuth
- simulator.RunnerInterface <- MockRunner
- plugin.DecoderPlugin <- mockDecoder (test)

The existing Runner assertion in simulator/runner.go is unchanged.

Fixes dotandev#564
Add FetchHistoricalContractBytecode to retrieve WASM bytecode for a
contract as it appeared in a specific transaction's result metadata.
This enables auditing a contract's code at a particular version by
referencing the deploy or upgrade transaction hash.

Add WasmBytesFromContractCodeEntry to extract raw WASM bytes from a
base64-encoded ContractCode ledger entry.

Expose the functionality through a new GetContractCode JSON-RPC
endpoint on the daemon server, accepting contract_id and tx_hash
parameters and returning the WASM bytes and code hash.

Fixes dotandev#557
Caritajoe18 and others added 24 commits February 26, 2026 13:10
Adds GetLatestLedgerSequence to internal/rpc and wires a health check into the dry-run execution flow. Warns users if the local node is more than 15 ledgers behind close dotandev#388
Introduce internal/signer package with a generic Signer interface that
decouples cryptographic signing from key storage:

- Signer interface: Sign(data) -> (sig, error), PublicKey() -> (key, error),
  Algorithm() -> string
- InMemorySigner: default Ed25519 implementation using crypto/ed25519,
  accepts hex-encoded seed or full private key for backward compatibility
- Pkcs11Signer: PKCS#11 HSM integration that dynamically loads a shared
  library at runtime. Signs via CKM_EDDSA mechanism; private key material
  never enters the SDK process. Configured through ERST_PKCS11_* env vars.
- SignerError: structured error type with Op/Msg/Err fields and Unwrap
- NewFromEnv factory: creates the appropriate signer from ERST_SIGNER_TYPE

Refactor audit.go to route through the Signer interface:

- Generate() now creates a temporary InMemorySigner internally, preserving
  the existing call signature for all current callers
- Gene- Gene- Gene- Gene- Gene- Gene- Gene- Gene- Geneepts any Signer
  implementation
- VerifyAuditL- VerifyAuditL- VerifyAuditL- VerifyAuditL- VerifyAuditL- Verify new signer tests (InMemory round-trip, invalid key, error
formatting, Pkcs11Config validation) + 2 new audit tests exercising
GenerateWithSigner directly. All 9 existing audit tests pass unchanged.

Closes dotandev#533
Extend the resource lookup system with two new resolution strategies
that run before falling back to Levenshtein-based fuzzy suggestion:

- resolvePartialID: resolves unique case-insensitive prefix matches
  against session IDs (e.g. 'def4' -> 'def456-1700001111')
- resolveByTxHash: resolves unique prefix matches against stored
  transaction hashes, treating them as implicit aliases

Introduce resolveSessionInput as a unified entry point that chains
exact ID -> partial ID prefix -> tx-hash prefix -> fuzzy suggestion.
Callers in session resume, session delete, and stats --session now
use the unified resolver instead of the previous suggest-only flow.

Add tests for resolvePartialID (unique prefix, ambiguous prefix,
no match, case-insensitivity) and resolveByTxHash.

Closes dotandev#525
Add --publish-ipfs and --publish-arweave flags to the debug command so
that a signed audit trail is pushed to decentralised storage immediately
after a simulation completes.

New package internal/decenstorage:
- Publisher.PublishIPFS  – pins the JSON-encoded AuditLog via the Kubo
  HTTP RPC /api/v0/add endpoint; returns the CID and public gateway URL.
- Publisher.PublishArweave – posts the payload to an Arweave-compatible
  HTTP gateway; returns the transaction ID and URL.
- PublishConfig resolves ERST_IPFS_NODE, ERST_ARWEAVE_GATEWAY, and
  ERST_ARWEAVE_WALLET from the environment when the corresponding flag
  is not supplied.

New debug flags:
  --publish-ipfs        pin signed audit trail to IPFS (requires --audit-key)
  --publish-arweave     upload signed audit trail to Arweave (requires --audit-key)
  --audit-key           Ed25519 private-key hex used to sign the audit log
  --ipfs-node           override Kubo node URL
  --arweave-gateway     override Arweave gateway URL
  --arweave-wallet      path to Arweave JWK wallet file

Tests: 9 unit tests covering success paths, server errors, empty
response IDs, defaults, and env-variable overrides.

Closes dotandev#398
…ev#380)

For transactions that touch >100 keys, split getLedgerEntries calls into
optimal batches of 100 keys each and dispatch them concurrently.

- Add BatchGetLedgerEntries on Client: delegates to GetLedgerEntries
  directly for footprints at or below the threshold (100 keys), and
  splits into concurrent batches of batchSize (100) for larger sets.
- Add chunkKeys helper to slice a key slice into fixed-size chunks.
- Add batchSize and largFootprintThreshold constants with explanatory
  comments; both are package-level to allow future tuning.
- Fix AllNodesFailedError.Unwrap to expose per-node errors so
  errors.Is / errors.As can traverse wrapped typed errors.
- Fix verification.go: replace xdr.Unmarshal (requires io.Reader) with
  LedgerKey.UnmarshalBinary for correct binary decoding.
- Fix pre-existing test build errors: mismatched xdr.Hash / xdr.ContractId
  types in verification_test.go, undefined LedgerEntryResult type in
  client_retry_test.go, missing closing brace in client_test.go.
- Fix ledger_test.go: introduce newMockLedgerClient helper that
  populates AltURLs so the failover loop executes; replace direct type
  assertions with errors.As for wrapped error types.

Closes dotandev#380
- Audited existing CI/CD components
- Refactored to eliminate reliance on implicit global state
- Added benchmarks and tests to validate improvements
- Updated internal architecture documentation
Extract validation logic from monolithic Config.Validate() into separate
Validator interface implementations: RPCValidator, NetworkValidator,
SimulatorValidator, and LogLevelValidator.

Add MergeDefaults() to separate default assignment from loading and
validation. Load() now follows distinct phases: load, merge defaults,
validate.

Add comprehensive unit tests covering missing fields, invalid types,
and out-of-bounds values for each validator.

Closes dotandev#528
Add missing license headers to files flagged by CI audit checks.
Run gofmt across all Go files to satisfy formatting checks.
Fix xdr.Unmarshal call in verification.go to use bytes.NewReader
instead of raw []byte (io.Reader interface requirement).
- Add mock_signature_verification field to SimulationRequest
- Implement check_signature_verification_mocks helper function
- Update execute_operations to support signature verification mocking
- Add basic tests for mock signature verification functionality
- Enable true/false states for signature verification in mock environment

Resolves: dotandev#613
@dotandev
Copy link
Owner

check ci, they're still failing

1 similar comment
@dotandev
Copy link
Owner

check ci, they're still failing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RPC] Stream decoding integration with Soroban RPC #167