Espresso 4: e2e tests - #475
Draft
lukeiannucci wants to merge 42 commits into
Draft
Conversation
Collaborator
|
Great! Can we start adding the E2E tests? |
Collaborator
Author
The code is in here, I need to ask celo about potentially running it in their CI. Not sure the cost / runners they have. Update i am trying some of them on what should be free runners. |
Closed
lukeiannucci
force-pushed
the
espresso/batcher
branch
from
July 28, 2026 20:49
f227870 to
a243125
Compare
lukeiannucci
force-pushed
the
espresso/e2e-tests-2
branch
2 times, most recently
from
July 31, 2026 12:28
1bc3c45 to
92c832c
Compare
philippecamacho
force-pushed
the
espresso/batcher
branch
from
August 5, 2026 21:44
8a27289 to
4ad160f
Compare
philippecamacho
force-pushed
the
espresso/e2e-tests-2
branch
from
August 5, 2026 22:09
76e8598 to
26b1b75
Compare
Addresses the stall found in the PR 459 review of peekNextBatch: the old streamer could discard a good batch when the channel manager tip and the streamer position described different blocks, stalling it for good. The v2 streamer owns the tip tracking: position and tip hash move together, repositioning is explicit via SetBatchPosition, and Reset / SetProperHead / Refresh / Update are gone. - pin espresso-streamers to 22c396aa (PR 36, op streamer v2) - construct the streamer in StartBatchSubmitting (needs a context and a synced L2 node to resolve its anchor), anchor it at the safe L2 head, and run it from its own poll loops via Start/Stop - drop peekNextBatch and the cachedSyncStatus provider; the loading loop now just Peeks batches extending the tracked tip and advances after the channel manager accepts them - re-anchor instead of skipping on ToBlock/AddL2Block failures - drop the dead BatchStreamerFromCLIConfig (no callers, v1-only API) Co-Authored-By: Claude Fable 5 <[email protected]>
philippecamacho
force-pushed
the
espresso/e2e-tests-2
branch
from
August 6, 2026 10:23
924d586 to
2e8e292
Compare
…t running after an error.
…r reset on the same heads.
…art does not block later start attempts. Co-Authored-By: Claude Fable 5 <[email protected]>
…t startup does not depend on the L2 endpoint serving the historical origin height. Co-Authored-By: Claude Fable 5 <[email protected]>
…oint so that the Espresso batcher never treats pre-activation blocks as its own. Co-Authored-By: Claude Fable 5 <[email protected]>
…k timeout so that a stalled endpoint cannot hang StartBatchSubmitting while it holds the start mutex. Co-Authored-By: Claude Fable 5 <[email protected]>
…cal-safe head so that already-derived blocks are not resubmitted after a restart. Co-Authored-By: Claude Fable 5 <[email protected]>
…choring on a cached unsafe floor block, whose reorg could permanently wedge fork selection. Co-Authored-By: Claude Fable 5 <[email protected]>
philippecamacho
force-pushed
the
espresso/e2e-tests-2
branch
from
August 8, 2026 22:46
131ce7a to
43f91cb
Compare
…ero LocalSafeL2 cannot floor the queue at genesis and re-enqueue derived history. Co-Authored-By: Claude Fable 5 <[email protected]>
…ocalSafeL2 cannot disable the stale-batch re-anchor floor and republish derived blocks. Co-Authored-By: Claude Fable 5 <[email protected]>
philippecamacho
force-pushed
the
espresso/e2e-tests-2
branch
from
August 8, 2026 23:20
43f91cb to
f37529c
Compare
…ent sync-status failure cannot empty the channel manager while leaving the streamer at its old cursor. Co-Authored-By: Claude Fable 5 <[email protected]>
… into RollupConfig Add the optional L2GenesisEspressoTimeOffset deploy-config field and its EspressoTime() accessor, plus a BatchAuthenticatorAddress L1 dependency, and wire both into DeployConfig.RollupConfig so generated rollup configs carry the Espresso fork time and the BatchAuthenticator address used by event-based derivation. Espresso is not a core OP Stack fork, so it is excluded from the ForkTimeOffset/SetForkTimeOffset fork-iteration helpers. Co-authored-by: OpenCode <[email protected]>
Add a deploy-espresso pipeline stage that runs the (PR #455) redesigned scripts/deploy/DeployEspresso.s.sol for chains whose intent has EspressoEnabled, deploying the BatchAuthenticator + TEE verifier (mock verifiers when NITRO_ENCLAVE_VERIFIER_ADDRESS is unset). The opcm wrapper matches the redesigned script's inputs (espressoOwner + sharedProxyAdmin, the latter taken from the chain's shared OP Stack ProxyAdmin) and outputs (batchAuthenticator, teeVerifierProxy, nitroTEEVerifier). Adds ChainIntent.EspressoEnabled/EspressoBatcher, ChainState.BatchAuthenticatorAddress, and activates Espresso at genesis for Espresso-enabled chains in CombineDeployConfig. Co-authored-by: OpenCode <[email protected]>
Add the espresso-enclave / espresso-no-enclave alloc types (config/init.go) with graceful skip when the mock TEE contracts are unavailable, wire the Espresso + fallback batchers, System.L1, SystemConfig.L1Allocs and the EspressoTime/BatchAuthenticatorAddress rollup fields into e2esys, give StartOption.BatcherMod access to the System, add GethInstance.Fork (and System.ForkL1) for L1 reorg tests, and add GetFaultDisputeSystemConfigForEspresso. Co-authored-by: OpenCode <[email protected]>
Generated binding for the EspressoTEEVerifier contract (espresso-tee-contracts submodule), used by the e2e enclave helpers to register enclave PCR0 hashes. Co-authored-by: OpenCode <[email protected]>
Port the non-Caff Espresso end-to-end tests from celo-integration-rebase-17 onto the upstreaming stack: liveness, batch authentication, batch inbox, stateless batcher, L1 reorgs, pipeline enhancement, soft-confirmation integrity, forced transactions, dispute game, batcher fallback, the Espresso enforcement hardfork transition, and the dev-node simple-transaction tests, plus their docker dev-node / attestation-verifier helpers. Caff-node tests and helpers are dropped (the Caff node is handled out of band by espresso-rollup-node-proxy). Adapted to the stack's renames: EspressoEnforcementTime -> EspressoTime, SwitchBatcher() -> SetActiveIsEspresso(bool), and the single-sourced FallbackAuthLeadTime. These tests still launch a real dockerized espresso-dev-node; a mock is a follow-up. Co-authored-by: OpenCode <[email protected]>
Widen BatcherService.EspressoClient and EspressoDriverSetup.Client from the concrete *MultipleNodesClient to the SDK's client.EspressoClient interface, and add the WithEspressoClientOverride DriverSetupOption so tests can inject an in-memory Espresso fake in place of a real espresso-dev-node. Production code never sets the override. Co-authored-by: OpenCode <[email protected]>
Add MockEspressoClient, an in-memory implementation of the SDK's client.EspressoClient interface for e2e tests. It models a HotShot chain as append-only blocks: SubmitTransaction appends to the pending block, a background ticker seals a block every tick so the height advances continuously (as the batcher's verification logic expects), and FetchNamespaceTransactionsInRange / FetchTransactionByHash / FetchLatestBlockHeight round-trip the payloads. The streamer performs no cryptographic verification of HotShot data, so the unused query methods are stubbed. Co-authored-by: OpenCode <[email protected]>
…v node Replace the dockerized espresso-dev-node with the in-memory MockEspressoClient. e2esys.System now owns a single shared mock (System.EspressoClient) for Espresso alloc types, injected into the primary and fallback batchers via WithEspressoClientOverride and stopped on System.Close. The launcher no longer starts Docker; EspressoDevNode is backed by mockEspressoDevNode exposing the shared client via Client(). A fixed dummy light-client address is used when ESPRESSO_SEQUENCER_LIGHT_CLIENT_PROXY_ADDRESS is unset (the streamer tolerates the resulting no-contract error). Tests that built their own client from EspressoUrls() now use espressoDevNode.Client(). The dead dev-node-docker code (container-info types, container-launch helpers, EspressoLightClientAddr, the docker smoke test) is removed; the shared DockerCli infra used by the attestation-verifier and EigenDA helpers is kept. Co-authored-by: OpenCode <[email protected]>
The in-memory mock Espresso client replaced the dockerized espresso-dev-node, so the tests and helpers that depend on external Docker services no longer have a backing service: - Remove TestE2eDevnetWithInvalidAttestation / TestE2eDevnetWithUnattestedBatcherKey (5_batch_authentication_test.go): both gate on the SP1 zk attestation-verifier Docker container, which the mock does not emulate. - Remove attestation_verifier_service_helpers.go (only consumed by those tests) and enclave_helpers.go (its sole live dependency was the attestation helper; the enclave tests were never ported). - Remove espresso_docker_helpers.go and the now-unused EigenDA / Docker helpers in optitmism_espresso_test_helpers.go (StartEigenDA, StopDockerContainer, EIGENDA_* consts, getContainerRemappedHostPort, determineDockerNetworkMode, FailedToLaunchDockerContainer, determineFreePort, and the net import). - TestE2eDevnetWithEspressoAndAltDaSimpleTransactions no longer starts an EigenDA proxy container: WithAltDa enables UseAltDA, which wires the system to the in-process altda.FakeDAServer, so the proxy was never actually contacted. Co-authored-by: OpenCode <[email protected]>
When restarting a TEE batcher mid-chain, CaffeinationHeightEspresso was set to espHeight (FetchLatestBlockHeight, i.e. the chain height / block count). The streamer treats that value as already processed and begins reading from the next height, so it skipped the HotShot block at espHeight where the restarted batcher re-submits its batches; safe L2 never advanced and the verifier stalled. Set it to espHeight-1 (the last already-sealed block) so the streamer reads from espHeight inclusive. Fixes TestBatcherSwitching and TestEspressoEnforcementHardfork. Co-authored-by: OpenCode <[email protected]>
RunSimpleL2BurnWithTimeout accepted a timeout but routed through helpers.SendL2TxWithID, which ignores the caller's context and imposes its own fixed 30s deadline on the verifier receipt wait. After a batcher switch (or with the fallback batcher posting plain calldata in multi-frame channels) the verifier can take well over 30s to re-derive, so the wait timed out and TestBatcherSwitching, TestEspressoEnforcementHardfork, and TestFallbackMechanismIntegrationTestChannelNotClosed failed. Add an Espresso-local sendL2TxAndVerify that honours the supplied ctx (otherwise identical to SendL2TxWithID) and use it from RunSimpleL2BurnWithTimeout, leaving the shared op-e2e helper untouched. Co-authored-by: OpenCode <[email protected]>
- Inject the in-memory mock Espresso client into the batchers these tests start by hand mid-run (WithEspressoClientOverride); without it the restarted/extra batcher has no Espresso client and never produces batches. - Set MaxPendingTransactions=0 (unbounded) for the batchers in these tests so the Espresso auth+batch tx pairs (routed through the ordered txmgr queue) publish concurrently instead of one-per-L1-block; otherwise L1 data availability lags and the verifier cannot derive recent blocks within the tests' windows. - Make GetBatcherConfig a pure snapshot of the batcher CLIConfig and move the channel-tuning (small frames + long channel duration, which force multi-frame channels split across L1 blocks) to explicit WithBatcher* options at the call sites, so the config mutation is visible and GetBatcherConfig does only what its name implies. Co-authored-by: OpenCode <[email protected]>
The test was skipped ("takes a long time to run") but was actually broken: the
hardcoded TEST_ESPRESSO_TRANSACTION fixture was RLP-encoded against an older
3-field EspressoBatch layout, so UnmarshalBatch failed with "rlp: too few
elements" once the SignerAddress field was added. Its final step also waited for
the fixture's L1-info deposit to land on the verifier, which can never happen: a
fixed genesis-era batch is not the next expected batch on a freshly-started
chain, so the batcher never derives it (the source of the long run / timeout).
- Regenerate TEST_ESPRESSO_TRANSACTION in the current 4-field layout (adds the
trailing SignerAddress element; otherwise byte-identical). Also used by the
already-passing TestDeterministicDerivationExecutionStateWithInvalidTransaction.
- Replace the impossible deposit-on-verifier assertion with the test's actual
purpose: the batcher streamer unmarshals the tx and recovers the real batcher
address from the prepended signature, and the batch carries an L1-info deposit.
- Remove the now-unused espressoTransactionDataSkippingUnmarshal helper and unskip.
Co-authored-by: OpenCode <[email protected]>
- 3_2: use batch.SignerAddress — batch.Signer() only exists in a newer espresso-streamers than the pinned one; broke compilation of every espresso-e2e job and go-lint - 14: TxManagerIntercept now also intercepts SendPairAsync — the batcher submits frames as auth+batch pairs via Queue.SendPair, so the simulated failure never fired and TestFallbackMechanismIntegrationTestChannelNotClosed waited out its whole budget; also bound StopBatchSubmitting with a short context, since a batcher whose every submission fails can never drain gracefully and only force-kills when the passed context expires - 15: rotate the SystemConfig batcher hash to the TEE batcher key when handing over inside the enforcement grace window — the verifier authorizes by L1 sender alone until EspressoTime + BatchAuthEnforcementDelaySecs (auth events are not scanned yet), so the phase-4 TEE batches were dropped as 'tx in inbox with unauthorized submitter' Both tests pass locally (388s / 379s vs deterministic 600s / 490s failures). Co-Authored-By: Claude Fable 5 <[email protected]>
…ration - 6: zero the BatchAuthenticatorAddress only after StartBatchSubmitting — the v2 streamer is constructed there and requires the real contract to validate batches read back from Espresso. Zeroing afterwards still makes the publish path send authenticateBatchInfo to the zero-address EOA, which succeeds without emitting the event, so batches land on the BatchInbox unauthenticated as the test intends - 8: TestBatcherWaitForFinality now asserts the safe head's L1 origin against the L1 chain's own finalized tag instead of the verifier's FinalizedL1 view — the batcher gates submission on the sequencer node's finality view, and the verifier's finality poller can briefly lag it, tripping the assertion without any batcher misbehavior Both pass locally (67s / 44s). Co-Authored-By: Claude Fable 5 <[email protected]>
…lity Use bigs.Uint64Strict instead of big.Int.Uint64, as required by the custom bigint analyzer. Co-Authored-By: Claude Fable 5 <[email protected]>
Make the missing Espresso tests visible in the tree and in CI output as SKIP instead of silently absent, each with a TODO describing what the port needs: - espresso/devnet-tests: stub package mirroring the reference repo's devnet test tier (smoke, batcher active-publish/restart/switching, challenge game, forced transaction, key rotation, withdrawal). The real bodies drive a docker-compose devnet and, in the TEE profile, a batcher in a real AWS Nitro enclave via the ec2-devnet-test.yaml workflow — none of that infrastructure exists here yet; the package doc carries the porting checklist. - espresso/environment/5_batch_authentication_test.go: restore TestE2eDevnetWithInvalidAttestation and TestE2eDevnetWithUnattestedBatcherKey as skipped stubs — their original bodies (dropped in 3dd70bc) gate on the SP1 zk attestation verifier, which the in-memory mock does not emulate. - espresso-e2e-tests.yaml: include ./espresso/devnet-tests/... in the test path and add the stub names to the core group filter so every stub surfaces as SKIP in CI. Co-Authored-By: Claude Fable 5 <[email protected]>
flake.nix/flake.lock are local tooling for running the e2e suite in a nix dev shell; they were swept into the previous commit by accident. Co-Authored-By: Claude Fable 5 <[email protected]>
Bring over espresso-devnet-tests.yaml, espresso-devnet-tests-tee.yaml and the reusable ec2-devnet-test.yaml from optimism-espresso-integration (celo-integration-rebase-17) so the devnet/real-TEE tier is visible in this repo next to its skipped test stubs (espresso/devnet-tests). Each carries a TODO header listing what enabling needs: the compose devnet + devnet_tools harness, compile-contracts.yaml and the cache-submodules action, and for the TEE path the Nitro-enclave EC2 runner infrastructure (ec2-github-runner enclave fork, AWS secrets and role, nitro-cli AMI). The two callers trigger on workflow_dispatch only until then; the reusable workflow is workflow_call-only and inert. Co-Authored-By: Claude Fable 5 <[email protected]>
Full diff of test functions against optimism-espresso-integration (celo-integration-rebase-17) found ten tests absent from this fork. Resolution by category: - espresso_dev_node_logs_test.go (TestAnsiEscapeCodeLineReader, TestEspressoDevNodeLogEntry, TestEspressoDeployedContractLogEntryReader): ported live — they unit-test the log-reader helpers that already ship here, and pass. - espresso/enclave-tests (TestE2eDevnetWithEspressoAndEnclaveSimpleTransactions): new stub package for the enclave tier — the real body needs a Nitro-enabled EC2 runner and the op-batcher enclave image. - TestEspressoDockerDevNodeSmokeTest: skipped stub — the dockerized espresso-dev-node it smoke-tests was replaced by the in-memory mock. - Caff-node tests (TestE2eDevnetWithEspressoFastConfirmationStability, TestE2eDevnetWithEspressoWithCaffNodeDeterministicDerivation, TestFastDerivationAndCaffNode, TestCaffNodeWaitForFinality, TestE2eDevnetWithEspressoDegradedLivenessViaCaffNode): intentionally NOT ported — the Caff node is no longer supported. The e2e workflow now also runs ./espresso/enclave-tests/... and lists the new stub and log-reader test names in the core group filter. Co-Authored-By: Claude Fable 5 <[email protected]>
Co-Authored-By: Claude Fable 5 <[email protected]>
Pins the same toolchain the espresso-e2e-tests workflow uses in CI (go 1.26, just, gotestsum, forge 1.2.3 official release binaries) so the tests can be run locally with 'nix develop' instead of mise. Co-Authored-By: Claude Fable 5 <[email protected]>
philippecamacho
force-pushed
the
espresso/e2e-tests-2
branch
from
August 8, 2026 23:47
3ef0782 to
7b0ced9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Espresso 4: e2e tests
Ports the Espresso end-to-end test suite into the monorepo as the
espresso/environmentpackage and runs it in CI on free GitHub runners. The tests exercise the full Espresso integration — batcher, streamer, derivation pipeline, BatchAuthenticator contract — against an in-memory mock Espresso dev node (espresso.NewMockEspressoClient), so no docker or external espresso-dev-node is needed in CI or locally.What's included
espresso/environment: the e2e test suite (soft-confirmation integrity, forced transactions, batcher fallback/switching, L1 reorgs, batch-inbox authentication, stateless batcher, pipeline enhancement, the EspressoTime enforcement hardfork, dispute game)..github/workflows/espresso-e2e-tests.yaml: runs the suite as four parallel matrix groups (core,integrity,fallback,forced-reorg) with a forge-artifacts cache; contracts-only, no cannon prestates needed.espresso/mock_client.go) shared by the batchers and the tests, injected viaWithEspressoClientOverride.StartBatchSubmitting, anchored at the safe L2 head, and runs its own poll loops (replaces the stall-pronepeekNextBatch/SetProperHeadflow flagged in PR 459).Espresso e2e tests skipped for now
Every skipped test exists in the tree as a
t.Skip(visible as SKIP in CI output) with a TODO describing what unblocks it.TestOutputAlphabetGameWithEspresso_ChallengerWins(13_dispute_game_test.go)disputematrix group is commented out in the workflowTestEnforceMajorityRule(12_enforce_majority_rule_test.go)t.Skip)TestE2eDevnetWithInvalidAttestation,TestE2eDevnetWithUnattestedBatcherKey(5_batch_authentication_test.go)3dd70bc97d^). The e2e devnet deploys permissive mock TEE verifiers, so the suite exercises the TEE auth flow (EIP-712 commitment,authenticateBatchInfo, event-based derivation auth) but not attestation verification itself — that is covered byBatchAuthenticator.t.solunit tests and the vendoredespresso-tee-contractsrepo.TestSmoke,TestBatcherActivePublishOnly,TestBatcherRestart,TestBatcherSwitching,TestChallengeGame,TestForcedTransaction,TestChangeBatchAuthenticatorOwner,TestWithdrawal(newespresso/devnet-testspackage)espresso/devnet-testsso the eventual port is a drop-in; the package doc carries the full porting checklist.TestE2eDevnetWithEspressoAndEnclaveSimpleTransactions(newespresso/enclave-testspackage)TestEspressoDockerDevNodeSmokeTest(espresso_dev_node_test.go)The test inventory was diffed function-by-function against the reference repo (
celo-integration-rebase-17). Two categories closed the gap without stubs:TestAnsiEscapeCodeLineReader,TestEspressoDevNodeLogEntry,TestEspressoDeployedContractLogEntryReader) — they exercise helpers that already ship here and pass.TestE2eDevnetWithEspressoFastConfirmationStability,TestE2eDevnetWithEspressoWithCaffNodeDeterministicDerivation,TestFastDerivationAndCaffNode,TestCaffNodeWaitForFinality,TestE2eDevnetWithEspressoDegradedLivenessViaCaffNode) — the Caff node is no longer supported.Every other test in
espresso/environmentis part of one of the four CI groups and passes.Devnet / real-TEE test tier — stubbed and disabled, pending port
The reference repo (optimism-espresso-integration) has a second, heavier test tier that this PR ships in stub/disabled form: the
espresso/devnet-testspackage (skipped stubs, above) running against a docker-compose devnet, and its workflows, now checked in under.github/workflows/with TODO headers:espresso-devnet-tests.yaml— standard compose profile (triggers reduced toworkflow_dispatchwhile disabled)espresso-devnet-tests-tee.yaml+ the reusableec2-devnet-test.yaml— runs each test group on an ephemeral AWS Nitro-enclave-enabled EC2 runner with the real enclave batcher and attestation flow (ESPRESSO_RUN_ENCLAVE_TESTS=true,COMPOSE_PROFILES=tee); the caller isworkflow_dispatch-only and the reusable workflow isworkflow_call-only, so neither runs automatically (reference)Enabling them needs infrastructure this repo doesn't have yet (each workflow's TODO header lists it): the compose devnet and
devnet_tools.goharness,compile-contracts.yamland thecache-submodulesaction, theEspressoSystems/ec2-github-runnerenclave fork, repository AWS secrets (role, subnet, security group), and the Nix+Docker+nitro-cli AMI. Until then, real-TEE coverage continues to run in the reference repo; this PR's CI runs the in-process e2e tier and surfaces the stubbed tier as SKIP.Test-suite fixes worth calling out
Queue.SendPair→TxManager.SendPairAsync; test doubles that wraptxmgr.TxManagermust overrideSendPairAsynctoo (seeTxManagerIntercept).EspressoTime→+ derive.BatchAuthEnforcementDelaySecs) the verifier authorizes batches by L1 sender only — handing the chain to the TEE batcher inside that window requires rotatingSystemConfig.batcherHashto its address (exercised by the hardfork test).StopBatchSubmittingforce-kills only when the passed context expires, so tests that wedge the batcher on purpose stop it with a short-lived context.CI status
All four espresso-e2e groups, go-lint, and ci-gate are green at the head of this branch.
🤖 Generated with Claude Code