fix(Spanner): Make System Tests Work & Work Faster

[cy-yun]

This PR addresses the ongoing flakiness, high database provisioning overhead, and parallel-execution collisions of the Spanner system tests.

By consolidating test database provisioning, we reduce overall execution time through database reuse. Furthermore, we've removed structural bottlenecks (such as ID collisions and schema mismatches) that prevented these tests from running concurrently. Configurations were also added to allow for both sequential and parallel execution of the tests (using paratest, non-functional mode).

Additionally, BackupTest was significantly refactored, and backup2 was removed entirely without impacting test coverage. This sped up the backup tests by roughly 33%.

Below is a detailed breakdown of the specific problems and fixes implemented in this PR:

1. Test Database Provisioning Overhead & DDL Consolidation

• Problem: The system test suite was creating and dropping a new test database for every single test class, adding massive overhead to the already long execution time. Furthermore, individual test classes independently executed DDL statements (CREATE TABLE), causing Table already exists collisions when tests were executed against a shared persistent database.
• Fix:
  • Introduced TestDatabaseManager to provision a shared Spanner test database and reuse it across all test classes.
  • Centralized all table and schema creation logic into unified traits: SystemTestCaseTrait.php (Standard Spanner) and PgSystemTestCaseTrait.php (Postgres Spanner).
  • Removed redundant CREATE TABLE and DROP TABLE operations from over a dozen individual test classes (e.g., BatchTest, WriteTest, PgBatchTest).

2. PostgreSQL DDL Compatibility & Schema Mismatches

• Problem: Reusing the same database across multiple test runs or classes caused PostgreSQL CREATE TABLE commands to fail because the tables already existed. Additionally, there was schema drift between individual tests.
• Fix:
  • Added IF NOT EXISTS to PostgreSQL DDL statements in the Pg*Test classes.
  • Reconciled schema drift and mismatches in PgQueryTest and PgPartitionedDmlTest to align exactly with the unified setup suite definitions.
  • Adjusted PgReadTest.php so that secondary index creation operates safely on shared databases without conflicting with concurrent tests.

3. Row ID Collisions on Persistent Databases

• Problem: Because multiple tests (like TransactionTest, OperationsTest, BackupTest) share the same Users test table, tests running sequentially on a persistent database were slowly filling up the table. Tests generated random keys using narrow ranges like rand(1, 346464), leading to frequent ALREADY_EXISTS errors during insertion due to the Birthday Paradox.
• Fix:
  • Updated the central SystemTestCase::randId() generator to use a massive 1-billion key space: rand(1, 999999999).
  • Replaced numerous instances of hardcoded, narrow rand() ranges in tests like TransactionTest.php and PgTransactionTest.php to use the unified self::randId() helper.
  • Used insertOrUpdate mutations in places where exact state tracking wasn't required to prevent persistent insertion failures.

4. BackupTest Reliability, Timeouts & Speed Optimization

• Problem: BackupTest operations were flaking due to transient errors, DEADLINE_EXCEEDED errors when GCP was slow, and improper polling configurations. Creating multiple backups was also heavily bottlenecking the test suite.
• Fix:
  • Refactored BackupTest to remove backup2, completely eliminating redundant backup operations without impacting test coverage and bringing the time down by ~33%.
  • Updated long-running operation polling to use maxPollingDurationSeconds via an extended timeout loop for all Backup and Restore operations.
  • Refactored the timeout try/catch logic into a DRY pollWithExtendedTimeout($op) helper method.
  • Caught and ignored DEADLINE_EXCEEDED exceptions emitted during intermediate polling attempts.

5. CI Configuration & Parallel Execution

• Problem: Spanner system tests were not executing in the standard CI pipeline, and their architecture blocked parallelization.
• Fix:
  • Enabled the Spanner test suite in phpunit-system.xml.dist now that stability is restored.
  • Architected tests and provided configurations to allow seamless execution through paratest in non-functional parallel mode.

[Hectorhammett]

It lgtm but there seems to be some tests failing 🤨