Skip to content

test(sqoop): verify concurrent import log isolation [QUALITY-12403] - #3044

Draft
paulaclemente wants to merge 1 commit into
pdi-20034-testfrom
quality-12403-sqoop-concurrent-it
Draft

paulaclemente wants to merge 1 commit into
pdi-20034-testfrom
quality-12403-sqoop-concurrent-it

Conversation

@paulaclemente

Copy link
Copy Markdown
Contributor

Summary

  • Add an integration test that runs two Sqoop imports in one Kitchen JVM and verifies overlapping Hadoop jobs, independent job/application IDs and counters, and the expected HDFS rows.
  • Start the second import after the first creates its HDFS output directory. Hold the first mapper open briefly and assert actual overlap; this avoids simultaneous Sqoop option initialization, which uses a shared Commons CLI OptionBuilder.

Dependency

Stacked on #3015 (pdi-20034-test), which supplies the integration-test harness. Retarget to master once that PR is merged.

Validation

  • Four successful runs with the gated, held-open fixture and Failsafe retries disabled; the final run also checked YARN application ID isolation.
  • The local Apple Silicon run used an ARM64 PostgreSQL image/platform override in the isolated checkout; that POM adjustment is not part of this PR.
  • Simultaneous Sqoop initialization was observed to fail intermittently before this gate; this test does not cover that startup race.
  • xmllint --noout and git diff --check.

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant