Skip to content

Rewrite the CSV file through a private copy next to it and never replace it with a fragment - #131

Open
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:csv-temp-files-and-locks
Open

vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:csv-temp-files-and-locks

Conversation

@vharseko

Copy link
Copy Markdown
Member

Closes the medium CodeQL alerts java/local-temp-file-or-directory-information-disclosure #20 #21 #22 and java/unreleased-lock #15, and fixes a data-loss bug found on the way.

CSV connector: how update and delete rewrite the file

doUpdate and doDelete rebuild the whole CSV — accounts, attributes, password hashes — in a File.createTempFile("csvfile", "tmp"), i.e. in the system temp directory with the umask permissions (0644 as a rule), where any local user can read it while it is written. Then the finally block unconditionally deleted the CSV and moved the temp file over it. Two consequences, both reproduced by the new tests against the old code:

  • a rewrite that fails half-way (the tests use a row with the wrong number of columns after the row being updated) replaced the data file with the rows written so far — everything after the failure was gone;
  • the CSV came back with the temp file's permissions: a file kept at 0600 was 0644 after every update or delete.

Now:

  • createRewriteFile() creates the copy next to the CSV (<csv>.<random>.tmp, outside the <csv>.<13 digits> pattern of the sync copies) with Files.createTempFile: owner-only on POSIX, same file system, so the replacement is a rename.
  • finishRewrite(tmp, rewritten) replaces the CSV only when the rewrite completed: the copy takes over the CSV's permissions (skipped on non-POSIX file systems) and moves into place with REPLACE_EXISTING + ATOMIC_MOVE (plain move as fallback). A copy of a failed rewrite is deleted. A failed replacement is now a ConnectorIOException instead of a logged-and-swallowed error that left the caller believing the operation succeeded.
  • lock.unlock() sits in its own inner finally in doCreate, doDelete, doUpdate (Bump commons-io:commons-io from 2.2 to 2.7 in /OpenICF-java-framework/connector-test-common #15): a RuntimeException from a close() can no longer keep the write lock forever and hang every later operation on the file. The three copies of the close-and-log code became closeQuietly.
  • FileUtils.moveFile was the only use of commons-io; the dependency is removed.

Bundle temp directory (#22)

LocalConnectorInfoManagerImpl created java.io.tmpdir/bundle-<random> with mkdir() (umask permissions) for the expanded lib/ and native/ entries of a connector bundle. It now uses Files.createTempDirectory, owner-only on POSIX; the name-probing loop goes with it.

Tests

  • RewriteSafetyTest (new, 4): update/delete keep a 0600 data file at 0600 (POSIX, otherwise skipped); a failed update/delete leaves the file byte-for-byte as it was. All four failed on the old code.
  • LocalConnectorInfoManagerTests.testBundleTempDirectoryIsPrivate (new): a freshly expanded bundle directory is rwx------ (POSIX, otherwise skipped). Uses its own LocalConnectorInfoManagerImpl so the factory cache and connector pools of the other tests are untouched.
  • The unlock() ordering has no test: it would need a close() that throws inside a private method.

Local runs: connector-framework-internal 470 tests, csvfile-connector 84 tests, all green.

Not changed

java/unreleased-lock #13 (sync: the read lock taken while downgrading from the write lock is released by the following try/finally; nothing can throw in between) and #14 (findObjectInFile: if (rwLock != null) lock … finally { if (rwLock != null) unlock }) are false positives of the heuristic and will be dismissed as such.

…ace it with a fragment

Update and delete rebuilt the CSV in a File.createTempFile in the system
temp directory - readable by other local users while the whole file,
passwords included, was written there - and moved it over the CSV
unconditionally, so a rewrite that failed half-way replaced the data
with the rows written so far, and the CSV came back with the temp file's
permissions instead of its own.

The copy is now created next to the CSV with owner-only permissions,
takes over the CSV's permissions and replaces it atomically, and only
when the rewrite completed; otherwise it is deleted. The write lock is
released in its own finally so that a failing close() can not keep it.

The directory a connector bundle is expanded into is created with
Files.createTempDirectory, owner-only on POSIX file systems.
@vharseko vharseko added security Security fix / CVE remediation java Pull requests that update java code tests Test additions or fixes framework OpenICF-java-framework connector:csvfile CSV file connector labels Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

connector:csvfile CSV file connector framework OpenICF-java-framework java Pull requests that update java code security Security fix / CVE remediation tests Test additions or fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant