Plume-Scripts: Scripts for programming and system administration

These scripts automate various programming and sysadmin tasks. This project contains utilities for:

• Shell scripting
• Continuous integration
• Search and replace
• Sorting
• Unique file contents
• Java
• LaTeX

Installation

To install, run the following (or put it at the top of a script). Then, the scripts are available in the directory named by $PLUME_SCRIPTS.

PLUME_SCRIPTS="/tmp/${USER:-$(id -un)}/plume-scripts"
if [ -d "$PLUME_SCRIPTS" ] ; then
  git -C "$PLUME_SCRIPTS" pull -q > /dev/null 2>&1
else
  git clone --depth=1 -q https://github.com/plume-lib/plume-scripts.git "$PLUME_SCRIPTS"
fi

If you want to use a specific version of plume-scripts rather than the bleeding-edge HEAD, you can run git checkout _SHA_ after the git clone command.

Most of the scripts use sh or bash, but some of the scripts use perl or python.

Shell scripting

cronic

A wrapper for cron jobs so that cron only sends email when an error has occurred. Run cronic --help for its options; also see http://habilis.net/cronic/.

lint-diff.py

Filter the output of tools such as lint, to only show output for changed lines in a diff or pull request. Documentation at top of file.

mail-e

Reads standard input, and if not empty calls the mail program on it. In other words, acts like mail -e and is useful when your version of mail does not support -e. This feature is useful in scripts and cron jobs, but is not supported in all versions of mail. Documentation at top of file.

path-remove

Cleans up a path environment variable by removing duplicates and non-existent directories. Can optionally remove certain path elements. Works for either space- or colon-delimited paths. Documentation at top of file.

Continuous integration

is-ci.sh

Prints "yes" if this process is running under CI. Prints nothing otherwise.

ci-org-and-branch

Outputs shell script code that sets variables CI_ORGANIZATION and CI_BRANCH to the head GitHub organization and branch for a CI job. Works under Azure Pipelines, CircleCI, GitHub Actions, and Travis CI. As a general rule, prefer set-ci-org-and-branch over this script. Documentation at top of file.

set-ci-org-and-branch

When sourced, sets variables CI_ORGANIZATION and CI_BRANCH holding the head GitHub organization and branch for a CI job. Works under Azure Pipelines, CircleCI, GitHub Actions, and Travis CI. Documentation at top of file.

ci-last-success.py

Prints the SHA commit id corresponding to the most recent successful CI job. Documentation at top of file.

git-changes

Outputs shell script code that sets variables CI_COMMIT_RANGE, CI_COMMIT_RANGE_START, and CI_COMMIT_RANGE_END to the start and end commits for a CI job, and CI_ORGANIZATION and CI_BRANCH as ci-org-and-branch does. Works under Azure Pipelines, CircleCI, GitHub Actions, and Travis CI. As a general rule, prefer set-git-range over this script. Documentation at top of file.

set-git-range

When sourced, sets variables CI_COMMIT_RANGE, CI_COMMIT_RANGE_START, and CI_COMMIT_RANGE_END holding the start and end commits for a CI job, along with everything that set-ci-org-and-branch, which it sources, sets. Works under Azure Pipelines, CircleCI, GitHub Actions, and Travis CI. Documentation at top of file.

Search and replace

preplace

Replace all matching regular expressions in the given files (or all files under the current directory). The timestamp on each file is updated only if the replacement is performed. Documentation at top of file.

search

Jeffrey Friedl's search program combines find and grep -- more or less do a 'grep' on a whole directory tree. search is more efficient, uses Perl regular expressions, and is much more powerful. This version fixes a tiny bug or two. For full documentation, see its manpage.

This program has been largely superseded by rg, and before that by pt and ag. However, it is still useful because it searches more thoroughly: in git-ignored files, and in compressed archives.

Sorting

sort-directory-order

Sorts the input lines by directory order: first, every file in a given directory, in sorted order; then, process subdirectories recursively, in sorted order. This is useful for users (e.g., when printing) and for making output deterministic. Documentation at top of file.

sort-compiler-output

Sorts the input errors/warnings by filename. Works for any tool that produces output in the standard format. This is useful for compilers such as javac that process files in nondeterministic order. By default, duplicate messages are printed only once; pass --keep-duplicates to print every message. Documentation at top of file.

Unique file contents

uniq-contents

Given a list of file names as arguments, print the ones with unique contents: for each group of files with identical contents, only the first one (in argument order) is printed. Documentation at top of file.

Java

classfile_check_version

Check that a class file's version is ≤ the specified version. This ensures that the class will run on a particular version of Java. Documentation at top of file.

jacoco-coverage-ratchet

Check that the Jacoco coverage has not decreased for any class, and that no class has disappeared from the report (say, because a class was deleted or renamed, or because a module was not analyzed). Pass --ignore-removed-classes to permit classes to disappear. Documentation at top of file.

LaTeX

latex-process-inputs

Determines all files that are recursively \input by a given LaTeX file. Documentation at top of file. The program has two modes:

• Inline mode (the default): Create a single LaTeX file for the document, by inlining \input commands and removing comments. The result is appropriate to be sent to a publisher.
• List mode: List all the files that are (transitively) \input. This can be useful for getting a list of source files in a logical order, for example to be used in a Makefile or Ant buildfile.