Git

MidnightBSD moved to git in early 2020 for all development.

Checkout MidnightBSD

Read Only use

git clone https://github.com/MidnightBSD/src.git

(also works with svn or svnlite with github)

Developer

git clone [email protected]:MidnightBSD/src.git

Checkout mports

git clone https://github.com/MidnightBSD/mports.git

Developer

git clone [email protected]:MidnightBSD/mports.git

Setup Author

There are at least two methods here. The first is to use git config and the second is environment variables.

Using git config

git config --global user.name "John Doe"
git config --global user.email "[email protected]"

If you only want it for the current repo, don't use the --global

If you screw up a commit and need to fix it before setting this

git commit --amend --author="John Doe <[email protected]>"

Environment Variable

It's global and overrides any git config settings, at least in some versions of git.

export GIT_AUTHOR_NAME="Midnight the Cat"
export GIT_AUTHOR_EMAIL="midnight@localhost"

Checkout Branches

(assuming you've cloned and are in the directory)

Change to the stable 1.2 branch git checkout stable/1.2

See what branch you're on and other local branches as well as the most recent commit on them git branch -v

View remote branches

(assumes remote is called origin) git remote show origin

Interactive Rebase

It's best to read up on this one. https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#Changing-Multiple-Commit-Messages

This will walk you through each commit starting at the one you specified, which should be the last "good" commit.

git rebase -i -p 0ad14ff3

For example, if you want to fix author info: git commit --amend --author="John Doe <[email protected]>" --no-edit

git rebase --continue

Renaming Remote Branches

rename locally

git checkout old-branch
git branch -m new-name

verify name

git branch -a

rename remote (assumes previous steps for local)

git push origin --delete old-name 
git push origin -u new-name

Delete remote branches

git push remote_name -d remote_branch_name

Workarounds

If you start seeing ESC[ type escape sequences with git commands, you likely need to fix your pager.

One solution is

git config --global core.pager "less -R"

Patch for a specific commit

git format-patch -1 <sha>

Vendor Branches

src committers need to make vendor branches to track third-party code.

Creating a new vendor branch

The first option starts with creating a new directory and git repo outside the src directory.

cd /some/where
mkdir foo
cd foo
git init
git checkout -b vendor/foo

The second option instead is to just run

git checkout --orphan vendor/foo

Commit the code

cp -R ~/foo/ .
git add -A -f
git commit -m "Import foo 1.0"

Use git add -A -f, not git add * or plain git add -A. The * glob skips dot files such as .gitignore, and both forms skip anything matched by a .gitignore in the upstream sources. See "Ignored files in upstream sources" below.

Import it into src repo

cd /usr/src
git remote add foo /some/where/foo
git fetch foo vendor/foo

Tag and push

git worktree add ../foo vendor/foo
cd ../foo
git tag --annotate vendor/foo/1.0
git show
git push --follow-tags origin vendor/foo

merge it

cd ../src
git subtree add -P contrib/foo vendor/foo
git show
git commit --amend
git push origin

Update existing vendor branch

git worktree add ../foo vendor/foo
cd ../foo
rsync -va --no-o --no-g --del --exclude=".git" ~/git/foo/ .
git add -A -f
git status
git diff --staged
git commit -m "Vendor import of foo 1.0"

Notes:

• The trailing slash on ~/git/foo/ matters: without it rsync copies into ./foo/foo instead of replacing the worktree contents.
• --no-o --no-g stops rsync from copying the tarball's owner onto the worktree. Otherwise git may refuse the directory with a "dubious ownership" error.
• If the upstream source is a git clone, export the release tag with git --work-tree=/path/to/export checkout v1.0 -- . rather than git archive. git archive leaves out files that upstream marks export-ignore in .gitattributes.

Ignored files in upstream sources

Many upstream projects ship a .gitignore. Once it is copied into the vendor worktree, git applies it, and a plain git add -A silently skips every file it matches. Nothing warns you: the files just never make it into the vendor branch. For example, libarchive's .gitignore contains *.cmake and Makefile, so every libarchive import before 3.8.11 was missing CTestConfig.cmake, contrib/shar/Makefile and examples/minitar/Makefile.

Always stage vendor imports with -f, which adds ignored files too:

git add -A -f

To see whether a .gitignore is hiding anything before you stage:

git status --ignored --short | grep '^!!'

To check afterwards that every upstream file was staged, compare the staged files with the upstream tree:

git ls-files | sort > /tmp/staged.txt
(cd ~/git/foo && find . -type f ! -path './.git/*' | sed 's|^\./||' | sort) \
    > /tmp/upstream.txt
comm -13 /tmp/staged.txt /tmp/upstream.txt

comm -13 prints files that are in upstream but not staged. The list should be empty, or contain only files you deliberately prune (see the package's MIDNIGHTBSD-Xlist).

If upstream is a git clone and you prune nothing, comparing tree hashes is quicker. The two hashes should be identical:

git write-tree                                 # in the vendor worktree
git -C ~/git/foo rev-parse 'v1.0^{tree}'       # upstream release tag

Tag

git tag -a vendor/foo/1.0

git push --follow-tags origin vendor/foo

Merge into master

cd ../src
git subtree merge -P contrib/foo vendor/foo

Never squash a vendor merge

The merge commit must keep the vendor commit as its second parent. Git uses that link to find the merge base the next time you merge from the vendor branch.

If the update goes through a pull request, merge it with Create a merge commit. Never use Squash and merge or Rebase and merge, and say so in the PR description. A squash turns the merge into an ordinary single-parent commit, so the vendor commit is no longer an ancestor of master.

Nothing looks wrong at first. The trouble comes with the next update: git subtree merge picks an older import as the base and reports large numbers of false conflicts. This happened with libarchive 3.8.8 (PR #451), and the 3.8.11 merge hit 54 bogus conflicts.

Before pushing, check that the vendor commit is an ancestor of the branch:

git merge-base --is-ancestor vendor/foo master && echo ok

If an earlier import was squashed, merge using the previous vendor tree as the base. Build two temporary commits on top of master, one with contrib/foo replaced by the old vendor tree and one with the new tree, then three-way merge them:

old=$(git rev-parse vendor/foo/1.0^{tree})      # last import that was squashed
new=$(git rev-parse vendor/foo/1.1^{tree})      # import being merged
for v in old new; do
    eval t=\$$v
    GIT_INDEX_FILE=/tmp/idx.$v git read-tree master
    GIT_INDEX_FILE=/tmp/idx.$v git rm -r -q -f --cached contrib/foo
    GIT_INDEX_FILE=/tmp/idx.$v git read-tree --prefix=contrib/foo/ $t
    eval c_$v=$(GIT_INDEX_FILE=/tmp/idx.$v git write-tree |
        xargs -I{} git commit-tree {} -p master -m tmp)
done
git merge-tree --write-tree --merge-base=$c_old master $c_new

git merge-tree prints the merged tree and lists any conflicts. Resolve the conflicts, then record the result as a merge commit with master as the first parent and the vendor commit as the second (git commit-tree <tree> -p master -p vendor/foo). Later merges then work normally again.

Exit the worktree

git worktree remove foo