Bad diff in (at least) one crate

[Alphare]

Hi, it's been a while! Glad to see this project's still going.

I was in the process of upgrading a dependency and saw an alarmingly huge diff. As you can see, it looks like the whole project's been rewritten!

Upon further inspection and looking for a simple case, I saw that the changelog had a very suspicious diff. Most of it should just be identical, with the 8.0.0 lines added.

So I cloned the repo, ran git diff --ws-error-highlight=new,old 7.0.3 8.0.0 CHANGELOG.md (the whitespace flag is here in case there's something funky going on) and, sure enough, the diff is much smaller there.

I don't have time to do more digging (I typed "diffing" here by accident heh), so I thought I'd leave a bug report.

[PigeonF]

I don't think diff.rs shows you a bad diff. While there may not be changes in the git diff, I think the issue is that version 7.0.3 was published from Windows, and 8.0.0 from Linux.

You have to diff the versions that are uploaded to crates.io, not the diff from the git repository.

$ file ~/.local/share/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/which-7.0.3/CHANGELOG.md
/Users/pigeonf/.local/share/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/which-7.0.3/CHANGELOG.md: ASCII text, with CRLF line terminators
$ file ~/.local/share/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/which-8.0.0/CHANGELOG.md
/Users/pigeonf/.local/share/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/which-8.0.0/CHANGELOG.md: ASCII tex

As you can see, the uploaded files have different line endings, which leads to the huge diffs.

P.S. This is probably due to the git core.autocrlf setting, which pushes the file with LF line endings, but keeps it checked out with CRLF line endings on Windows.

[Boddlnagg]

I came across the same problem looking at the diff for my crate midir.
Most files show every single line as changed (except Cargo.lock and Cargo.toml, interestingly; it looks like Cargo.toml gets rewritten by the publishing process).
It is very likely that I published one of the versions from Windows, the other from Linux.
It would be great if diff.rs could apply some filter here to handle such cases more gracefully and show the "real" diff.

Edit: It seems like there's already a PR to fix this: #88

[xfbs]

Ah yeah, this one is a bit tricky. Windows has that feature bug where they insert \r before the newline character \n. This can trip up a lot of workflows, and it results in brittle special handling for windows systems in many tools (eg git, git-lfs). I think a long-term fix would be to always publish crates on systems that don't have this issue (Linux/macOS, WSL2 on Windows), and maybe for Cargo to strip these broken newlines when you publish.

It is a bit tricky, because those lines are different (they differ in invisible characters). I don't think it is smart to disregard invisible characters by default. The issue here is that a different diff algorithm would show this better, like if instead of rendering it as:

-target
-Cargo.lock
-.vscode
-.cargo
-*.iml
+target
+Cargo.lock
+.vscode
+.cargo
+*.iml
.idea

We could render this as:

target<CR>
Cargo.lock<CR>
.vscode<CR>
.cargo<CR>
*.iml<CR>
.idea

Where only the carriage return is shown as "inserted" or "deleted", and highlighted in green/red.

Or maybe we could have an opt-in toggle that would allow people to "Legacy newline normalization" or something like that. The reason I am thinking this is that there are cases where invisible characters in source code can result in a change of behavior, so we should default to showing all changes — but either render them in a way that doesn't create noise (as in this case) or have opt-in features to hide them.

At least this is what I am thinking. I could be wrong about this.

[Boddlnagg]

I would be absolutely happy with an opt-in toggle that switches the diff behavior.