Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,19 @@ because it turns other people's test suites red.

### Added

- **A preset for unusual file names: `filename-handling`.** It answers "will
my system store, show and give back a file name it did not expect?" with
fifty names in seven groups: scripts from Polish to Korean, names that look
like other names, leading spaces and dots, shell and SQL metacharacters,
names that mean something to a web server or a desktop, names read as
values, and names at the length limits. Every one is written byte for byte
on Windows, Linux and macOS - measured on NTFS, ext4 and APFS. On another
file system a name may be refused, and the run then ends with code 8 and
names it. The files are `txt` unless `--format` says otherwise, and the
names about length count the format's extension in. Four names are
expected to be accepted, the rest are left to your system's policy with a
reason.

- **Every format has its full name.** `tfg formats` has a `NAME` column
(`jxl` is JPEG XL, `png` Portable Network Graphics), `tfg formats jxl`
gives it on a `name` line, and `tfg formats --json` carries it under the new
Expand Down Expand Up @@ -507,6 +520,32 @@ because it turns other people's test suites red.

### Fixed

- **A report shows a character nobody can see in a file name as an escape.**
`verify`, `cleanup`, the notes of a run, every error message and the
refusals in the window printed such a character as it was, in a file name
and in the name of a folder. A right to left override then made the terminal draw
another name than the one on the disk, and a zero width space made two
names look the same. Such a character is printed as an escape now, such as
`\u202e` for a right to left override. A name without one is printed
as before. The manifest and every `--json` report still carry the
exact name.

- **A recipe the tool writes shows a character nobody can see as an escape.**
`tfg preset eject` wrote a right to left override, a zero width space or a
line separator into the recipe as it was, so the file read as something
other than what it held, and a YAML 1.1 reader such as PyYAML refused it at
the line separator. Such a value is written in double quotes now, with the
character as an escape that every YAML reader turns back into the same
character. Every other value is written as before. A run started in the
window records a different recipe hash only when a value holds such a
character.

- **A space from outside ASCII at the start or the end of a recipe value is
kept.** A file name beginning with an ideographic space or a no break space
lost it, and the file was written under a different name than the recipe
asked for, with nothing said. A plain space or a tab at the ends of an
unquoted value is still not part of it, as in any YAML file.

- **A file name from 238 to 255 bytes long is written.** Every system stores
such a name, and none of them got one: each file is written under a longer
temporary name first, and that one was over the limit. The same held for a
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -639,7 +639,8 @@ when a number is a placeholder of ours rather than a limit of yours. Presets are
ordinary recipes underneath - `tfg preset eject size-boundaries` prints the
recipe and you edit it from there.

One preset ships today, `size-boundaries`. More are designed.
`tfg preset list` names every preset your build ships, and the Presets
screen of the window offers the same ones.

## 🖥️ The desktop window

Expand Down
5 changes: 4 additions & 1 deletion internal/audit/audit.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@ type Difference struct {
}

func (d Difference) String() string {
// Shown on a copy, so the paths a program compares stay as they are. See
// core.Shown for the name this printed wrongly (O241).
d.Path, d.Want, d.Got = core.Shown(d.Path), core.Shown(d.Want), core.Shown(d.Got)
switch d.Kind {
case Missing:
return fmt.Sprintf("missing %s", d.Path)
Expand Down Expand Up @@ -191,7 +194,7 @@ func (e *EscapeError) Error() string {
"the manifest lists %q, which lands outside %s once the links on the way are followed. "+
"This tool never reads or removes anything outside the directory it was pointed at, so it will not act on this manifest. "+
"Check that the directory is the one the run wrote to, and that nothing inside it points elsewhere.",
e.Path, e.Dir)
e.Path, core.Shown(e.Dir))
}

// resolved turns a manifest entry into the path on disk, refusing one that
Expand Down
12 changes: 6 additions & 6 deletions internal/cli/cleanup.go
Original file line number Diff line number Diff line change
Expand Up @@ -117,13 +117,13 @@ func previewCleanup(cands []audit.Candidate, path, dir string, force, asJSON boo
return writeJSON(out, errOut, report, ExitOK)
}

fmt.Fprintf(out, "%s would be removed from %s:\n", core.Count(countRemovable(cands, force), "file", "files"), dir)
fmt.Fprintf(out, "%s would be removed from %s:\n", core.Count(countRemovable(cands, force), "file", "files"), core.Shown(dir))
for _, c := range cands {
if c.Removable(force) {
fmt.Fprintf(out, " remove %s\n", c.Path)
fmt.Fprintf(out, " remove %s\n", core.Shown(c.Path))
continue
}
fmt.Fprintf(out, " keep %s - %s\n", c.Path, skipNote(c, force))
fmt.Fprintf(out, " keep %s - %s\n", core.Shown(c.Path), skipNote(c, force))
}
fmt.Fprintf(errOut, "Nothing was removed. Run the same command with --yes to remove them.\n")
return ExitOK
Expand Down Expand Up @@ -153,7 +153,7 @@ func applyCleanup(ctx context.Context, cands []audit.Candidate, path, dir string
}
report.Files = append(report.Files, cleanupEntry{Path: o.Path, Action: "kept", Reason: o.Reason})
if !asJSON {
fmt.Fprintf(errOut, "kept %s - %s\n", o.Path, o.Reason)
fmt.Fprintf(errOut, "kept %s - %s\n", core.Shown(o.Path), core.Shown(o.Reason))
}
}
// Kept counts every entry that is not removed, which is what the entries
Expand All @@ -179,7 +179,7 @@ func applyCleanup(ctx context.Context, cands []audit.Candidate, path, dir string
if blocked > 0 {
fmt.Fprintf(errOut, "tfg: the manifest was kept. It is the only record of %s still on disk.\n", core.Count(blocked, "file", "files"))
} else if err := os.Remove(path); err != nil {
fmt.Fprintf(errOut, "tfg: cannot remove the manifest %s: %s\n", path, describeError(err))
fmt.Fprintf(errOut, "tfg: cannot remove the manifest %s: %s\n", core.Shown(path), describeError(err))
return ExitIO
}
}
Expand All @@ -194,7 +194,7 @@ func applyCleanup(ctx context.Context, cands []audit.Candidate, path, dir string
return writeJSON(out, errOut, report, ExitOK)
}

fmt.Fprintf(out, "%s removed from %s\n", core.Count(removed, "file", "files"), dir)
fmt.Fprintf(out, "%s removed from %s\n", core.Count(removed, "file", "files"), core.Shown(dir))

// A file left behind is not a silent outcome. It was reported above, and
// the exit code has to carry it too or a script never learns.
Expand Down
13 changes: 13 additions & 0 deletions internal/cli/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"syscall"

"github.com/donislawdev/TestingFilesGenerator/internal/audit"
"github.com/donislawdev/TestingFilesGenerator/internal/core"
"github.com/donislawdev/TestingFilesGenerator/internal/damage"
"github.com/donislawdev/TestingFilesGenerator/internal/engine"
"github.com/donislawdev/TestingFilesGenerator/internal/format"
Expand All @@ -33,7 +34,19 @@ import (
// So the system's sentence is swapped for ours and every layer of our own
// context above it is kept. The number it carried stays, because a number means
// the same thing in every language and it is what somebody puts into a search.
//
// And a character nobody can see is shown rather than left to act, in every
// message at once (O241). A system error wrapped under our own sentence
// repeats the path it failed on in its own words, raw, after our sentence had
// shown it escaped - found in a review on 2026-09-25. One funnel covers every
// command, including the ones that print a message nobody wrote with a name
// in mind.
func describeError(err error) string {
return core.ShownText(inOurWords(err))
}

// inOurWords is describeError before anything is escaped.
func inOurWords(err error) string {
if err == nil {
return ""
}
Expand Down
8 changes: 4 additions & 4 deletions internal/cli/generate.go
Original file line number Diff line number Diff line change
Expand Up @@ -564,7 +564,7 @@ func echoBoundaries(targets []engine.Target, planned []engine.PlannedFile, errOu
fmt.Fprintf(errOut, "boundary %q around %s:\n", t.ID, core.ExactBytes(t.BoundaryLimit))
for _, f := range planned {
if f.Target == t {
fmt.Fprintf(errOut, " %-26s %s\n", f.Name, core.ExactBytes(f.Plan.Bytes))
fmt.Fprintf(errOut, " %-26s %s\n", core.Shown(f.Name), core.ExactBytes(f.Plan.Bytes))
}
}

Expand Down Expand Up @@ -656,7 +656,7 @@ func saveManifest(res *engine.Result, opt engine.Options, errOut io.Writer) int
// way is a chance for the saver and the claim to mean different files.
path := engine.ManifestPath(opt)
if err := res.Manifest.Save(path); err != nil {
fmt.Fprintf(errOut, "tfg: cannot write the manifest to %s: %s\n", path, describeError(err))
fmt.Fprintf(errOut, "tfg: cannot write the manifest to %s: %s\n", core.Shown(path), describeError(err))
// What that leaves behind, because the line above is about the manifest
// and the person's problem is the files. Rule 6: a run that wrote files
// nothing can remove says so rather than leaving it to be discovered by
Expand All @@ -667,10 +667,10 @@ func saveManifest(res *engine.Result, opt engine.Options, errOut io.Writer) int
if n := len(res.Manifest.Files); n > 0 {
fmt.Fprintf(errOut,
"tfg: %s written and nothing to record what this run left. Cleanup works from a manifest, so clearing %s is a job by hand.\n",
core.Count(n, "file", "files"), opt.OutDir)
core.Count(n, "file", "files"), core.Shown(opt.OutDir))
}
return ExitIO
}
fmt.Fprintf(errOut, "manifest: %s\n", path)
fmt.Fprintf(errOut, "manifest: %s\n", core.Shown(path))
return ExitOK
}
13 changes: 7 additions & 6 deletions internal/cli/verify.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ Flags:
dir = filepath.Dir(path)
}
if info, statErr := os.Stat(dir); statErr != nil || !info.IsDir() {
fmt.Fprintf(errOut, "tfg: cannot read the directory %s. Check the path and that you have permission to read it.\n", dir)
fmt.Fprintf(errOut, "tfg: cannot read the directory %s. Check the path and that you have permission to read it.\n", core.Shown(dir))
return ExitIO
}

Expand Down Expand Up @@ -150,7 +150,7 @@ func reportVerify(diffs []audit.Difference, claimed int, path, dir string, asJSO
}

if wrong > 0 {
fmt.Fprintf(errOut, "tfg: %s does not match %s - %s:\n", dir, path, core.Count(wrong, "difference", "differences"))
fmt.Fprintf(errOut, "tfg: %s does not match %s - %s:\n", core.Shown(dir), core.Shown(path), core.Count(wrong, "difference", "differences"))
echoMismatches(diffs, errOut)
echoOtherRuns(diffs, errOut)
return ExitVerify
Expand All @@ -160,11 +160,11 @@ func reportVerify(diffs []audit.Difference, claimed int, path, dir string, asJSO
// "everything is fine" about zero files invites somebody to trust a run
// that never happened.
if claimed == 0 {
fmt.Fprintf(errOut, "%s claims no files, so there was nothing to check.\n", path)
fmt.Fprintf(errOut, "%s claims no files, so there was nothing to check.\n", core.Shown(path))
echoOtherRuns(diffs, errOut)
return ExitOK
}
fmt.Fprintf(out, "%s matches %s: %s checked\n", dir, path, core.Count(claimed, "file", "files"))
fmt.Fprintf(out, "%s matches %s: %s checked\n", core.Shown(dir), core.Shown(path), core.Count(claimed, "file", "files"))
echoOtherRuns(diffs, errOut)
return ExitOK
}
Expand Down Expand Up @@ -215,11 +215,11 @@ func echoOtherRuns(diffs []audit.Difference, errOut io.Writer) {
for _, name := range names {
files := byRecord[name]
if len(files) == 0 {
fmt.Fprintf(errOut, "note: %s is another run's record, and nothing else here belongs to it.\n", name)
fmt.Fprintf(errOut, "note: %s is another run's record, and nothing else here belongs to it.\n", core.Shown(name))
continue
}
fmt.Fprintf(errOut, "note: %s is another run's record. %s here %s to it: %s.\n",
name, core.Count(len(files), "file", "files"), belongs(len(files)), someOf(files))
core.Shown(name), core.Count(len(files), "file", "files"), belongs(len(files)), someOf(files))
}
}

Expand Down Expand Up @@ -249,6 +249,7 @@ func groupedByRecord(diffs []audit.Difference) map[string][]string {

// someOf names the first few and counts the rest.
func someOf(names []string) string {
names = core.ShownEach(names)
if len(names) <= otherRunExamples {
return strings.Join(names, ", ")
}
Expand Down
107 changes: 107 additions & 0 deletions internal/core/unseen.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
package core

import (
"strconv"
"strings"
"unicode/utf8"
)

// Shown is s the way a person should read it: every character HoldsUnseen
// finds, and every byte that is not UTF-8, written as the escape %q would use
// for it, and nothing else changed. No quotes are added.
//
// For every line this tool prints about a name or a path that came from a
// recipe, a preset, a manifest or a directory listing (O241). Measured on
// 2026-09-24: verify reported a missing "photo", right to left override,
// "gpj.txt" as the terminal drew it, which is "phototxt.jpg", and an extra
// "in", zero width space, "voice.txt" as "invoice.txt" - a report naming files
// other than the ones on the disk, two of which could not be told apart.
//
// Without quotes, because a name holding nothing of the kind comes out byte
// for byte as it always did, and every report line of every run that never
// met such a name stays what scripts and people already read. The escape is
// not ambiguous inside a file name: a backslash is refused in one on every
// system (engine/filename.go). In a Windows path it reads as a separator
// followed by a letter and a number, which a person does not mistake for one.
//
// Never for what a program reads. The manifest and every --json report carry
// the name exactly, because a program compares it byte for byte.
func Shown(s string) string {
if !HoldsUnseen(s) && utf8.ValidString(s) {
return s
}
return shown(s, false)
}

// ShownText is Shown for a whole message rather than one name: the line
// breaks and tabs it is laid out with stay as they are, and everything else
// nobody can see is escaped.
//
// For the places a message is turned into words for a person - the command
// line's describeError and the window's refusals - because an error wrapped
// from the operating system repeats the path it failed on in its own words,
// after this tool's sentence has already shown it. Measured on 2026-09-25,
// from a review: "cannot create the output directory" showed the folder
// escaped and the "mkdir" part after it showed it raw.
func ShownText(s string) string {
return shown(s, true)
}

func shown(s string, layout bool) string {
var b strings.Builder
for i := 0; i < len(s); {
r, size := utf8.DecodeRuneInString(s[i:])
switch {
case r == utf8.RuneError && size == 1:
q := strconv.Quote(s[i : i+1])
b.WriteString(q[1 : len(q)-1])
case layout && (r == '\n' || r == '\t'):
b.WriteRune(r)
case !strconv.IsPrint(r):
q := strconv.QuoteRune(r)
b.WriteString(q[1 : len(q)-1])
default:
b.WriteString(s[i : i+size])
}
i += size
}
return b.String()
}

// ShownEach is Shown for every name of a list, for the lines that name a few
// files one after another.
func ShownEach(names []string) []string {
out := make([]string, len(names))
for i, name := range names {
out[i] = Shown(name)
}
return out
}

// HoldsUnseen reports whether s holds a character a person reading it cannot
// see: a character that changes the direction of the text around it, one of
// no width, a byte order mark, a separator that breaks a line without being a
// line break, a space that is not the space bar's, a tag character, and every
// other one Go does not count as printable.
//
// It exists because file names are exactly where such characters are put on
// purpose. A name with a right to left override shows its extension in the
// wrong place, one with a zero width space prints as a name it is not, and
// both are test cases this tool writes (docs/NAMES-PRESET-2026-09-24.md). The
// file keeps its name. What a person reads about it has to show the character
// rather than let it act (O241), and a recipe has to carry it in a form that
// can be read and edited (O244).
//
// The class is strconv.IsPrint turned around, and that is a choice: it is the
// class %q escapes, and the refusals of this tool have quoted names with %q
// all along. One rule means one name looks the same in a refusal, in a report
// and in a recipe. A combining mark is printable and stays as it is. The
// space is printable, every other space is not.
func HoldsUnseen(s string) bool {
for _, r := range s {
if !strconv.IsPrint(r) {
return true
}
}
return false
}
6 changes: 3 additions & 3 deletions internal/engine/engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -491,7 +491,7 @@ func Run(ctx context.Context, files []PlannedFile, opt Options) (*Result, error)
}

if err := os.MkdirAll(opt.OutDir, 0o755); err != nil {
return res, fmt.Errorf("cannot create the output directory %s: %w", opt.OutDir, err)
return res, fmt.Errorf("cannot create the output directory %s: %w", core.Shown(opt.OutDir), err)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}

// The directory is taken before the manifest name is, and the two are not
Expand All @@ -511,7 +511,7 @@ func Run(ctx context.Context, files []PlannedFile, opt Options) (*Result, error)
if errors.Is(err, fs.ErrExist) {
return res, &RunInProgressError{Path: lockPath, Dir: opt.OutDir}
}
return res, fmt.Errorf("cannot start a run in %s: %w", opt.OutDir, err)
return res, fmt.Errorf("cannot start a run in %s: %w", core.Shown(opt.OutDir), err)
}
// Given back however this run ends, including one stopped part way: the
// signal cancels the context, Run returns, and this runs. What it cannot
Expand All @@ -538,7 +538,7 @@ func Run(ctx context.Context, files []PlannedFile, opt Options) (*Result, error)
if errors.Is(err, fs.ErrExist) {
return res, &CollisionError{Path: manifestPath, Manifest: true}
}
return res, fmt.Errorf("cannot start a run in %s: %w", opt.OutDir, err)
return res, fmt.Errorf("cannot start a run in %s: %w", core.Shown(opt.OutDir), err)
}

// Past this point the run owns the name and may write. Started says so, and
Expand Down
Loading
Loading