Builds mc-player-320-v1, the built-in model behind Blockwork's detection
blocks: YOLOv8n fine-tuned to find Minecraft players, 320 px input.
For automation where it's allowed (single-player, your own servers, testing). Don't use it against other players on servers that forbid it.
capture.py drives a local Fabric server with the Carpet mod over RCON. For
each sample it spawns fake players, places your spectator player as the
camera, freezes the world with /tick freeze, and takes two F2 screenshots:
with the players, then with them moved away. Pixels that changed are visible
player pixels; projecting each player's known position splits them per
player (label.py). Nothing outside the game window is captured.
- Install Minecraft Java Edition 26.3 with the Fabric loader. The published
model was captured on 26.3 with Carpet
fabric-carpet-26.3+v260915(dataset.yamlrecords the settings); other versions with/tick(1.20.3 and newer) should work but haven't been tried. - Use the
server/folder (onlystart.shand its README are tracked). Get the Fabric server launcher for that version from fabricmc.net, and the matching Carpet jar from Modrinth or github.com/gnembon/fabric-carpet into itsmods/folder. - Start the server once, read the EULA it points to, and set
eula=trueineula.txtif you accept it. - In
server.propertiesset:Put the same password inenable-rcon=true rcon.port=25575 rcon.password=<something> online-mode=trueserver/rcon-password.txtforcapture.py --password-file. Then start the server again and runop <your name>in its console. - Build the client mod in
mods/no-chat-render(mods/no-chat-render/build.sh, needs a JDK 21) and putbuild/blockwork-nochat.jarin the client's Fabricmods/folder. It keeps the "Saved screenshot" chat lines out of the frames. It looks for the 26.3 client jar under~/.local/share/PrismLauncherand the Fabric libraries underserver/libraries; setMC_CLIENT_JARandFABRIC_LIBSif yours are elsewhere. - Join
localhostwith that client. In Options: FOV Normal (70), Clouds Off, Particles Minimal, Entity Shadows Off, View Bobbing Off, FOV Effects 0%. Keep the window size fixed while capturing.
python3 -m venv venv && venv/bin/pip install numpy pillow evdev
venv/bin/python capture.py --password-file server/rcon-password.txt --mc-dir ~/.minecraft --camera <your name> --samples 1500Each sample picks a camera mode (--modes, default spectator, survival and
creative). Survival and creative shots show the HUD and the first-person hand
with a random held item; spectator shots hide the HUD. The camera player
stands on an invisible barrier block while frozen, since /tick freeze
doesn't stop players falling. Very dark frames are skipped. The mode of each
image goes in modes.txt.
Click into the game window within 5 seconds, then leave the keyboard and
mouse alone (it presses F1 and F2 through a virtual keyboard, which needs
access to /dev/uinput). 1500 samples take roughly an hour. It roams far
from spawn, so expect new chunks to generate.
Skins come from the fake players' names: in online mode Carpet uses the
skin of the account with that name, and unknown names get default skins. For
more variety, pass --names names.txt with one name per line.
Check the labels before training:
venv/bin/python preview.py dataset 30 # boxes drawn into dataset/preview/Needs an NVIDIA GPU (CPU works, slowly).
venv/bin/pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
venv/bin/pip install ultralytics onnx onnxslim
venv/bin/python train.py --data datasetIt prints validation scores, exports dataset/mc-player-320-v1.onnx, copies
it into Blockwork's models folder, and prints its SHA-256. Once the file is
hosted (a release asset of this repo), put its URL, hash and size in KNOWN
in Blockwork's blockwork-core/src/detect/models.rs so it can download it.
AGPL-3.0 (see LICENSE). The starting weights are Ultralytics YOLOv8n, which
are AGPL-3.0, so the fine-tuned model is too.
venv/bin/python -m unittest test_label test_capture # capture.py needs numpy and Pillow