Markovito Vision Internship — INAOE

Computer-vision work developed during my research internship at INAOE (Instituto Nacional de Astrofísica, Óptica y Electrónica, Puebla, Mexico) for Markovito, INAOE's RoboCup@Home service robot, built on the Robotnik RB-1 mobile-manipulator platform.

This is my personal public copy of the contributions I authored during the internship (Aug–Nov 2024). The Markovito team's main repository is private. Institutional datasets, trained weights, and code owned by INAOE are not redistributed here — only my own scripts and documentation.

---

What's inside

1. Shelf-object detection pipeline (dataset → training → live validation)

An end-to-end pipeline to build and exploit a custom object-detection dataset for shelf items:

• Video-to-frame conversion (convert_video.py) to generate per-object image folders.
• SAM-assisted segmentation with an interactive GUI (GUI_try.py) for selecting and delimiting objects and auto-saving masks and labels per class.
• Data augmentation (rotations, brightness/contrast, scale) generating augmented images with consistent bounding boxes, consolidated into a YOLO-compatible .yaml dataset config.
• YOLOv8 fine-tuning (yolo_train.py) from pretrained weights.
• Live validation (run_yolo.py) with a webcam under variable lighting and cluttered backgrounds.

In total I processed 1,500+ RGB images across several object classes. The end-to-end pipeline was validated on one class (apple); extending the trained model to the full class set was blocked by tooling constraints at the time, and is the natural next step.

2. Person detection and tracking in ROS Noetic

ROS nodes for the robot's person-following behavior:

• camara_node.py — RGB-D frame acquisition published to a ROS topic.
• clothing_detector_node.py — person/clothing detection with a Modanet-pretrained YOLOv3 model (ailia), ROI extraction, and HSV hue histograms as per-person descriptors.
• visualization.py — database creation/loading and persistent-ID assignment across frames.

System-level validation on the robot was limited by an RGB-D camera visualization issue at the time; the nodes and the identification logic are documented in the internship reports.

---

How to run

# Detection pipeline
conda create -n markovito_vision python=3.8
conda activate markovito_vision
pip install ultralytics opencv-python

python convert_video.py      # video → frames
python GUI_try.py            # SAM segmentation + labelling
python yolo_train.py         # fine-tune YOLOv8 (expects data.yaml)
python run_yolo.py           # live webcam validation

# Tracking module (requires ROS Noetic)
pip install ailia rospy rospkg pyyaml
roscore
python clothing_detector_node.py
python camara_node.py
python visualization.py

---

Author

Alan Baladier Nicolás Beltrán Durán B.Eng. Mechatronics, Universidad de las Américas Puebla License: MIT

Contact

For questions or comments about the code:

• 📧 [email protected]
• 📧 [email protected]

You may also open issues or suggestions directly in this repository.

---