This is my college capstone project (September 2024 - June 2025). The goal was to build an interactive, real-time golf simulation game that only requires a standard webcam, eliminating the need for expensive hardware sensors.
It uses MediaPipe to track the player's swing posture, calculates the physical forces, and simulates a realistic ball trajectory in Unity using a custom-built 3D physics engine.
- Video Demo: Watch the real-time simulation in action
- Speech Report: View the full PDF documentation
- Post： View the full PDF documentation
- Analyzed Report: View the full PDF documentation
This project integrates three major systems: User Interface, MediaPipe Pose, and Unity Engine.
- The screen shows the default perspective after the game is launched, where players can select their clubs and prepare to hit the ball.
- The interface design clearly displays hitting information (distance, strokes, etc.) and is integrated with the course terrain built in Unity.
The system uses a camera to capture the user's swing video, and the MediaPipe Pose module performs image preprocessing and feature extraction to locate 33 human body keypoints (covering the head, shoulders, elbows, wrists, hips, knees, and ankles). This coordinate data drives two core functions:
- Initial Ball Velocity Estimation: The system performs real-time tracking of the hand and ball positions. At the moment of the hit, it extracts the position and time difference of the last two frames before the hit to calculate the initial velocity of the ball generated by the swing action, serving as the basis for subsequent physics simulation parameters.
- Skeleton Animation Drive: The full-body keypoint information captured by MediaPipe Pose is transmitted via TCP to the virtual character skeleton in Unity[cite: 1]. It maps to the system's various joint points, driving the character to instantly present a swing action synchronized with the user in the 3D scene.
- The system uses TCP as the connection method between Unity and MediaPipe.
- When MediaPipe detects the human keypoints, it directly transmits the coordinate data to Unity via a TCP connection and instantly updates the skeleton animation.
- 1-Wood (Driver): Used primarily for tee-offs, featuring the largest volume and longest shafts, generally providing the farthest hitting distance (300-400 yards).
- Irons & Wedges: Irons have heavier clubheads and shorter shafts than woods, with smaller clubheads and greater loft angles. Wedges are a type of iron but feature the highest loft angles.
- Putters: Feature a slender shaft and a flat-bottomed clubhead.
- Loft Angle: The angle between the clubface and the shaft. An increase in loft angle results in a higher trajectory but a decrease in distance.
All ball types are standardized to a mass of 45.93 grams and a diameter of 1.68 inches to comply with international standards. Performance adjustments are based on:
- Layers: Usually 2 to 5 layers; more layers represent finer performance adjustments like higher control and spin effects.
- Friction: Affects the rolling distance after landing.
- Spin Resistance: Lower spin resistance allows the ball to spin more, enhancing lift and increasing flight distance.
Below are screenshots from different stages of development, showing the integration of pose tracking and the physics simulation environment:
