ULSAN — Researchers have developed artificial intelligence that reconstructs the movements of multiple people in three dimensions without interruption, even in footage where the camera angle changes frequently, as in films and sports broadcasts.
The Ulsan National Institute of Science and Technology (UNIST) said on the 15th that a team led by Professor Baek Seung-ryul of its Graduate School of Artificial Intelligence has developed Multi-THuMBS, an AI technology that consistently tracks and reconstructs the three-dimensional body shapes and movements of multiple people in edited video with frequent scene cuts.
Three-dimensional human reconstruction analyzes a person's body shape, posture and position in video to generate a solid model and motion data. Applying AI makes such reconstruction possible from a single ordinary video, without multi-camera rigs.
Existing methods, however, relied on single continuous shots with uninterrupted camera movement, making them difficult to apply to films, broadcasts and sports coverage that stitch together footage from multiple angles. When a cut abruptly changes the viewpoint and camera position, the systems would mistake the profile of a person previously seen from the front for someone else, or misread a change in framing as actual movement and reconstruct the person in the wrong place.
The team solved the problem by building a "shared 3D space" that binds the background and camera positions of the images immediately before and after a cut into a single coordinate system. Human models reconstructed separately from each shot are then aligned step by step within that space according to their position, body orientation and camera angle.
The researchers then combined the 3D distance between people placed in the same space with information on clothing and posture to match the same individual one-to-one across the cut. If the distance between two figures exceeded one meter, the system blocked the match, preventing errors in which a newly appearing or departing person is confused with someone already in the scene. Finally, the team smoothed the trajectories of joints over time, eliminating jumps in joint position and feet sliding across the floor.
In performance tests applying scene-cut editing to multi-camera motion data (EgoHumans), Multi-THuMBS reduced joint position error by about 31% and motion acceleration error by about 44% compared with GVHMR, the previous best-performing method. Identity-switch errors, in which the same person is misidentified as someone else, fell about 62%, and camera position estimation error was halved.
The technology also showed high accuracy on actual film data (AVA) and footage from the sitcoms "Friends" and "The Big Bang Theory." The rate at which joint positions matched before and after a cut reached 90.7%, far ahead of the 62.7% recorded by the comparison method. Motion jitter fell to about one-fifth of previous levels, and errors in which feet slide along the floor dropped by more than half.
Using the technology, large volumes of 3D motion data can be built from existing film, broadcast and sports footage alone, without dedicated specialty recording equipment. It is expected to find wide use in precision sports analysis, such as offside rulings in soccer, in virtual and augmented reality content, in digital human production and in securing training data for motion-generating AI.
On Jeong-wan, a researcher and the paper's first author, said the method currently applies to video that shows the same physical space even after a cut. "We plan to develop technology in follow-up research that can continuously track people even in scenes that move to an entirely different location," the researcher said.
"Existing 3D reconstruction technology tended to lose track of people the moment a scene changed," Baek said. "This technology can reconstruct the movements of multiple people in 3D without interruption even in edited video, so it will contribute to a range of digital content industries."
The research was joined by UNIST researchers Muhammad Salman Ali, Muneeb A. Khan, Park Sun-woo and Moon In-woong, along with Professor Jang Hyung-jin of the University of Birmingham in Britain and researchers from CJ's AI and DT division. The work was supported by the National Research Foundation of Korea (NRF) under the Ministry of Science and ICT, the Institute of Information & Communications Technology Planning & Evaluation (IITP) and CJ, and was presented in September at the European Conference on Computer Vision (ECCV 2026) in Malmo, Sweden, one of the world's three largest computer vision conferences.