Mapi is more than just a question and answer show, Mapi is an augmented reality 3D character created by Iralta Virtual Production with multi-camera virtual production technology in collaboration with Mediacrest and RTVE.
Mapi is broadcast in the access prime time of the First channel of RTVE and is the adaptation of the revolutionary and successful program of Japanese public television NHK Chiko's Challenge. Adapting the weekly Japanese program Chiko Challenge and its workflow to the needs of a daily format has been a technological and artistic challenge.
The challenge was to create a 3D character that could do the same as a flesh and blood actor, a half-human, half-3D hybrid character that would interact with the rest of the actors, contestants and real objects.
The 3D model of the character is positioned in virtual space thanks to a Mosys tracking system installed in the cameras and in the actress, and this allows us to know the place that the character occupies on the physical and virtual set.
Facial capture with Faceware transfers the gestures of the actress to the 3D model in real time using "machine learning" technology, the facial capture software learns in each program how to improve Mapi's expressiveness.
In this virtual AR production we integrate thanks to Unreal, the real-time rendering engine, the video signals of 4 cameras, the set lighting and the screen videos to improve the integration of the 3D virtual character in the real scene and thus increase realism.
The integration of the 3D character on the real image has been developed so that the production team can have the Mapi 3D image available in real time in 4 cameras and thus be able to "click" the signal that the director and the producer of the program want. .
The workflow designed for this virtual production project has allowed us to produce more than 5,400 minutes of AR content in real time without the need for offline post-production of the 3D content.
This new way of producing 3D and VFX animation and animation content completely changes the pattern established since years when post-production was something that was done after recording. Thanks to virtual production, post-production and animation take place in real time and parallel to filming, so the rest of the actors can interact live with the elements of the set and the 3D characters.