Mapi is more than just a Q&A program, Mapi is an augmented reality 3D character created by Iralta Virtual Production with multi-camera virtual production technology in collaboration with Mediacrest and RTVE.
Mapi is broadcast in the access prime time of the First RTVE chain and is the adaptation of the revolutionary and successful program of the Japanese public television NHK Chiko's Challenge. Adapting the Japanese weekly format program Chiko Challenge and its workflow to the needs of a daily format has been a technological and artistic challenge.
The challenge was to create a 3D character that could do the same as a flesh-and-blood actor, a hybrid half-human, half-3D character that interacted with the rest of the actors, contestants and real objects.
The 3D model of the character is positioned in the virtual space thanks to a Mosys tracking system installed in the cameras and in the actress and this allows us to know the place that the character occupies in the physical and virtual dish.
The facial capture with Faceware transfers the gestures of the actress to the 3D model in real time using "machine learning" technology, the facial capture software learns in each program how to improve the expressiveness of Mapi.
In this virtual AR production we integrate thanks to Unreal, the real-time rendering engine, the video signals of 4 cameras, the lighting of the set and the videos of the screens to improve the integration of the 3D virtual character in the real scenario and thus increase realism.
The integration of the 3D character on the real image has been developed so that the production team can have in real time the image of Mapi 3D in 4 cameras and thus be able to "puncture" the signal that the director and the director of the program want.
The workflow designed for this virtual production project has allowed us to produce more than 5,400 minutes of AR content in real time without the need for offline post-production of 3D content.
This new way of producing 3D and VFX animation content and animation totally changes the pattern established since years where post-production was something that was done after recording. Thanks to virtual production, post-production and animation occur in real time and in parallel to the shooting so that the rest of the actors can interact live with the elements of the set and the 3D characters.