During the iPhone game project I have written about before we had to implement a highly dynamic 3D scene in great detail here at itemis. From a third-person perspective, the player looks at a semi-transparent ball rolling through a gully towards a waterfall. To implement this with the limited resources of the mobile device we came up with an adaption of the rear projection technique as known from film productions.
The game screen as seen by the player internally consists of different layers. The top-most layer contains 2D graphics and displays information such as speed or remaining time. Behind this layer (or plate) the dynamic 3D game scene is being rendered. This layer contains the actual game elements the player controls. It has been built with OpenGL and displays the player’s character inside a rolling ball.
Now, in addition to this quite common setup we implemented a third layer in the back that plays a full motion video of a pre-rendered game scene. Even though this technique had been used for games such as Rebel Assault on the PC between the late 80′s and the early 90′s I am not aware of any iPhone game that integrates the game layer as tight as we did with matting and alpha compositing.
In no case we wanted to deliver a separate, affixed game layer above the movie. Instead, we tried to create the impression of embedded game elements (a) as an integrated part of the pre-rendered movie. With the help of an animated alpha mask (holdout matte) that corresponds to the wave movements of the river (b) we were able to let the water appear above the ball (c). And by using a matte with a true alpha channel we were even able to let submerged parts of the scene shine through the water surface as you can see on (d).
NIVEA FOR MEN WaterfallRace is available on the App Store for free and I will write another post including a screencast once the scheduled update for known bugs is out.
We were able to make this compositing feature work on the 1st generation iPod Touch and iPhone with the SDK 3.0. Please let me know if you are interested in the technical details on how to configure the EAGLView and MPMoviePlayerController or how to sync with the movie on the targeted iPhone OS 3.1. Also, please comment on this post if there’s another development aspect of this project you want to learn more about.