“We’re actually teaching the model how to draw based on real video.” Nvidia’s researchers used a standard machine learning approach to identify different objects in a video scene: cars, trees, buildings, and so forth..The team then used what’s known as a generative adversarial network, or GAN, to train a computer to fill the outline of objects with realistic 3D imagery. The system can then be fed the outline of a scene, showing where different objects are, and it will fill in stunning, slightly shimmering detail..“We wondered what we could do with artificial intelligence to change the rendering process.” Cantazaro says the approach could lower the barrier for game design..Besides rendering whole scenes the approach could be used to add a real person to a video game after feeding on a few minutes of video footage of them in real life..He suggests that the approach could also help render realistic settings for virtual reality, or to provide synthetic training data for autonomous vehicles or robots.. More details