Terrain Transformation - Virtual Background Transformation!

Cycle GAN is a technique for training unsupervised image translation models via the GAN architecture using unpaired collections of images from two different domains

Applications:

Autonomous cars
Movie background editing.

The goal is to develop a deep learning model that will provide images of different weather conditions like snowy/rainy, to train autonomous cars at all the terrains. This model can also be used to change the terrains in movie clips

Synopsis

Some places have clear sky and greenery most of the time, and rarely experience snow or rain. We expect the self- driving cars to operate consistently whether rain or snow.

This AI framework lets machines imagine: How a place looks l ike when i t’ s raining or snowing, by looking at the image taken during normal ( clear / sunny) weather and vice-versa.

Challenges

To retain resolution of input data. High resolution input data needs to be fed to the model for training.
Finding Open- Source High resolution images of different terrains and pre-processing them, to obtain training data.
To make model Architectural changes to obtain High Resolution Images.

Feature

Input video of driving sequence of particular terrain is passed to a Model, which transforms it into sequence of different terrain. Eg: Driving sequence of normal ( sunny) terrain is transformed into Snowy Terrain

Cycle-Gan

Cycle GAN is a technique for training unsupervised image translation models via the GAN architecture using unpaired collections of images from two different domains. Cycle GAN uses a cycle consistency loss to enable training without the need for paired data.

Cycle GAN Applications: Season translation, object transfiguration, style transfer, and generating photos from paintings.

The Cycle GAN is an extension of the GAN architecture that involves the simultaneous training of two generator models and two discriminator models.

One generator takes images from the first domain as input and outputs images for the second domain, and the other generator takes images from the second domain as input and generates images for the f irst domain. Discriminator models are then used to determine how plausible the generated images are and update the generator models accordingly.

How it works

Preparation of the training Dataset:

Input video is converted to frames and resized to a suitable dimension.
Passing frame by frame to Cycle GAN model : Cycle GAN Converts image from one domain into another. ( eg: Sunny to Snowy.
Cycle GAN works without labelled datasets, but manages to produce results of similar quality. This is major advantage as i t frees up t ime, dedicated to sort training data.
Finally, transformed Images are combined to form video.