In the past, AI researchers have tried to get around this problem by using an approach called data augmentation. Using an imaging algorithm again as an example, in cases where there is not a lot of material to work with, they would try to work around this problem by creating “distorted” copies of what is available. Distorting, in this case, can mean cropping an image, rotating it, or flipping it. The idea here is that the network never sees the exact same image twice.
The problem with this approach is that it would lead to a situation where the GAN would learn to mimic these distortions, instead of creating something new. NVIDIA’s new Adaptive Discriminator Augmentation (ADA) approach still uses data augmentation, but does so adaptively. Instead of distorting the images throughout the training process, it does so selectively and just enough for the GAN to avoid overfitting.
The potential outcome of NVIDIA’s approach is more significant than you might think. Train an AI to write a new one text-based adventure game is easy because there is so much material that the algorithm can work with. The same is not true of many other tasks that researchers might ask of GANs. For example, training an algorithm to spot a rare neurological brain disorder is difficult precisely because of its rarity. However, a GAN trained in NVIDIA’s ADA approach could work around this problem. As a bonus, doctors and researchers could share their findings more easily because they are working from a database of images created by AI, not patients in the real world. NVIDIA to share more information on its new ADA approach at the next NeurIPS Conference, which begins on December 6.