Linking Multiple Text-to-Image Models Increase Image Accuracy

AI programs trained to generate images using a dataset of text-to-image pairs

MIT researchers develop a new method that uses multiple models to create more complex images with better understanding. The internet had a collective feel-good moment with the introduction of DALL-E, an artificial intelligence-based image generator. Systems like DALL-E, DALL-E 2, and Mid journey are AI programs trained to generate images from text descriptions using a dataset of text-to-image pairs. These AI-based image generators use natural language to bring your imagination to life and beyond.

Making DALL.E even more creative:

DALL.E 2, is inspired by Salvador Dalí and WALL-E uses a 'diffusion model.' The diffusion model helps encode the entire text into one description to generate an image. The lovable robot WALL-E uses natural language to produce whatever mysterious and beautiful image your heart desires. DALL.E 2 could effectively model object positions and relational descriptions, which is challenging for existing image generation models.

DALL-E 2 uses something called a diffusion model, where it tries to encode the entire text into one description to generate an image.  DALL-E 2 is good at generating natural images but has difficulty understanding object relations sometimes. DALL.E 2 might make a green truck and a red house to swap these colors around. The magical models behind image generation tools work by suggesting interactive refinement steps to get the desired image or output.

To generate more complex images with better understanding, scientists from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) structured the typical model from a different angle. They structured the typical model from a different angle to generate more complex images with better understanding. The team's approach can handle this type of binding of attributes with objects, and especially when there are multiple sets of things, it can handle each object more accurately.