DALL·E 2 is the new version of DALL·E, a generative language model that takes sentences and creates corresponding original images.

At 3.5B parameters, DALL·E 2 is a large model but not nearly as large as GPT-3 and, interestingly, smaller than its predecessor (12B). Despite its size, DALL·E 2 generates 4x better resolution images than DALL·E and it’s preferred by human judges +70% of the time both in caption matching and photorealism.

As they did with DALL·E, OpenAI didn’t release DALL·E 2 (you can always join the never-ending waitlist). However, they open-sourced CLIP which, although only indirectly related to DALL·E, forms the basis of DALL·E 2.

(CLIP is also the basis of the apps and notebooks people who can’t access DALL·E 2 are using.) Still, OpenAI’s CEO, Sam Altman, said they’ll eventually release DALL·E models through their API — for now, only a few selected have access to it (they’re opening the model to 1000 people each week).

How DALL·E 2 works
I’ll explain DALL·E 2 more intuitively soon, but I want you to form now a general idea of how it works without resorting to too much simplification. These are the four key high-level concepts you have to remember:

CLIP: Model that takes image-caption pairs and creates “mental” representations in the form of vectors, called text/image embeddings (figure 1, top).

Prior model: Takes a caption/CLIP text embedding and generates CLIP image embeddings.
Decoder Diffusion model (unCLIP): Takes a CLIP image embedding and generates images.
DALL·E 2: Combination of prior + diffusion decoder (unCLIP) models.
DALL·E 2 is a particular instance of a two-part model (figure 1, bottom) made of a prior and a decoder. By concatenating both models we can go from a sentence to an image. That’s how we interact with DALL·E 2. We input a sentence into the “black box” and it outputs a well-defined image.


For better explanation you can look at the Instagram slides to understand the big words.

So what it means is that this AI can translate your words with those terms mentioned as part of what's embedded in its systems to generate an Image.

What's the comparison with Midjourney.

I didn't hop on this on time but I had the opportunity to use Midjourney and might pay for an additional opportunity layer this week.

But first let's enjoy some art by DALL-E and maybe in some other thread, I'll try manipulating some of these artworks.

Some artworks from @openaidalle

More...

What's Midjourney
MidJourney is an AI-powered system that creates images from user prompts. On their website, they describe themselves as: “An independent research lab. Exploring new mediums of thought. Expanding the imaginative powers of the human species.”

While DALL·E is designed to generate anything you can imagine – including the mundane or ugly -Midjourney has a bias towards creating painterly, aesthetically-pleasing images by default. Given the choice, MidJourney prefers to create images with complimentary colours, artistic use of light and shadow, sharp details, and composition with satisfying symmetry or perspective.

In the words of its founder, “we just want it to be easy to use – and we want the pictures to look good.”
Unlike DALL·E 2 or DALL·E Mini, Midjourney doesn’t work as a web app. Nor is there any coding required.

Instead, you use it in a popular chat app called Discord, by ‘talking to’ a bot. (Discord, for those unfamiliar, is very similar to Slack, and just like Slack, it works both in your browser and in a standalone app, on both desktop and mobile. )

Artwork by Midjourney AI

Harry Potter as a black African boy with dreadlocks and blue eyes.

First was the initial four artwork
Second is the final upscaled

The third is the Photoshop retouched version