Contrastive fashions like CLIP have been proven to study sturdy representations of photographs that seize each semantics and magnificence. To leverage these representations for picture technology, we suggest a two-stage mannequin: a previous that generates a CLIP picture embedding given a textual content caption, and a decoder that generates a picture conditioned on the picture embedding. We present that explicitly producing picture representations improves picture variety with minimal loss in photorealism and caption similarity. Our decoders conditioned on picture representations also can produce variations of a picture that protect each its semantics and magnificence, whereas various the non-essential particulars absent from the picture illustration. Furthermore, the joint embedding area of CLIP permits language-guided picture manipulations in a zero-shot vogue. We use diffusion fashions for the decoder and experiment with each autoregressive and diffusion fashions for the prior, discovering that the latter are computationally extra environment friendly and produce higher-quality samples.
