Ways to Use Adobe Firefly

Here's a little Photoshop trick I used in this example: After selecting each area to fix, I pressed Q (for Quick mask), then Ctrl-L (Levels), and dragged the lower slider down from 255 to a much lower value, such as 50 or 100. This faded out the selection, making it a transparent mask instead of an opaque one. Then I pressed Q again to get out of quick mask mode, typed the prompt I wanted into the contextual taskbar, and pressed Generate. Fading out the mask is important if you want Generative Fill to respect the original contents of the area, instead of drawing something entirely new to completely replace what was there.

Photoshop's little window for generating images seems like a work in progress. It doesn't have any history of your previously used prompts and doesn't have sliders to control the image generation. If it had sliders for things like how much it replaced or respected the original image, I wouldn't need to keep undoing the generation, going through the process of adjusting the transparency of a mask and then re-typing the prompt each time I tried a new generative fill with a different amount of change. The window does have an option to load a separate file as a reference image, although again there's no slider for the weight or amount of influence the reference image will have.

Besides Generative Fill, the other amazing new function that Photoshop uses Firefly for is Generative Expand. This lets you un-crop an image, seamlessly extending it beyond its original borders. Even though other models can do this basic technique (called outpainting), the results from Generative Expand are some of the best I've seen. Even with high-resolution images, Adobe Firefly comes up with impressively seamless results, matching the contents, style, perspective, and lighting from the original shot as it invents new areas of the image. 

If you use DALL-E 3 to generate images, you've probably noticed that it doesn't support many different aspect ratios. But that's what Photoshop is for. If you take a square image from DALL-E into Photoshop, it's quick and easy to make a wide-screen version: 

If you use Stable Diffusion locally on your computer, you're in luck. Not only can Photoshop be used for final retouching of images, but there are also plug-ins that better integrate Photoshop with Stable Diffusion. You can do AI functions that Firefly doesn't support, such as real-time painting in Photoshop, while having Stable Diffusion transform the look or style of the image according to your prompt or settings. Photoshop is one of only two paint programs that have this level of AI integration possible via plug-ins, with the other one being the open-source paint program Krita.

Images generated by Firefly need to pass through a content filter that tries to detect any inappropriate content in what it is generating, or in the surrounding image that it is extending or modifying. This means that you may not be able to un-crop an image if there's a subject in the image that Firefly's content filter doesn't approve of.

In some cases, you may need to create a new layer, scribble over the subject that is causing the problem to cover it up, and then try again with your Generative Expand or Generative Fill. You can delete the cover-up layer afterwards, but even when workarounds like this are possible, it is strange that you can find yourself in an adversarial relationship with your own paint program.

Note that content filtering only applies to the functions that use Firefly. Other AI-powered tools in Photoshop such as the extremely useful Remove tool, or the intelligent Select Subject function, can be used with any image. 

While Firefly is a champion of inpainting and outpainting, it can also create new images from scratch, based only on a prompt. Firefly can be used to create images on firefly.adobe.com, and also from within other Adobe apps, including Adobe InDesign and Adobe Express.

I tried a version labelled "Firefly Image 3 (preview)," and used it to generate images based on some of the same prompts that I also tested in DALL-E 3 and Midjourney version 6. Here's an example prompt I tried:

• Architectural beach house with natural material architecture, with a roof terrace to enjoy the late-afternoon sun and waterfront view. Near the beach, a bike path with bicyclists, and idyllic gardens. 

In my first few tests, Firefly didn't give me a "bike path" or any "bicyclists," but only scattered some unmanned bicycles around the image, sometimes placing them on the roof of the beach house. Some of the images came out looking more like a photo collage than a single, coherent photograph. 

In testing the system, I found that sometimes when it seems to ignore words or phrases from a prompt, it helps to edit the prompt down to a much shorter phrase, including only a few of the words that you care about. Remember that you have Photoshop available afterwards, so there's no reason to be too specific about small stuff like a person's eye color, if you can use Generative Fill to fix that later.