Ideation
Design
Project management
Once Upon Innovation
From Concept to Creation:
RealFill’s Generative AI Experience
Step into the innovative realm of generative AI with Realfill, a capability developed by Google. In this project I introduce how I created a high-fidelity prototype that showcases its possibilities.
The project was done as part of my freelance work
Project management
Use case
Research
Once Upon Innovation
From Concept to Creation:
RealFill’s Generative AI Experience
Step into the innovative realm of generative AI with Realfill, a capability developed by Google. In this project I introduce how I created a high-fidelity prototype that showcases its possibilities.
The project was done as part of my freelance work
THE TALE BEGINS
I was honored to be part of the ideation team for ‘Realfill,’ Google’s cutting-edge generative AI capability. Imagine my excitement when I was asked to take it further—exploring its potential beyond the initial concept and envisioning how it could serve users as a standalone app.
THE TALE BEGINS
I was honored to be part of the ideation team for ‘Realfill,’ Google’s cutting-edge generative AI capability. Imagine my excitement when I was asked to take it further—exploring its potential beyond the initial concept and envisioning how it could serve users as a standalone app.
Imagine the impossible
Google’s RealFill
During a visit to Google, I had the opportunity to meet with researchers from the innovation department. I proposed the idea of expanding the boundaries of an image, going beyond the constraints of the camera’s frame, while keeping it authentic to the original scene. What was once a concept born from imagination is now a reality—thanks to RealFill.
Imagine the impossible
Google’s RealFill
During a visit to Google, I had the opportunity to meet with researchers from the innovation department. I proposed the idea of expanding the boundaries of an image, going beyond the constraints of the camera’s frame, while keeping it authentic to the original scene. What was once a concept born from imagination is now a reality—thanks to RealFill.
Acknowledgements from Google’s article (Link to article)
The goals
The goals
1
Defining scope
Establishing the boundaries of RealFill within the platform meant identifying the starting and ending points of the dedicated flow, ensuring a clear and seamless experience.
1
Defining scope
Establishing the boundaries of RealFill within the platform meant identifying the starting and ending points of the dedicated flow, ensuring a clear and seamless experience.
2
Balancing Ease and Authenticity
Realfill demands more user input than baseline generative tools, offering significantly more authentic results. The goal was to design an experience that simplifies the process while clearly demonstrating the value of their extra effort.
2
Balancing Ease and Authenticity
Realfill demands more user input than baseline generative tools, offering significantly more authentic results. The goal was to design an experience that simplifies the process while clearly demonstrating the value of their extra effort.
3
Accommodating User Levels
Realfill's technology serves a wide range of users, from beginners to experts. I aimed to create a flexible system that adapts to varying skill levels while maximizing capabilities.
3
Accommodating User Levels
Realfill's technology serves a wide range of users, from beginners to experts. I aimed to create a flexible system that adapts to varying skill levels while maximizing capabilities.
4
Cost worthy
Generating high-quality images with Realfill comes at a higher cost. To ensure efficiency, I focused on minimizing unnecessary use without compromising the experience.
4
Cost worthy
Generating high-quality images with Realfill comes at a higher cost. To ensure efficiency, I focused on minimizing unnecessary use without compromising the experience.
Quest
The hypothesis
By designing a dedicated flow within the image editing application, Realfill's new capabilities can be implemented to meet the diverse needs of users while keeping development costs manageable.
Quest
The hypothesis
By designing a dedicated flow within the image editing application, Realfill's new capabilities can be implemented to meet the diverse needs of users while keeping development costs manageable.
Journey of discovery
With the goals and scope defined, I delved deeper—exploring the technology, researching user needs, and benchmarking other photo editing applications.
Journey of discovery
With the goals and scope defined, I delved deeper—exploring the technology, researching user needs, and benchmarking other photo editing applications.
Defining what’s possible
Development capabilities
RealFill is a generative outpainting model that completes missing regions of an image with content faithful to the original scene. RealFill utilizes a few reference images, which can vary significantly in viewpoint, lighting, and style, to personalize its output. While this process involves greater time and cost compared to baseline generative models, it delivers unmatched flexibility and authentic, high-quality results without the need for precise alignment.
Defining what’s possible
Development capabilities
RealFill is a generative outpainting model that completes missing regions of an image with content faithful to the original scene. RealFill utilizes a few reference images, which can vary significantly in viewpoint, lighting, and style, to personalize its output. While this process involves greater time and cost compared to baseline generative models, it delivers unmatched flexibility and authentic, high-quality results without the need for precise alignment.
RealFill vs. Basic generative model results
Example 01
Target image
Reference images
RealFill result
Baseline result 01
Baseline result 02
Example 02
Target image
Reference images
RealFill result
Baseline result 01
Baseline result 02
User prospective:
How can people use this?
User research uncovered two key use cases that would benefit users:
Struggling to capture the full frame, such as when taking selfies.
Wanting to adjust their photo to fit a specific frame size (e.g., for social media or printing).
User research uncovered two key use cases that would benefit users:
Struggling to capture the full frame, such as when taking selfies.
Wanting to adjust their photo to fit a specific frame size (e.g., for social media or printing).
Based on these needs, I defined personas ranging from basic to expert users. I then adapted the solution to ensure a seamless experience for each user type, recognizing that both groups sought editing capabilities.
Based on these needs, I defined personas ranging from basic to expert users. I then adapted the solution to ensure a seamless experience for each user type, recognizing that both groups sought editing capabilities.
Basic users
Want quick, easy results with minimal effort, without concern for small details.
Basic users
Want quick, easy results with minimal effort, without concern for small details.
Advanced users
Seek control over the image, aiming to fine-tune and perfect it according to their vision.
Advanced users
Seek control over the image, aiming to fine-tune and perfect it according to their vision.
The Benchmark
inspirations
Most photo editing platforms with generative AI extensions are limited in their capabilities. They typically allow for symmetrical image extensions, either vertically or horizontally, and are confined to a fixed frame size. Unlike RealFill, they lack authentic completion abilities. As a result, the output tends to be less realistic and fails to capture the same level of authenticity.
The Benchmark
inspirations
Most photo editing platforms with generative AI extensions are limited in their capabilities. They typically allow for symmetrical image extensions, either vertically or horizontally, and are confined to a fixed frame size. Unlike RealFill, they lack authentic completion abilities. As a result, the output tends to be less realistic and fails to capture the same level of authenticity.
The enchanted outcome
Understanding the technology's capabilities and limitations, along with the users' needs and the competition's shortcomings, guided me in shaping the app's experience.
The enchanted outcome
Understanding the technology's capabilities and limitations, along with the users' needs and the competition's shortcomings, guided me in shaping the app's experience.
Flexible by Design
Creating a flexible system that adapts to user needs was key. With just a target image and desired frame size, the system intelligently selects compatible reference images from the user’s library (if available) and generates a result with minimal effort, accommodating both basic users who rely on system suggestions and advanced users who fine-tune the output.
Don’t worry — if no reference images are available, the system will use hallucination generative models to create an image and let users add references later for even more authentic results.
Flexible by Design
Creating a flexible system that adapts to user needs was key. With just a target image and desired frame size, the system intelligently selects compatible reference images from the user’s library (if available) and generates a result with minimal effort, accommodating both basic users who rely on system suggestions and advanced users who fine-tune the output.
Don’t worry — if no reference images are available, the system will use hallucination generative models to create an image and let users add references later for even more authentic results.
Smart Previews
Providing users with a blurred preview before generating a high-quality image offers a glimpse of what’s possible, sparking curiosity and encouraging engagement. This approach balances user intrigue with cost efficiency by minimizing unnecessary high-quality generations.
Smart Previews
Providing users with a blurred preview before generating a high-quality image offers a glimpse of what’s possible, sparking curiosity and encouraging engagement. This approach balances user intrigue with cost efficiency by minimizing unnecessary high-quality generations.
RealFill/SceneFill
Final prototype
The gallery below shows the final detailed design
RealFill/SceneFill
Final prototype
The gallery below shows the final detailed design
Open the app, snap a new photo, or select one from your library to begin editing.
Main screen - Snap a photo
Main screen - Select from device
Navigate to the “expand” option to unlock the ability to push beyond existing boundaries and create the frame you envision.
Editing photo options
Easily switch frame sizes to explore different looks, with results adjusting seamlessly.
Frame options - Stories
Frame options - Landscape
Frame options - Circle
With just a click, a high-quality, authentic image is generated in seconds.
RealFill high-quality result
Customize further by resizing or repositioning the target image and managing reference images, as much or as little as you like.
Adjusting target image size & placement as wanted
.
Reference photos settings
Not quite right? Re-generate with ease. When using RealFill, adjustments will stay true to the original scene, while hallucinative generative models (used without references) offer broader, less authentic results.
Baseline result 01
Baseline result 02
Baseline result 03
Once satisfied, continue enhancing your photo with other editing tools or save it to your collection.
Main screen - Saved collections
A dream come true
I am honored to have been part of the ideation process for this groundbreaking technology at a company like Google and grateful for the opportunity to design an experience that brought it to life.
This project highlights the key considerations I focused on while introducing the first version of RealFill’s capabilities. Moving forward, user testing will be essential to refine the experience and explore the addition of more advanced features.
A dream come true
I am honored to have been part of the ideation process for this groundbreaking technology at a company like Google and grateful for the opportunity to design an experience that brought it to life.
This project highlights the key considerations I focused on while introducing the first version of RealFill’s capabilities. Moving forward, user testing will be essential to refine the experience and explore the addition of more advanced features.
Let’s talk
mayag86@gmail.com