Content Aware Photo Editor

Hi there folks 👋

This project is my initiative on researching content aware image resizing. The repository contains a few neat things:

• Implementation of Seam Carving algorithm: slow one working for O(w*h*max(dw, dh)), and a fast one working for O(w*h). The fast algorithm is also applicable for runtime usage as it adds no additional complexity over image traversing. You can find the code at lib-seamcarving module;
• Wrappers around Glide and Picasso. The wrappers can be found at: lib-seacarving-glide and lib-seamcarving-picasso modules;
• Sample app. Demonstrates how to use image libraries wrappers in realtime;
• Editor. This is the main module of the whole repository. Demonstrates how to apply seam carving at image editing applications.

Introduction

Seam Carving

The main idea of seam carving is to provide a lossless way of retargeting images by removing the content from "not enough interesting" areas. You can see the example of the process below.

Original image	Seams to remove	Result

As you can see the image has been trimmed by removing content (highlighted in red) in-between important objects (balloons).

Energy function

To build seams according to the limitations we use an heuristic function called an "energy" function. This function helps us to determine the important parts of the image.

For most of the operations the Sobel operator is more than enough to find right areas as it calculates a gradient vector's magnitude approximation. This approximation plays a big role in edge detection. The implementation works really good in our case as well. See the examples below:

Original image	Energy	Notes
		An example with clearly distinguishable background
		Clouds make the background less distinguishable than on example above, but still keeps the object aside from background
		Hardly distinguishable background

Compexities concerns

Criterion	Complexity
Runtime	O(w*h)
Memory	O(w*h)

So as we said before no "extra" complexity over image traversal. This is a really good complexity for software processing algorithms.

Libraries

We picked the most famous libraries for Android and adapted our algorithm to them. These libraries are Glide and Picasso.

Glide

We provide SeamCarvingTransformation to use Seam Carving algorithm with Glide.

You can easily pull the library and integrate it into your codebase. The usage is pretty straightforward:

Glide.with(itemView.context)
    .load(imageRes)
    .apply(RequestOptions.bitmapTransform(SeamCarvingTransformation(sampling = 2)))
    .into(imageView)

Picasso

Picasso structure is quite similar to the one in Glide, therefore we also provide SeamCarvingTransformation to adapt our algorithm to the library.

The example is given below:

Picasso.get()
    .load(imageRes)
    .transform(SeamCarvingTransformation(sampling = 2))
    .into(imageView)

Examples

We also prepared a small demo library (you can find it in the sample app module). The main idea of the library is to show two things:

1. Prove that we can use the algorithm in runtime;
2. Compare the algorithm with one of existing approaches.

Center Crop	Seams Carving

P.S.: More examples available at the sample app!

Editor

The star of this repo is the editor (the app module). The editor supports manual handling of the seam carving with some advanced tooling onboard. You can use brushes 🖌 to keep/remove objects from the image. Moreover, the editor also provides a mechanism for making accurate adjustments to the algorithm.

Remove objects

The editor allows your to keep the same image size but remove some unwanted objects from it. Highlight the area that you wanna remove and then click apply. That is really easy!

Original image	Editor	Result

Keep objects

Moreover, the editor introduce a way to seek for guidance from the user and provide a way to them to highlight the objects that they want to keep if algorithm fails. For example, it can be really useful for objects that are hardly distinguishable from the background. The process is demonstrated below:

Original image	Editor	Result

Questions

Still have some questions? Contact me by opening an issue.

License

MIT License

Copyright (c) 2022 Alexander Dadukin

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.