New algorithm arranges pictures artistically

Should we add graphic designers and art curators to the list of jobs that will be replaced by future computers?
February 21, 2014

New graphic algorithm arranges images by visual characteristics such as size and color (credit: Bernhard Reinert/MPI)

In an Oxford study last year on jobs being replaced by computerization, the authors recommended that workers “will have to acquire creative and social skills.” The movie Her suggests some “social” skills may soon not be needed, and a new image-design algorithm suggests some “creative” skills won’t be either.

The algorithm, developed by Max Planck Institute for Informatics researchers and running on a computer’s GPU (for speed), interactively arranges pictures by certain visual characteristics such as size and color to create an overall harmonious image, replacing hours of doing it by hand.

Starting from a common layout (left), the user’s objective is inferred from placement of three primitives (push pins), leading to a layout organized vertically by size (middle), and after a different placement, additionally by brightness horizontally (right) (credit: Bernhard Reinert et al./ACM Transactions on Graphics)

“The main feature of our program is that it is able to capture the requests of the user,” explains Tobias Ritschel, head of the research group. “We infer the desired layout of all primitives from interactive placement of a small subset of example primitives.” Translation: the user just selects a layout, clicks on three or more pictures, and drags them to certain points; the program then quickly arranges the picture correspondingly.

The method even works for semantic features, such as mortality rates and population size (see 4:16 in the SIGGRAPH Asia 2013 video below).

Ritschel says this method provides novel possibilities for photographers, photo editors, museum curators, and website designers. In development: 3D image arrangements. Oops, add 3D-printing designer skills to the list.

Abstract of ACM Transactions on Graphics (TOG) — Proceedings of ACM SIGGRAPH Asia 2013 paper

We propose an approach to “pack” a set of two-dimensional graphical primitives into a spatial layout that follows artistic goals. We formalize this process as projecting from a high-dimensional feature space into a 2D layout. Our system does not expose the control of this projection to the user in form of sliders or similar interfaces. Instead, we infer the desired layout of all primitives from interactive placement of a small subset of example primitives. To produce a pleasant distribution of primitives with spatial extend, we propose a novel generalization of Centroidal Voronoi Tesselation which equalizes the distances between boundaries of nearby primitives. Compared to previous primitive distribution approaches our GPU implementation achieves both better fidelity and asymptotically higher speed. A user study evaluates the system’s usability.