Visualizing Social PhotographyLev Manovich. 2013.
Forthcoming in
Aperture magazine #214 (2014).
This summer the Museum of Modern Art in New York asked the
Software Studies Initiative, a program I started in 2007 to explore how visualization could be used as a research tool and perhaps a means to present their photography collection in a novel way. We received access to approximately twenty thousand digitized photographs, which we then combined using our software in a single very high resolution image. This allowed us to view all the images at once, scrolling from those dating from the dawn of the medium to the present, spanning in the meantime countries, genres, techniques, and photographers’ diverse sensibilities. Practically every iconic photograph was included—images I had seen reproduced repeatedly. My ability to easily zoom in on each image and study its details, or zoom out to see it in its totality, was almost a religious experience.
Looking at twenty thousand photographs simultaneously may sound amazing, since even the largest museum gallery includes about a hundred works at the most. And yet, MoMA’s collection, by twentieth-century standards, is meager compared with the massive reservoirs of photographs available on media sharing sites such as Instagram, Flickr, and 500px. (Instagram alone already contains over one billion photographs, while Facebook users upload over ten billion images every month.) The rise of “social photography,” pioneered by Flickr in 2005, has opened fascinating new possibilities for cultural research. The photo-universe created by hundreds of millions of people might be considered a mega-documentary, without a script or director, but this documentary’s scale requires computational tools—databases, search engines, visualization—in order to “watch.”
Mining the constituent parts of this “documentary” can teach us about vernacular photography and habits that govern digital image making. When people photograph one another, do they privilege particular framing styles, ala a professional photographer? Do tourists visiting New York photograph the same subjects; are their choices culturally determined? And when they do photograph the same subject (for example, plants on the
High Line Park in West Manhattan), do they use the same techniques?
To begin answering these questions, we can use computers to analyze the visual attributes and content of millions of photographs and their accompanying descriptions, tags, geographical coordinates, upload dates and times, and then interpret the results. While this research only began a few years ago, there are already a number of interesting projects that point toward future
“computational visual sociology” and
“computational photo criticism.” In 2009, David Crandall and his colleagues from the Computer Science Department at Cornell University published a paper titled
Mapping the World’s Photos based on analysis of approximately thirty-five million Flickr photographs. As part of their research, they created a map consisting of locations where all these images were taken (
"world heat map"). Areas with more photos appear brighter, while those with fewer photographs are dark. Not surprisingly, the United States and Western Europe are brightly illuminated, while the rest of the world remains in the dark, indicating more sporadic coverage. But the map also reveals some unexpected patterns—the shorelines of most continents are very bright, while the interiors of the continents, with the notable exceptions of the States and Western Europe, remain completely dark.
Using their collected photo set, Crandall and his team also determined the most photographed locations in twenty-five metropolitan areas. This led to novel discoveries—New York’s fifth most photographed location was the midtown Apple store; Tate Modern ranked number two in London.
A photo-mapping project
Locals and Tourists created in 2010 by data artist and software developer Eric Fisher addressed a question likely prompted by such information: how many of these images were captured by tourists or local residents, and how does this distinction can reveal different patterns? Fisher’s “Locals and Tourists” plotted the locations of large numbers of Flickr photographs by using color to indicate who took them: blue pictures by locals, red pictures by tourists, yellow pictures might have been made by either group. In total he mapped 136 cities, then shared these maps on Flickr. In his
map of London we see how tourists frequent a few well-known sites, all in central London, while locals cover the whole city, but document less assiduously.
These pioneering projects use metadata to reveal telling patterns in social photography. However, they did not use actual images in their visualizations, a practice first explored, to my knowledge, by artist James Salavon. For projects such as
Every Playboy Centerfold, 1988–1997 and
Homes for Sale, 1999-2002, Salavon composited a number of images to reveal the photographic conventions used to represent particular subjects. His more recent work, Good and Evil '12, 2012, consists of two panels, each showing approximately twenty-five thousand photographs returned by Bing image search for the one-hundred most positive or negative words in English.
Media artists like Salavon demonstrate how visualization may uncover patterns in the content of large image collections. In 2007, I set up a research lab to explore this idea further and to develop open-source visualization tools that can be used by anyone working with images—art historians, film and media scholars, curators. One of our software tools can analyze visual properties (such as contrast, gray scale, texture, dominant colors, line orientations) and some dimensions of content (presence and positions of faces and bodies) of any number of images. Another tool can use results of this analysis to position all images in a single high-resolution visualization sorted by their properties and metadata. We used these tools to visualize a variety of image collections, ranging from
every cover of Time magazine between 1923 and 2009, a total of 4,535 covers, to
one million manga pages.
In our recent project,
Phototrails, Art History PhD student Nadav Hochman, designer/programmer Jay Chow and myself started to explore patterns among photos uploaded to social media sites. In the first stage of the project, we downloaded and analyzed 2.3 million Instagram photographs from thirteen global cities. One of our visualizations shows
53,498 photos shared by people in Tokyo over a few consecutive days. The progression of people’s dominant activities throughout the day—working, having dinner, going out—is reflected in changing colors and relative brightness of images. No day is the same. Some are shorter than others, or the progression between different activities is very gradual, while in others it is sharper. Together, these photos create an “aggregate documentary” of Tokyo—a portrait of the city’s changing temporal patterns aggregated from thousands of documented activities.
But are aggregated documentaries new? Dziga Vertov’s 1929 experimental documentary film
Man with a Movie Camera, which portrays a single day in the life of a Soviet city, might be considered a precursor to the form. The film combines footage shot in three separate Ukrainian cities—Odessa, Khartiv, and Kiev—over a three-year period. Vertov wanted to communicate particular ideas such as construction of a communist society that guided the selection and editing of his footage.
Our visualizations of human habits rendered through Instagram photographs do not reflect a single directorial point of a view. Even so, they are as subjective as more traditional photography. Just as a photographer decides on framing and perspective, we make formal decisions about how to map the images, organizing them by upload dates, or average color, or brightness, and so on. But by visualizing the same set of images in multiple ways (here is an
example which uses a collection of artworks by Mark Rothko, we remind viewers than no single visualization offers an objective interpretation, just as no single, traditional documentary image could ever be considered neutral. Instead, the diversity of the Instagram photographs highlights the variety of complex patterns of life unfolding in cities that can never be fully visually captured in a single visualization, despite our ability use millions of Instagram photographs.
( November 2013.)
