While digital media authoring programs such as Photoshop and After Effects incorporate certain image processing techniques (blur, sharpen, and edge detecting filters, motion tracking), there are hundreds of features that can be automatically extracted from still and moving images. Some of the algorithms are incorporated in MATLAB; others are available via C libraries such as open CV and openFrameworks; still others are described in computer science publications. While some of the techniques can be used without the knowledge of computer programming and statistics, many others require knowledge of C programming.
Which of the algorithms can be particularly useful for cultural analysis?
Can we create (relatively) easy-to-use tools which will allow non-technical users to perform automatic analysis of visual media?These are the questions we are investigating.
Analysis of video game play:
Color values extracted from 400 seconds of video recording of the game play (lower RGB graphs). In the upper graph, we combine analysis of frame differences with grey scale values. This graph can be used to automatically classify different types of user activity during game play such as world exploration, combat, and menu use.
A selection of still frames from the recording of playing "Fatal Frame 2" being analyzed in the graphs above.
Analysis of art images:.
Visualizing the development of modernism using 35 canonical art history images.
top: the image set.
right: a single image and its analysis ( Pissaro, 1898.)
left: the graphs which show the data for the whole image set, with the Pissaro's painting indicated in red.
More details: ArtHistory.viz
Related projects:
Digital Formalism: The Vienna Vertov Collection