November 2008
Friday, November 21, 2008
Lev Manovich is interviewed about Cultural Analytics for BBC World Service "Digital Planet" show
On November 21, 2008, Lev Manovich was interviewed for BBC World Service "Digital Planet" show (to be aired in a few weeks). The subject of the interview was Cultural Analytics.
Wednesday, November 19, 2008
Visualizing Cultural Patterns in N Art Magazine
The Software Studies project "Visualizing Cultural Patterns" appeared in a video clip by N Art Magazine as part of a feature on researchers at the Center for Research in Computing and the Arts, Calit2. The clip was aired by ABC on Sunday 10/26/08 at 4PM PST.
http://www.youtube.com/v/uMMiJw0aAuU
The spot was recorded by Jeremy Douglass, with the HIPerWall being run by So Yamaoka (not pictured).
For more on related Software Studies research, see:
ArtHistory.viz | Mining 200,000+ Images of Art
Cultural Analytics Research Environment
Cultural Pattern Recognition, or Seeing Through Images | Automatic Analysis of Visual Media and User Interactions
http://www.youtube.com/v/uMMiJw0aAuU
The spot was recorded by Jeremy Douglass, with the HIPerWall being run by So Yamaoka (not pictured).
For more on related Software Studies research, see:
ArtHistory.viz | Mining 200,000+ Images of Art
Cultural Analytics Research Environment
Cultural Pattern Recognition, or Seeing Through Images | Automatic Analysis of Visual Media and User Interactions
Tuesday, November 18, 2008
Software Studies ganha bolsa do NEH Humanities High Performance Computing (HHPC)
O Grupo de Software Studies da Universidade da Califórnia em San Diego (UCSD) recebeu a bolsa de pesquisa do NEH Humanities High Performance Computing (HHPC).
Abaixo uma breve descrição da bolsa HHPC do site do National Endowment for the Humanities:
"A Computação de Alta Performance para as Humanidades faz referência ao uso de máquinas de alta performance em projetos nas áreas de ciências sociais e humanidades. Hoje em dia somente um pequeno número de pesquisadores está utilizando e tirando proveito da computação de alta performance. Mas, assim como as ciências tem começado a se utilizar de forma mais intensa ao longo dos anos do potencial da computação de alta performance, as humanidades estão, da mesma forma, também começando esse processo. Pesquisadores das humanidades quase sempre lidam com grandes quantidades de dados sem estrutura. Esses dados podem estar em forma de jornais históricos, livros, dados de eleições, fragmentos arqueológicos, conteúdos de áudio ou vídeo. A bolsa para a Computação de alta performance nas humanidades dá a oportunidade de organizar, analisar e melhor compreender, além de visualizar esses dados."
A bolsa prevê o uso dos supercomputadores do Departamento de Energia no National Energy Research Scientific Computing Center (NERSC), no Lawrence Berkeley National Laboratory, assim como treino e assessoria.
Essa bolsa permitirá que o nosso projeto de pesquisa Cultural Analytics alcance um outro patamar em termos de análise. O grupo planeja analisar a visualizar padrões com grandes conjuntos de dados: milhares de filmes e animações, machinima, gravações de jogos de videogames, imagens de arte, design gráfico, entre outros.
A bolsa prevê o uso dos supercomputadores do Departamento de Energia no National Energy Research Scientific Computing Center (NERSC), no Lawrence Berkeley National Laboratory, assim como treino e assessoria.
Essa bolsa permitirá que o nosso projeto de pesquisa Cultural Analytics alcance um outro patamar em termos de análise. O grupo planeja analisar a visualizar padrões com grandes conjuntos de dados: milhares de filmes e animações, machinima, gravações de jogos de videogames, imagens de arte, design gráfico, entre outros.
Software Studies Initiative receives NEH Humanities High Performance Computing awards
We have received NEH Humanities High Performance Computing (HHPC) grant.
This is the short description of HHPC from NEH site:
"Humanities High-Performance Computing (HHPC) refers to the use of high-performance machines for humanities and social science projects. Currently, only a small number of humanities scholars are taking advantage of high-performance computing. But just as the sciences have, over time, begun to tap the enormous potential of HPC, the humanities are beginning to as well. Humanities scholars often deal with large sets of unstructured data. This might take the form of historical newspapers, books, election data, archaeological fragments, audio or video contents, or a host of others. HHPC offers the humanist opportunities to sort through, mine, and better understand and visualize this data."
The grant provide computer time on Department of Energy supercomputers at the National Energy Research Scientific Computing Center (NERSC) at the Lawrence Berkeley National Laboratory, as well as training and support.
This award will allow us to take our Cultural Analytics projects to the next level. We plan to analyze and visualize patterns in large visual data sets: thousands of feature films and animations, machinema, video recordings of gameplay, art images, graphic design, etc.
This is the short description of HHPC from NEH site:
"Humanities High-Performance Computing (HHPC) refers to the use of high-performance machines for humanities and social science projects. Currently, only a small number of humanities scholars are taking advantage of high-performance computing. But just as the sciences have, over time, begun to tap the enormous potential of HPC, the humanities are beginning to as well. Humanities scholars often deal with large sets of unstructured data. This might take the form of historical newspapers, books, election data, archaeological fragments, audio or video contents, or a host of others. HHPC offers the humanist opportunities to sort through, mine, and better understand and visualize this data."
The grant provide computer time on Department of Energy supercomputers at the National Energy Research Scientific Computing Center (NERSC) at the Lawrence Berkeley National Laboratory, as well as training and support.
This award will allow us to take our Cultural Analytics projects to the next level. We plan to analyze and visualize patterns in large visual data sets: thousands of feature films and animations, machinema, video recordings of gameplay, art images, graphic design, etc.
Sunday, November 9, 2008
ArtHistory.viz | visualizing modernism
ArtHistory.viz explores the applications of cultural analytics to art history.
Lev Manovich / November 2008.
Testing cultural analytics approach using a sample data set of 35 canonical art history images - from Courbet (1849) to Malevich (1914).
Sample image set:
Method:
1) automatically measure images to extract various statistics and features.
2) make graphs: points represents images, Y = time, X = values on one or more dimensions.
In this sample graph, we plot reverse skew value for each painting vs. the year the painting was made.
Art history and other cultural disciplines conceptualize history of culture in terms of small sets of categories: movements, national schools, historical periods, -isms, etc.
Graphs: superimposing standard art historical categories on the data.
But why force contiuous cultural developments and dynamics into small sets of categorical boxes? Using numbers to represent cultural atifacts allows us to visualize cultural developments as continuos curves.
Graph: the movement from 19th century realism to early 20th century modernism.
We can compare the patterns in different cultural data sets both visually and quantitatively.
Graph: comparing the changes in paintings before 1900 vs. the paintings after 1900 using linear trendlines.
We may find the differences between cultural sets which at first sight appear to be identical.
We may find the similarities between the sets which are thought to be very different.
Graph: comparing the change in median values of "realist" paintings vs. "modernist" paintings.
Images can be analyzed on hundreds of different visual dimensions.
Graph: the movement from 19th century realism to early 20th century modernism - using "shape count" value. (Procedure: automatically count the number of shapes in each image disregarding very small shapes.)
It is important to keep in mind that depending on the dimensions we chose to analyze, the resulting pictures of cultural patterns will be also different.
Graph: comparison of 6 different summaries of image statistics.
We can use standard statistical techniques to discover cultural patterns - or to confirm the patterns we may see if we visualize the data.
Graph: correlation between the changes in binary histogram length and median values.
Visualizing cultural data can reveal the relative statistical dispersion of different cultural data sets in relation to each other.
Graph: comparing dispersion in two data sets (post-impressionism and 1910-1915 abstraction).
Quantitative automatic analysis can potentially allow us to develop quantitative measures for "cultural innovation," "cultural openess" (openness to adopting elements from other cultures), "the speed of cultural change," and other cultural dimensions. The following is a very early experiment (not to be taken too seriously.
Which artists were "ahead of their time"?
Graph: a diagonal line shows the general tendency in the movement from realism to modernism from 1849 to 1914. The paintings which are above the trend line are "ahead" of the general trend.
Multiple graphs can be combined to study the individual artifacts within the context of larger data sets.
Graph: top: the image set; right: a single image and its analysis ( Pissaro, 1898); left: the graphs which show the data for the whole image set, with the Pissaro's painting indicated in red.
Visual cultural data can be also clustered into new classes - thus allowing us to describe cultural dynamics in more precise way than the use of verbal labels allows. (Practical examples are coming up soon...)
Lev Manovich / November 2008.
Testing cultural analytics approach using a sample data set of 35 canonical art history images - from Courbet (1849) to Malevich (1914).
Sample image set:
Method:
1) automatically measure images to extract various statistics and features.
2) make graphs: points represents images, Y = time, X = values on one or more dimensions.
In this sample graph, we plot reverse skew value for each painting vs. the year the painting was made.
Art history and other cultural disciplines conceptualize history of culture in terms of small sets of categories: movements, national schools, historical periods, -isms, etc.
Graphs: superimposing standard art historical categories on the data.
But why force contiuous cultural developments and dynamics into small sets of categorical boxes? Using numbers to represent cultural atifacts allows us to visualize cultural developments as continuos curves.
Graph: the movement from 19th century realism to early 20th century modernism.
We can compare the patterns in different cultural data sets both visually and quantitatively.
Graph: comparing the changes in paintings before 1900 vs. the paintings after 1900 using linear trendlines.
We may find the differences between cultural sets which at first sight appear to be identical.
We may find the similarities between the sets which are thought to be very different.
Graph: comparing the change in median values of "realist" paintings vs. "modernist" paintings.
Images can be analyzed on hundreds of different visual dimensions.
Graph: the movement from 19th century realism to early 20th century modernism - using "shape count" value. (Procedure: automatically count the number of shapes in each image disregarding very small shapes.)
It is important to keep in mind that depending on the dimensions we chose to analyze, the resulting pictures of cultural patterns will be also different.
Graph: comparison of 6 different summaries of image statistics.
We can use standard statistical techniques to discover cultural patterns - or to confirm the patterns we may see if we visualize the data.
Graph: correlation between the changes in binary histogram length and median values.
Visualizing cultural data can reveal the relative statistical dispersion of different cultural data sets in relation to each other.
Graph: comparing dispersion in two data sets (post-impressionism and 1910-1915 abstraction).
Quantitative automatic analysis can potentially allow us to develop quantitative measures for "cultural innovation," "cultural openess" (openness to adopting elements from other cultures), "the speed of cultural change," and other cultural dimensions. The following is a very early experiment (not to be taken too seriously.
Which artists were "ahead of their time"?
Graph: a diagonal line shows the general tendency in the movement from realism to modernism from 1849 to 1914. The paintings which are above the trend line are "ahead" of the general trend.
Multiple graphs can be combined to study the individual artifacts within the context of larger data sets.
Graph: top: the image set; right: a single image and its analysis ( Pissaro, 1898); left: the graphs which show the data for the whole image set, with the Pissaro's painting indicated in red.
Visual cultural data can be also clustered into new classes - thus allowing us to describe cultural dynamics in more precise way than the use of verbal labels allows. (Practical examples are coming up soon...)
Friday, November 7, 2008
Cultural Pattern Recognition, or Seeing Through Images | Automatic Analysis of Visual Media and User Interactions
The existing work in visualization of media data typically relies on existing metadata (such as Flickr community-contributed tags). In contrast, Cultural Analytics methodology calls for the use of image processing and computer vision techniques to automatically analyze large sets of visual cultural objects to automatically generate descriptions of their form and content. These numerical descriptions can be then graphed and also analyzed statistically.
While digital media authoring programs such as Photoshop and After Effects incorporate certain image processing techniques (blur, sharpen, and edge detecting filters, motion tracking), there are hundreds of features that can be automatically extracted from still and moving images. Some of the algorithms are incorporated in MATLAB; others are available via C libraries such as open CV and openFrameworks; still others are described in computer science publications. While some of the techniques can be used without the knowledge of computer programming and statistics, many others require knowledge of C programming.
Which of the algorithms can be particularly useful for cultural analysis?
Can we create (relatively) easy-to-use tools which will allow non-technical users to perform automatic analysis of visual media?These are the questions we are investigating.
Analysis of video game play:
Color values extracted from 400 seconds of video recording of the game play (lower RGB graphs). In the upper graph, we combine analysis of frame differences with grey scale values. This graph can be used to automatically classify different types of user activity during game play such as world exploration, combat, and menu use.
A selection of still frames from the recording of playing "Fatal Frame 2" being analyzed in the graphs above.
Analysis of art images:.
Visualizing the development of modernism using 35 canonical art history images.
top: the image set.
right: a single image and its analysis ( Pissaro, 1898.)
left: the graphs which show the data for the whole image set, with the Pissaro's painting indicated in red.
More details: ArtHistory.viz
Related projects:
Digital Formalism: The Vienna Vertov Collection
While digital media authoring programs such as Photoshop and After Effects incorporate certain image processing techniques (blur, sharpen, and edge detecting filters, motion tracking), there are hundreds of features that can be automatically extracted from still and moving images. Some of the algorithms are incorporated in MATLAB; others are available via C libraries such as open CV and openFrameworks; still others are described in computer science publications. While some of the techniques can be used without the knowledge of computer programming and statistics, many others require knowledge of C programming.
Which of the algorithms can be particularly useful for cultural analysis?
Can we create (relatively) easy-to-use tools which will allow non-technical users to perform automatic analysis of visual media?These are the questions we are investigating.
Analysis of video game play:
Color values extracted from 400 seconds of video recording of the game play (lower RGB graphs). In the upper graph, we combine analysis of frame differences with grey scale values. This graph can be used to automatically classify different types of user activity during game play such as world exploration, combat, and menu use.
A selection of still frames from the recording of playing "Fatal Frame 2" being analyzed in the graphs above.
Analysis of art images:.
Visualizing the development of modernism using 35 canonical art history images.
top: the image set.
right: a single image and its analysis ( Pissaro, 1898.)
left: the graphs which show the data for the whole image set, with the Pissaro's painting indicated in red.
More details: ArtHistory.viz
Related projects:
Digital Formalism: The Vienna Vertov Collection
Sunday, November 2, 2008
MotionGraphics.viz
Researcher: Lev Manovich
In the 20th century, intellectuals devoted lots of energy to analyzing lens-based narrative visuals (photography and cinema) and modern non-figurative art. Animation, graphic design, typography, information design, and other areas of visual culture were mostly ignored. in fact, if you are to search for books which theoretically analyze graphics, you will find only a single title published in France in the end of 1960s: Jacques Bertin, Semiology of Graphics (English edition, 1983).
In the 1990s, most areas of culture industry switched to software-based production. As a result, graphic design (as well as as other areas of visual culture I listed above) assumed much more central position in contemporary culture. Additionally, visual culture became hybrid. Today, a still design or a moving image sequence now typically combine many previously separate media. Such hybrids are now the norm.
A case in point are contemporary motion graphics (commercials, music videos, film and TV titles, and other short forms). They are as prominent today as film and TV narratives - but they cannot be adequately described using the concepts of film theory. Motion graphics typically combine multiple media and techniques (live action video, 2D and 3D animation, typography, effects, compositing, etc.). Instead of being divided into a number of discrete shots, a work often is a single visual flow which constantly changes over time. (For a more detailed analysis, see the chapter "After Effects, or How Cinema Became Design" in Lev Manovich's book Software Takes Command.)
Cultural Analytics approach can be used to analyze motion graphics (as well as other areas of contemporary visual culture largely ignored by academic theory.) The analysis and visualization of how different image parameters change over time allows us to describe moving images in new ways . We can graph temporal patterns, and compare them across different films.
Below are some the results of our explorations into different ways of visualizing temporal changes in motion graphics.
A comparative matrix: four works + different analytical graphs.
video for “Go” by Common, directed by Convert/MK12/Kanye West, 2005: sampled frames
Mapping visual parameters in video for “Go."
X axis: time (in frames). Y axis: parameter values (all parameters were normalized to 0-255 range.)
Mapping visual parameters in video for “Go": detailed analysis of the first 30 sec.
Temporal analysis of video for "Go" by Common. Grayscale profiles (mapped into a 3D (first 32 seconds of the video only).
Björk's music video: a set of sampled frames.
Temporal analysis of Björk's music video (sampled at 15fps). Each horizontal line corresponds to a separate frame of video (sampled at 4fps). The program draws a horizontal line through the middle of each frame, and copies the colors into a row of a new image.
Temporal analysis of Björk's music video. Each vertical line corresponds to a separate frame of video (sampled at 15fps). The value represented is a median of a frame.
The same data mapped into two values to represent the rhythm of changes between figurative parts (dark brown) and abstract part (light brown).
Legend:
In the 20th century, intellectuals devoted lots of energy to analyzing lens-based narrative visuals (photography and cinema) and modern non-figurative art. Animation, graphic design, typography, information design, and other areas of visual culture were mostly ignored. in fact, if you are to search for books which theoretically analyze graphics, you will find only a single title published in France in the end of 1960s: Jacques Bertin, Semiology of Graphics (English edition, 1983).
In the 1990s, most areas of culture industry switched to software-based production. As a result, graphic design (as well as as other areas of visual culture I listed above) assumed much more central position in contemporary culture. Additionally, visual culture became hybrid. Today, a still design or a moving image sequence now typically combine many previously separate media. Such hybrids are now the norm.
A case in point are contemporary motion graphics (commercials, music videos, film and TV titles, and other short forms). They are as prominent today as film and TV narratives - but they cannot be adequately described using the concepts of film theory. Motion graphics typically combine multiple media and techniques (live action video, 2D and 3D animation, typography, effects, compositing, etc.). Instead of being divided into a number of discrete shots, a work often is a single visual flow which constantly changes over time. (For a more detailed analysis, see the chapter "After Effects, or How Cinema Became Design" in Lev Manovich's book Software Takes Command.)
Cultural Analytics approach can be used to analyze motion graphics (as well as other areas of contemporary visual culture largely ignored by academic theory.) The analysis and visualization of how different image parameters change over time allows us to describe moving images in new ways . We can graph temporal patterns, and compare them across different films.
Below are some the results of our explorations into different ways of visualizing temporal changes in motion graphics.
A comparative matrix: four works + different analytical graphs.
video for “Go” by Common, directed by Convert/MK12/Kanye West, 2005: sampled frames
Mapping visual parameters in video for “Go."
X axis: time (in frames). Y axis: parameter values (all parameters were normalized to 0-255 range.)
Mapping visual parameters in video for “Go": detailed analysis of the first 30 sec.
Temporal analysis of video for "Go" by Common. Grayscale profiles (mapped into a 3D (first 32 seconds of the video only).
Björk's music video: a set of sampled frames.
Temporal analysis of Björk's music video (sampled at 15fps). Each horizontal line corresponds to a separate frame of video (sampled at 4fps). The program draws a horizontal line through the middle of each frame, and copies the colors into a row of a new image.
Temporal analysis of Björk's music video. Each vertical line corresponds to a separate frame of video (sampled at 15fps). The value represented is a median of a frame.
The same data mapped into two values to represent the rhythm of changes between figurative parts (dark brown) and abstract part (light brown).
Legend:
Saturday, November 1, 2008
Software Art at CFAV, Recife (Brazil)
Seminar about Software Art at the Centro de Formação em Artes Visuais do Recife (CFAV) in Recife, Brazil
Seminar: Software art
Lecturers:
Cicero Silva (Software Studies coordinator, Brazil, São Paulo) + Amy Alexander (Founder Runme.org) - webconference
Lecturers:
Cicero Silva (Software Studies coordinator, Brazil, São Paulo) + Amy Alexander (Founder Runme.org) - webconference
Jarbas Jacome (Pernambuco)
Clyton Galamba (Pernambuco)
When and where:
Nov 19, 2008
7pm
Auditório Porto Digital
Address: Rua do Apolo, 181, Bairro do Recife (map)
CFAV: Pátio de São Pedro Casa, 11, Santo Antonio, Recife, PE
Phones: 55 81 3232 2848 / 55 81 3232 2858, from 9am to 5pm
Phones: 55 81 3232 2848 / 55 81 3232 2858, from 9am to 5pm
Palestra sobre Software Arte no CFAV (Centro de Formação em Artes Visuais) em Recife
Palestrantes:
Cicero Silva (Coordenador do grupo de Software Studies no Brasil) + Amy Alexander (Fundadora do site de Software Arte Runme.org) - webconferência
Jarbas Jácome (Pernambuco)
Clyton Galamba (Pernambuco)
Onde e quando
Dia 19 de novembro de 2008
19h
Local: Auditório Porto Digital
Rua do Apolo, 181, Bairro do Recife (mapa)
Centro de Formação em Artes Visuais – CFAV
Endereço: Pátio de São Pedro Casa 11 Santo Antonio, Recife – PE
Informações: (81) 3232 2848 / 3232 2858, das 9h às 17h
Informações: (81) 3232 2848 / 3232 2858, das 9h às 17h
Subscribe to:
Posts (Atom)