Analyzing billions of events in real-time

• 
• Tweet

INTRODUCING DRUID: REAL-TIME ANALYTICS AT A BILLION ROWS PER SECOND

WHY GENERIC MACHINE LEARNING FAILS

new article by Lev Manovich: "There is Only Software"

• 
• Tweet

Depending on the software I am using, the “properties” of a media object can change dramatically. Exactly the same file with the same contents can take on a varirty of identities depending on the software being used.

What does this finding means in relation to the persisting primacy of the term “digital” in understanding new media? Let me answer this as clear and direct as I can. There is no such thing as “digital media.” There is only software – as applied to media data (or “content”.)

Download full article (1500 words):
There is Only Software (PDF).


----------------------------
Reference:
Lev Manovich. "There is Only Software." 2011 version. Available at www.softwarestudies.com

Web 2.0 Summit 2011: The Data Frame

• 
• Tweet

From Summit announcement:

"This year, data has taken center stage in the networked economy. We live in a world clothed in data, and as we interact with it, we create more—data is not only the Web's core resource, it is at once both renewable and infinite. No longer tethered to the PC, each of us bathes in a continuous stream of data, in real time, nearly everywhere we go.

In the decade since search redefined how we consume information, we have learned to make the world a game and the game our world, to ask and answer "what’s happening," "what’s on your mind," and "where are you?" Each purchase, search, status update, and check-in layers our world with data. Billions of times each day, we pattern a world collectively created by Twitter, Zynga, Facebook, Tencent, Foursquare, Google, Tumblr, Baidu, and thousands of other services. The Database of Intentions is scaling to nearly incomprehensible size and power."

Structure Big Data 2011 conference

• 
• Tweet

Check out the conference program for trends in big data business

new article by Lev Manovich: "Trending: The Promises and the Challenges of Big Social Data"

• 
• Tweet

Trending: The Promises and the Challenges of Big Social Data (PDF).

In this article I address some of the theoretical and practical issues raised by emerging “big data”-driven social science and humanities. My observations are based on my own experience over last three years with big data projects carried out in my lab at UCSD and Calit2 (softwarestudies.com). The issues which we will discuss include the differences between “deep data” about a few and “surface data” about the many; getting access to transactional data; and the new “data analysis divide” between data experts and the rest of us.

----------------------------
Reference:
Lev Manovich. "Trending: The Promises and the Challenges of Big Social Data."
Debates in the Digital Humanities, edited by Matthew K. Gold. The University of Minnesota Press, forthcoming 2012. PDF available at http://lab.softwarestudies.com/2011/04/new-article-by-lev-manovich-trending.html.

interactive media visualization by Jonathan Harris

• 
• Tweet

In 2007 well-known digital artist and designer Jonathan Harris created a project which features a number of interactive interfaces to a large photo collection. These interfaces are excellent examples of what I call "media visualizations" (see my article What is Visualization? 2010).

Interactive interface:
http://thewhalehunt.org/whalehunt.html

Explanations of interface modalities:
http://thewhalehunt.org/interface.html

new software studies book: "Programmed Visions: Software and Memory" by Wendy Chun

• 
• Tweet

book description (from Amazon):

"New media thrives on cycles of obsolescence and renewal: from celebrations of cyber-everything to Y2K, from the dot-com bust to the next big things--mobile mobs, Web 3.0, cloud computing. In Programmed Visions, Wendy Hui Kyong Chun argues that these cycles result in part from the ways in which new media encapsulates a logic of programmability. New media proliferates "programmed visions," which seek to shape and predict--even embody--a future based on past data. These programmed visions have also made computers, based on metaphor, metaphors for metaphor itself, for a general logic of substitutability.

Chun approaches the concept of programmability through the surprising materialization of software as a "thing" in its own right, tracing the hardening of programming into software and of memory into storage. She argues that the clarity offered by software as metaphor should make us pause, because software also engenders a profound sense of ignorance: who knows what lurks behind our smiling interfaces, behind the objects we click and manipulate? The less we know, the more we are shown. This paradox, Chun argues, does not diminish new media’s power, but rather grounds computing’s appeal. Its combination of what can be seen and not seen, known (knowable) and not known--its separation of interface from algorithm and software from hardware--makes it a powerful metaphor for everything we believe is invisible yet generates visible, logical effects, from genetics to the invisible hand of the market, from ideology to culture."


Hardcover: 224 pages
Publisher: The MIT Press (April 30, 2011)
Language: English
ISBN-10: 0262015420
ISBN-13: 978-0262015424

new book: "The Secret War Between Downloading and Uploading: Tales of the Computer as Culture Machine" by Peter Lunenfeld

• 
• Tweet

book description (from Amazon):

"The computer, writes Peter Lunenfeld, is the twenty-first century's culture machine. It is a dream device, serving as the mode of production, the means of distribution, and the site of reception. We haven't quite achieved the flying cars and robot butlers of futurist fantasies, but we do have a machine that can function as a typewriter and a printing press, a paintbrush and a gallery, a piano and a radio, the mail as well as the mail carrier. But, warns Lunenfeld, we should temper our celebration with caution; we are engaged in a secret war between downloading and uploading--between passive consumption and active creation--and the outcome will shape our collective futures.

In The Secret War Between Downloading and Uploading, Lunenfeld makes his case for using digital technologies to shift us from a consumption to a production model. He describes television as "the high fructose corn syrup of the imagination" and worries that it can cause "cultural diabetes"; prescribes mindful downloading, meaningful uploading, and "info-triage" as cures; and offers tips for crafting "bespoke futures" in what he terms the era of "Web n.0" (interconnectivity to the nth power). He also offers a stand-alone genealogy of digital visionaries, distilling a history of the culture machine that runs from the Patriarchs (Vannevar Bush's WWII generation) to the Hustlers (Bill Gates and Steve Jobs) to the Searchers (Larry Page and Sergey Brin of Google fame). After half a century of television-conditioned consumption/downloading, Lunenfeld tells us, we now find ourselves with a vast new infrastructure for uploading. We simply need to find the will to make the best of it."

Hardcover: 144 pages
Publisher: The MIT Press (April 29, 2011)
Language: English
ISBN-10: 0262015471
ISBN-13: 978-0262015479

new book: "The Philosophy of Software: Code and Mediation in the Digital Age" by David Berry

• 
• Tweet

Book description (from Amazon):

This book is a critical introduction to code and software that develops an understanding of its social and philosophical implications in the digital age. Written specifically for people interested in the subject from a non-technical background, the book provides a lively and interesting analysis of these new media forms.

Hardcover: 200 pages
Publisher: Palgrave Macmillan (May 10, 2011)
Language: English
ISBN-10: 9780230244184
ISBN-13: 978-0230244184
ASIN: 0230244181

Ubicomp lecture series at UCSD / Spring 2011

• 
• Tweet

lecture schedule

Content and communication strategies in 4535 covers of Time magazine

• 
• Tweet

Some time ago we created a set of visualizations of all 4535 covers of Time magazine published between 1923 and 2009. Here are the visualizations which which patterns in content and communication strategies of Time.

-------------------------
Flickr user laurence.wheway have added deep zoom interface to our visualization of 4535 covers of Time magazine.

4535 Time covers: deep zoom

Same visualization on Flickr (without deep zoom):



-------------------------
The following set includes visualization which use our manual annotations of all covers of Time magazine (4553 covers, 1923-2009). We use manual annotations to tag semantic features which are hard to analyze automatically using digital image analysis. Of course this takes time so we would not do this on large data sets such as our 1 million Manga pages data set - but for smaller sets such as Time covers, this is a useful approach.

Project credits:
Data collection + additional software + data managment: Jeremy Douglass
Visualization software and design: Lev Manovich

Horizontal visualizations:
X-axis: publication data (left to right).
Y-axis: brightness mean (for b&w covers) or saturation mean (for color covers).

Square visualizations:
X-axis: brightness mean.
Y axis: brightness standard deviation.

-------------------------
Here are graphs which show historical patterns in some of the metadata we added:



Here are some of these patterns shown using our "media visualization" method (including original images in the graphs):

How many women and people of color appeared on the covers of Time magazine?


All Time covers for 1923-1989: 3480 total
Covers showing people: 2583


All Time covers for 1923-1989: 3480 total
Covers showing people: 2583
Covers showing women: 260

.
All Time covers for 1923-1989: 3480 total
Covers showing people: 2583
Covers showing people of color: 306

-------------------------
In the next set, we used color frames around the covers which have particular metadata tags:


All Time covers for 1923-1989: 3480 total
Covers showing people: 2583
Covers showing women (framed in blue): 260


All Time covers for 1923-1989: 3480 total
Covers showing people: 2583
Covers showing people of color (framed in red): 306

-------------------------
The next set shows historical changes in content (compositions illustrating concepts vs. [portraits of particular individuals) and media used (photos vs drawings).

Compositions vs. portraits - we placed red frames around the covers which we call "compositions" - they represent an idea, an object, an event etc, as opposed to portraits of particular individuals which dominate first decades of publication.



Before Photoshop -
The covers with blue frames around them use photos (as opposed to drawings or paintings):