ReadWriteWeb

The travel geeks at Thomson have created a data visualization you can dance to. They tracked the top-level dance genres over the past century, and expressed the data as an animated map that moves from parent genre to descendant, proliferating over time.

The mapmakers used data from the books Bass Culture, Last Night a DJ Saved My Life and The All Music Guide to Electronica, as well as Wikipedia. They marked the birth of each genre in five year periods. As well researched as it might be, the exercise wasn't without controversy, however.

A freely accessible index of 5 billion web pages, their page rank, their link graphs and other metadata, hosted on Amazon EC2, was announced today by the Common Crawl Foundation. "It is crucial [in] our information-based society that Web crawl data be open and accessible to anyone who desires to utilize it," writes Foundation director Lisa Green on the organization's blog.

The Foundation is an organization dedicated to leveraging the falling costs of crawling and storage for the benefit of "individuals, academic groups, small start-ups, big companies, governments and nonprofits." It's lead by Gilad Elbaz, the forefather of Google AdSense and the CEO of data platform startup Factual. Joining Elbaz on the Foundation board is internet public domain champion Carl Malamud and semantic web serial entrepreneur Nova Spivack. Director Lisa Green came to the Foundation by way of Creative Commons.

Mobile analytics firm Flurry is expanding its product offering with an announcement today of AppCircle Re-Engagement, a tool for iOS developers to attempt to energize users who have downloaded apps but may not be using them. Flurry has correctly identified the problem of decreasing app engagement over periods of time and now enters a crowded and growing space of tools for developers to increase app participation.

Flurry is extending its analytics program to target specific demographics. The company couches this ability as a new offering for developers. In reality, it is not. There are a variety of companies that provide analytics and ways to market from actionable data.

Google has updated its Government Requests tool with data from the first half of this year. For the first time, the report discloses the number of users or accounts specified, not just the number of requests. Google also made the raw data behind government requests available to the public.

Google launched its interactive transparency report last year. U.S. requests for google user data have spiked in the past six months, and Google complies 93% of the time. Google's transparency efforts have displeased some governments, but its compliance with requests have upset some civilians, too. In this increasingly weird new world, Google can only err on the side of more transparency while pushing for better laws.

You make data. A lot of it. From Web browsing to link sharing to photos published online, from phone bills to medical records to online banking - almost all of us produce an incredible amount of electronic data that slips right through our fingers - often into the gaping maw of a corporate world without our best interests in mind.

What if you could easily capture all that data yourself, though? What if you could use it like fuel for apps built for you to view, sort and take action based on all that data? What if you could offer selective access to outside parties to that data? That's the vision behind the Locker Project, an open source personal data platform, and Singly, its corporate partner for hosted installs of the data lockers. Singly 1.0 launches to developers today at the Web 2.0 Summit (live at 2:40 PST). It's got financial backing from the leaders of WordPress, TechStars and multiple VC firms and a knock-out team of famous developers building it. What does it look like? Check out the first screen shots below.

According to new data from comScore, 6.8% of Web traffic in the U.S. comes from "non-computer" devices such as smartphones and tablets. This is an increase from 6.2% in the previous quarter.

Phones account for the majority of non-computer traffic. Mobile devices drive 4.4% of total digital traffic, tablets contribute 1.9%, and other non-computer devices send 0.5% of traffic.

Matt LeMay, platform manager at Bit.ly, says that social data can tell us who we are – and who we want to be. Speaking at the Monktoberfest today in Portland, Maine, talked about some of the insights that Bit.ly gets from looking at sharing and click data for Bit.ly links. LeMay has learned that what people share isn't what they click on – and if you want followers, be a cat, not a chicken.

As one of the seven companies that participated in the Alpha release of this new Insights data, we've been very excited about the opportunities this new data presents for pages and their fans. Let's take a closer look at why.

While most discussion around social analytics slips and slides across the valley floor of a wide crevasse between practitioners and business leaders, there has been one metric everyone agreed was important: fans. Fans have been an obvious place to start because your number and everyone else's have been public for years. Also, there's a natural, implied value proposition in the metric because it's about affinity. Surely having a large number of people with expressed affinity for your brand is a good thing (especially if it's more than your competitor has).

With quadrillions of dollars on the line, banks and financial institutions pay close attention to the emerging exaflood of available data about their customers and the world around them. Here at the O'Reilly Strata conference on big data, the panel on big data in the banking world was fascinating. It's likely an indication of the way the rest of the world is likely to move in the near future - at least if you believe the predictions of the people on the panel.

Huge opaque markets are about to become transparent because of new regulations and that means a whole lot of new data available for analysis. Scalable processing of that data will require outsourcing, giving birth to new industries. Millions of people will need to be trained to deal with all this. Below, my notes from this fascinating panel discussion.

"Big data enables new ways to create value, it's going to change the basis of competition," Michael Chui of the McKinsey Global Institute said this morning to kick off O'Reilly's big data conference, the Strata Summit. The next two days are all about the rise of information that has to be dealt with on scale, big data, and its consequences. "It will change the way companies, sectors and economies compete," says Chui.

McKinsey published an exhaustive 150 page report on big data this Spring, which argued that data will soon become an economic input as important as labor and capital. It's not just about pure economics, though. As Edd Dumbill, chair of Strata, put it today, our relationship with big data needs to serve humans - not turn humans into the servants of machines and information overload. "We know that big data can help us, it may be the case that big data has to help us." Below, a live video stream of the next two days' proceedings addressing this mega-opportunity and trend.