Sunlight downloaded a CSV list of prison populations in all 50 US states and created a map to visualize which states have the largest prisoner populations. Cross reference those numbers with the list of supporters who have singed on to the National Prison Commission Bill and what do you see? Surprise, surprise, not a single Senator from the three states with the biggest prison populations (California, Texas and Florida) are excited to see the the current criminal justice system studied and reformed.

That disinterest could be a matter of economics or satisfaction with the status quo. Or the aversion to studying the matter may represent a desire to avoid embarrassment. The explanation can't be drawn from the data, but it is good data to have. It's not data that Data.gov will help you get your hands on. That catalog of public data remains limited to some of the least controversial data sets online. Three weeks after it's launched, Data.gov keeps adding more and more data sets to its collection - but a search for the word "prison" still brings up zero results.

Take a country with the highest documented incarceration rate and total documented prison population in the world, where under-represented ethnic groups are incarcerated multiple times more than the dominate majority population. Isn't that the kind of situation where you'd like to know whether or not a politician supports just studying the matter? Who does and who doesn't? Easy access to government and public data can help shed important light on current events like this and that's why people get excited about calls to open data up to the public in better and better ways.

Ask Twitter what people are saying on that site about the swine flu and you can get the full story to parse until you're blue in the face. The new Facebook openness is like interoperability between different telephone handset manufacturers but conversation remains closed between individuals. Conversation on Facebook is no more easy to analyze today than it was yesterday; that's the real opportunity here, not just the ability to send and receive Facebook messages through different applications.

The Good News

Facebook users will now be able to send and recieve all kinds of data through other websites and applications. Think Tweetdeck both reading from and writing to Facebook. It is good news that Facebook has decided to implement the work it's been participating in with the Activity Streams working group.

This should enable Facebook messaging through different interfaces to suit different users' needs and desires. It should help cut back on user criticism about UI changes to the site - there will now be different services that serve up the data in different ways - take your pick.

Facebook told us today that every field displayed in the Facebook Newsfeed will be made available through the Activity Streams API, for example, content, username, timestamp, and source if we're talking about something pulled in from offsite. Facebook's, David Swain, told us he expects to "see a lot of innovation on top of the stream. We expect developers to filter it, remix it, and serve it up in new and interesting ways."

The Not So Good News

Unfortunately, the data that developers are able to work with is severely limited. They will simply be able to make a call for a user to Facebook and get back the friends' streams that this particular user has the permission to see. The data can't be cached for later analysis, the company isn't sure yet how far back in the archives access will be permitted (though they say the goal is to expose all the way back), and the Terms of Service will prohibit eyes outside of a user's Facebook friends from seeing the massive amounts of friend-limited data.

In other words, this is permission to build more interfaces for Facebook. That's cool, but that's not really what the world needs - more interfaces for giving Facebook love.

Facebook holds a mind-blowing amount of conversational data. The company is analyzing it extensively and it has an omniscient view of conversations across all the networks of friends and privacy restrictions. It uses that aggregate data analysis to make business decisions and to sell advertisements.

The rest of us are only allowed to give Facebook more data and to get back a sliver per user that will facilitate more user-level participation in amassing more data at Facebook.

The most common analogy used when standards are discussed is the railroads. When railroads agreed to use the same width of rail, then trains could travel from network to network and transcontinental transport became possible. Today's announcement by Facebook is something like that, and that's great. But imagine the analytic value that could be drawn from getting reports about train times and rail conditions across the line. That's the kind of data that allows for more intelligent and interesting decisions to be made. That's explicitly prohibited, in this case, by Facebook.

How is global climate change being dealt with? It's being dealt with through global sharing of data for analysis. How is science being advanced? In some of the most exciting cases, it's being advanced by public sharing of huge data stores that anyone can then analyze for patterns. What's been some of the most important data in the United States over the last few centuries? It's census data that has been made publicly available in anonymous aggregate but in a format that could be cross-referenced with geography, commercial, and other data to better understand the world we live in.

The data that Facebook controls, conversations and social connections, could be used for analysis of real-time social patterns which could lead to world-shaking new insights. Do we get access to that data? No.

Why not? We don't get that access because Facebook was built on a fundamental promise of privacy and a complex system of privacy controls. Privacy is good, it's very good. But, the census gathers and exposes personal data without violating privacy. Lots of systems do.

Facebook's David Swain says that privacy on Facebook is evolving and the company is interested in facilitating this kind of analysis, but that Facebook is "not hearing users clamor for interesting analysis." That's a disingenuous argument coming from a company that is famous for pushing through important technologies against the wishes of its users, much less without their request.

The era of user-published content is still catching up to millions of people, but innovation is fast moving beyond that point. In order to deal with pressing global problems and truly leverage what's possible with contemporary internet technology, we need data-centric hooks extended for developers to grab hold of and build the future. Facebook is one of the biggest stores of data that innovation could be built upon. Once that kind of openness is achieved, then we'll say that Facebook is truly opening.

Data Store Uses Google Docs

The Data Store and its accompanying blog are more directly aimed at consumers. The Data Store houses a collection of all of the statistics that the newspaper has published. Unlike the New York Times, however, the Guardian does not yet provide an equivalent to the Data Visualization Lab. Interestingly, the Guardian chose Google Docs as its repository for this data. Here, for example, is the Guardian's spreadsheet with the U.S. public debt since 2001.

Future of the Newspaper?

We have already seen a number of very interesting uses of the NYT APIs, and we will surely see a lot of interesting and useful applications that will now make use of the Guardian's data. What is more interesting, however, is that some newspaper are clearly beginning to understand where the future of their business lies (the Guardian now also exposes full-text RSS for all of its sections).  If anything, this feels like a far smarter way to go than the MediaNews Group's misguided attempt to put proprietary printers into readers' living rooms.

Yawnlog is a wacky new site that lets you track how much sleep you're getting, note how good the sleep was, record your dreams and compare all of that information with your friends. This is no laughing matter! Imagine cross referencing aggregate sleeping hours and moods with a timeline of historically significant events. Silly as this service might sound, we think it sounds pretty cool, too.

Unfortunately the site doesn't seem particularly well constructed, but we sure love the idea. (Update: Lead developer Matt Blake emailed to let us know that the site was put together in a 24 hour hackathon and is only a few days old. Fair enough, hopefully Blake will be able to continue developing it.) We found Yawnlog over on KillerStartups, where you'll find a lot of apps that look like they were written by people who aren't getting enough sleep but some good ones too.

Imagine the possibilities! Say you're thinking of taking a job at one place that reports an aggregate of restless, stunted sleep patterns - but another potential employer's people have an unusually high rate of dreams about flying! Which job are you going to take? Say Republicans slept better after Obama's first major speech last night than Democrats did after Bush's first speech in office. Talk about tangible evidence of bi-partisan comfort with the Presidency! How much sleep, on average, does the staff of ReadWriteWeb get compared to other big tech blogs? More and better sleep, I'd bet.

Of course all of that would require far more adoption of a technology like Yawnlog than is ever going to happen, but we do like to fantasize. This kind of aggregate sleep pattern analysis will probably be a bullet point in the brochure for the forthcoming Google Brain Implant.

In the meantime, we got a kick out of Yawnlog and will see how long we can stick with using it. It's proof that you can socialize anything online. If you'd like to put our sleepy heads together, my profile is here, friend me up and let's share some dreams. I've added Yawnlog to my Morning Coffee Firefox plug-in, so I'll be reminded to make an entry first thing each morning.