However, I'm having problems inputting an Excel spreadsheet into a Wiki. i.e. my data has a lot of rows and columns. Does anyone know of a wiki that makes it really easy to input tabular data? And probably more importantly, makes it easy for users to edit and add to that data. Please let me know either in the comments here, or by emailing readwriteweb AT gmail.com. Thanks!

We've been critical of the sparse offering from Data.gov but Sunlight's plan looks like it could surpass what even the most ambitious government program is likely capable of.

Because of politics and scale there's only so much the government is going to be able to do. There are legal hurdles and boundaries the government can't cross that we can. For instance: there's no legislative or judicial branch data inside Data.gov and while Data.gov links off to state data catalogs, entries aren't in the same place or format as the rest of the catalog. Community documentation and collaboration are virtual impossibilities because of the regulations that impact the way Government interacts with people on the web.

We think we can add value on top of things like Data.gov and the municipal data catalogs by autonomously bringing them into one system, manually curating and adding other data sources and providing features that, well, Government just can't do.

Johnson says the site, which is being developed with publicly visible open source code, will allow users to contribute sets of data, documentation to go along with data and links to places around the web that are using any set of data cataloged. It sounds like a ProgrammableWeb type site for public data.

In addition to government information, Sunlight will also be including derivative data sets, like information from the campaign contribution tracker OpenSecrets. That's something it's very hard to imagine a government website including.

Can Sunlight build a one-stop-shopping destination for public data, and will people make use of that? Time will tell, but it sounds like a very important project.

While data portability is a hot topic of the day, there hasn't been a lot of tangible work done around the details yet. Idealware's guide could make implementation of these themes much more manageable. Readers may also be interested in this related discussion about data portability use cases over at the DataPortability.org public discussion.

But now, students, journalists, and everyone else can access data straight from the source - the actual statistics published by the UN.

Using UN Data

Underneath the search box on the homepage are a few popular searches, based on what users are searching for the most. You can either click on those or enter in your own keywords to begin.

For example, to see data about a country, you just enter in the country's name and click "Search." The search result will take you to a landing page where some general information about the country is provided, like population, GDP, life expectancy, etc. You can click the link to "view full profile" to get the full details of that country's statistics.

To the left, is a box where you can apply filters to your search by specifying that you want to include or exclude certain information from your search.

Below the country's profile are all the search results for that country. Each result has a "Download" link to download the data, but there is also a handy "Preview" link which will open a small window displaying the data, so you don't have to navigate away from the page you're on to see if that data is what you were looking for. Another link, "Explore," allows you to delve into to related data sources around that topic.

You can do more than search for data on specific countries or regions, though. You can also search for data about global statistics, like greenhouse gas emissions or global solar production statistics (well, that's what I looked up).

What's great about the UN Data site is not just the ability to access this huge database of statistics (currently 55 million+ records), but the well-designed way that such massive amounts of data have been provided to the public. Anyone can use this database - school kids, techies, even mom and dad. This is one site that is definitely worth a look.

Although the data repository Infochimps has been open for a year or so now, the company is making a big announcement at the DEMOfall 09 conference today. Now, in addition to simply being one of the best sources for finding raw data online, you're able to share your data - or even list it for sale - through new site features which the company hopes will encourage businesses to open up their commercial datasets to the world.

At Infochimps, data can either be hosted on site in a standard format like CSV, XML, or YAML, or it can simply point to an external source. While the data itself cannot be manipulated on the site, the metadata like the description and tags can be edited by anyone who creates an account at Infochimps.org. The site founders have seeded the site with some data already - like the data from the comprehensive knowledge archive network - but the majority of the data is user-submitted. They've also partnered with Amazon to share Infochimps data with Amazon's Public Data Sets service. To date, one-third of Amazon's Public Data Sets were contributed by Infochimps.

New Announcements: Sell and Share Your Data

Today, the company is opening up and allowing anyone to upload their own datasets. The data can be any raw data that has an open license. To get started, users just upload it to the site and Infochimps will then handle the storage and distribution. Also, for any truly valuable data, be that commercial data a company wants to share or some sort of data manipulation - like editing awful census data into a format for use with MySQL databases - users can now charge for downloads, too. The price for the dataset can be set to any amount, however Infochimps makes its money by taking a 20% cut of all data sold.

There really isn't a company that's doing anything like Infochimps. Amazon's Public Data Sets comes close, but isn't nearly as extensive. And although other repositories of data exist, (Archive.org and the newly launched Data.gov come to mind), these resources focus on one particular type of data as opposed to providing a search engine for all data.

Those who have data to share or sell can now do so as of today: just visit Infochimps.org to get started. ReadWriteWeb readers who use this service can use the code RWWrocks to get in.

Called the Data Liberation Front, the project team said in a Google blog post today that it has "liberated" more than half of the major Google services. "In the upcoming months," writes project lead Brian Fitzpatrick, "we also plan to liberate Google Sites and Google Docs (batch-export)."

Making sure the door isn't locked if users choose to leave a service is a required, if less exciting, part of the data portability movement. Just as important as a bulk dump of user data is the option for users to easily and securely port data online from service to service for immediate personalization based on past activity at a legacy site. Google is a market leader in that kind of data portability as well.

The information on DataLiberation.org does not include instructions for deleting your data from Google's servers. The project is taking suggestions for acts of liberation on a Google Moderator page and is publishing updates on Twitter.

GeoXtract doesn't require you do do any manual processing of your data or programming and you don't even have to modify your data set in any particular way prior to using the program.The map created by the program can be published to the geoXtract web site, to your own site, or simply saved to your computer.

Here's an example of a map made with geoXtract showing Clearwater, Florida beach motels:

Each map created can be saved as a "project" within the program, so if you ever need to recreate or update the map, it's easy to do.

The application comes in four versions: Starter, Plus, Pro, and Enterprise. The Starter version is free and lets you work with .XLS and .CSV data sets. You're also limited to 3 saved projects with no more than 100 records each. The paid versions increase these limits while also enabling the ability to work with various other file types, like .MDBs or even OLEB/ODBC in the Enterprise version.

You can download the application for yourself from here.

The motion that was submitted to the city council points out several reasons why endorsing open data, open standards and open source policies is an good idea and city councillors has resolved to:

* Identify immediate opportunities to distribute more of its data;
* Index, publish and syndicate its data to the internet using prevailing open standards, interfaces and formats;
* Develop appropriate agreements to share its data with the Integrated Cadastral Information Society (ICIS) and encourage the ICIS to in turn share its data with the public at large
* Develop a plan to digitize and freely distribute suitable archival data to the public;
* Ensure that data supplied to the City by third parties (developers, contractors, consultants) are unlicensed, in a prevailing open standard format, and not copyrighted except if otherwise prevented by legal considerations;
* License any software applications developed by the City of Vancouver such that they may be used by other municipalities, businesses, and the public without restriction.

Other cities like Washington, DC, Portland and Toronto have expressed interest in adopting open policies for their information and data, but so far Vancouver is the only one to explicitly spell it out in an agenda motion.

Problems with data transferring, usability and licensing issues have kept many municipalities from adopting such policies. Hopefully Vancouver can overcome these issues and become the world's first truly "open" city.

New federal CIO Vivek Kundra is in charge of the site, which will act as a central repository for government data, including XML, CSV, KML files and more. At launch a mere 47 data sets are included and they appear to lean towards the least controversial matters. None the less, it's exciting to see the effort happening. Hopefully some awesome mashups are on the way!

There are many, many sets of data available from the federal government but the Data.gov site says it was selective about quality and standards when choosing what to include. It's hard not to compare other sources of government data and feel disappointed, though. The privately built USGovXML.com contains far more data and was built by one independent developer over four months. That site lists ten Department of Interior XML feeds, for example, none of which appear on Data.gov. You can find a feed of food recalls there, but not on Data.gov.

Twenty six government agencies are represented in the catalog, though not all are offering raw data. The FBI is listed as a source but only offers a widget that can be placed on websites, not access to raw data.

New York Times data wonk Derek Willis pointed out that the initial offerings are non-controversial. "Most are from USGS, EPA and National Weather Service," Willis observed this morning. "No [data from] Department of Homeland Security, State or DOJ."

Likewise, a search of the data sets for keywords like food, prisons and drug all bring up zero results. Those are examples of particularly important topics because they are matters of justice and injustice - shedding light into dark corners where injustices are being perpetrated is one of the most important things that government data and the subsequent computer assisted reporting can accomplish.

There are no RSS feeds available for the whole catalog or search queries, something that would be very useful for tracking additions of new data. We expect that will change soon.

People will no doubt argue that some data is much better than no data, and while that's true: for a new federal office to engage with such an important topic with the weight of history and the whole administration behind it and then come up with something this limited is disappointing.

API and mashup watcher John Musser of ProgrammableWeb was more generous than we are about the initial offerings:

"They're off to an excellent start. It's a big step in accessibility of government data. As we've been seeing with other v1 gov-data efforts, like the recently available data on senate votes: step one is give people structured data like xml, step two (or later) is to make it available via an API. They have a healthy amount of metadata. The number of data sets is not that large, but of course it's just the beginning."

It is just the beginning and we applaud the launch of this effort. We hope that the initial launch will pale in comparison to the long term value of this collection of data.

The folks at Sunlight Labs, Google, O'Reilly/TechWeb and Craig Newmark just launched a new part of their Apps for America contest to build the best mashups and data visualization tools for data in the new Data.gov site. Check it out!

See also the newly launched Whitehouse.gov/open - launches today just keep popping up.

Called AWS Hosted Public Data Sets, the service will enable you to use public data within your Amazon EC2 environment. Select public data sets will be hosted on AWS for free as an Amazon EBS snapshot.

However, as there is no indication that the Gutenberg Project will be added to AWS, we've calculated how long it would take to download and upload the 80GB UGI Virtual Conformer Library, one of the listed data sets AWS plans to host.

Using a residential cable provider in California, it would take 22 hours 36 minutes to download, and 3 days 36 minutes to upload to a server in the same state. However, if the server was in New York and we accessed it from California, it would take 3 days 42 minutes to download, and 7 days 14 hours to upload. Clearly inefficient.

People have been searching for better ways to access public data sets for some time, and AWS Hosted Data Sets may just be the answer they've been looking for; allowing anyone to do the type of computing that in the past has been limited to large organizations with lots of money.

Current data sets that Amazon are working on include: annotated Human Genome data, PubChem and UGI Virtual Conformer libraries, the U.S. Census, various labor statistics, and various economic and transportation databases.

AWS will continue to add to the collection over time, and this is where you come in.

If you have a public data set and hold the rights to the distribution of it, you can submit a request on the AWS Public Hosted Data Sets site to have it included.

This is huge.

Further, even though Microsoft keeps talking about data portability, this announcement is not about their participation in DataPortability.org. Today's announcement is strictly about contact data, and it is strictly about Microsoft and these five sites. So while it is technically about making some data more open and portable, the old silos still exist -- there are just a few new paths cut between a handful of them. (To be fair, Microsoft's end of the contact data is open to everyone via their Contacts API, which has been around since last April. DataPortability.org, though, is about using existing, open standards to share all user data.)

Microsoft did mention their involvement in DataPortability.org in their announcement of today's news, however, saying that this and it are natural steps in their "pursuit of data portability to put users at the center of their online experience, while at the same time being thoughtful about balancing user security and privacy with the experience." Here's hoping everyone continues to open up.

Update: We just heard back from Google - here is the official statement about which phones will be able to contribute traffic data to Google Maps:

Google Maps products that include location services will make use of this information for traffic. This includes the downloadable Google Maps for Mobile product for mobile phones as well as the Google Maps application for Android phones. One exception is the maps functionality that Google provides for the iPhone - the iPhone does not provide any location data that is used for traffic crowdsourcing at this time.

Google is obviously aware of the potential privacy issues involved here, but according to Dave Barth, the product manager for Google Maps, Google will "find the start and end points of every trip and permanently delete that data so that even Google ceases to have access to it." This way, nobody can find out where a car actually came from.

According to Google, enough users use these phones to make this project feasible. In my experience however, the data for these arterial roads isn't quite as trustworthy as the data for highways.

Overall, though, this is a great project and the more users are aware of it, the better the data will become. However, it is also worth noting that some GPS manufacturers have been using data from their users' GPS systems to crowdsource traffic data for years, though chances are that Google will be able to recruit more users and hence create a better experience for its users.

Social networking site MySpace is releasing a set of APIs later today that will allow developers to implement protocols for exchanging and synchronizing user profiles between their own applications and MySpace profiles. Users can share their basic profile information, as well as their MySpace photos, videos, and friend list.

As we reported earlier, MySpace has signed up a number of high profile partners for this launch: Yahoo, Twitter, eBay and Photobucket.

As Steve notes, if MySpace's Data Availability API is adopted by a large enough number of sites, it could make MySpace the de facto host for social network profiles.

The details around Facebook Connect are still a bit unclear, but Facebook will probably update its developer community about its plans at next month's F8 conference.

For users, being able to have their profiles linked between different services is a major step forward in terms of usability. It would be very convenient to have a Twitter profile automatically synch with a MySpace (or any other service's) friend list, or only having to update photos on Photobucket and then see them directly on MySpace.

"Of course there should be a simple Delete button. It should do the typical "Are you sure? YOu can use Deactivate if you think you might come back?" confirmation process. But it is NOT Facebook's place to babysit users. If someone deletes their info and regrets it... well, that's life.

I despair of companies ever learning this though. Remember the horror stories about canceling AOL accounts? Everyone agreed that AOL should make it easy and yet here we are with not only Facebook but a bunch of other social media sites making it hard to control my data.

For all of the lip service given to data portability and that it's my data, most companies don't really believe it. Oh, they'll make it easy to import your data from other services... but export? Delete? Not so much."

But in recent years, that has changed. Alexa now faces competition from Compete, which launched a similar public service in 2006 (our coverage), and from Quantcast, which was founded in 2005. Both of those companies gather data from numerous outside sources and their rankings are generally seen as more accurate than Alexa's.

"In recent months we've heard from our Alexa users that understanding Internet usage beyond Alexa Toolbar users was increasingly of interest," wrote Alexa in the announcement of their rankings overhaul. Recent months? The inaccuracy of the toolbar-based rankings has been discussed for years, which is why we think this might be too little, too late for Alexa.

Beyond the problem of public perception, Alexa also still displays their data in non-standard ways. The hard-to-understand pageviews per million, reach per million, and rank are not easily compared to other data sources, which makes Alexa's information less useful than it could be, even if it is presumably now more accurate.

Historical data on Alexa is currently only available for the past 9 months while the company recalculates old data with its new ranking algorithm.