data - ReadWriteWeb http://www.readwriteweb.com/feeds/search/data en Copyright 2012 Richard MacManus readwriteweb@gmail.com Tue, 14 Feb 2012 12:45:00 -0800 http://www.sixapart.com/movabletype/?v=4.35-en http://blogs.law.harvard.edu/tech/rss Data.gov Captchas Get Political Spotted this evening on the U.S. government's public data web site, Data.gov. A Republican programmer at captcha provider reCAPTCHA having some fun?

Hat-tip ReadWriteWeb's Marketing manager Elyssa Pallai, who stumbled upon this humor gem today.

]]>

]]> Discuss]]>
http://www.readwriteweb.com/archives/datagov_captchas_get_political.php http://www.readwriteweb.com/archives/datagov_captchas_get_political.php Humor Sun, 20 Dec 2009 00:31:22 -0800 Richard MacManus
Guardian Launches Search Engine for Government Data The Guardian, ostensibly a UK newspaper, but also a major proponent for opening data held by governments to use by outside software developers, has launched some software of its own: a search engine that unearths datasets and pathways to data sets provided by governments around the world. World Government Data Search is now live.

Yesterday the UK government released its new data site, data.gov.uk, to rave reviews (including ours). The new Guardian search engine searches across the UK, US, New Zealand and Australian governments' data sites. The company also offered up a gallery of the 10 best visualizations and mash-ups built on top of government data like this.

]]> The Guardian quotes developer Ben Fry on the future of searching government data: "This is only going one way: there is no trend towards less data."

Following an era when the quantity of data available online increased in orders of magnitude, thanks largely to easy publishing tools for end-users like blogging and social networks, many people expect the next era of development online to focus on strategic moves to make the most valuable data available in standardized formats that facilitate innovation by 3rd parties independent of the original sources of the data.

If large, standardized data sets are a new language, then it's time for a new period of literature to be written.

]]> Discuss]]>
http://www.readwriteweb.com/archives/guardian_launches_search_engine_for_government_dat.php http://www.readwriteweb.com/archives/guardian_launches_search_engine_for_government_dat.php News Thu, 21 Jan 2010 09:55:44 -0800 Marshall Kirkpatrick
Question about Wikis I have an Excel spreadsheet of data about RSS vendors that I want to share with the world. I recently did a report for a client, in which I defined and analysed 'The RSS Space'. As part of that report, I created a spreadsheet that mapped RSS vendors to categories - with a lot of help from some brilliant minds in the VC and RSS worlds. So what I want to do is publish that vendor mapping data onto a public wiki (I have my client's permission), so that everybody can publically expand and grow it - and benefit from it.

However, I'm having problems inputting an Excel spreadsheet into a Wiki. i.e. my data has a lot of rows and columns. Does anyone know of a wiki that makes it really easy to input tabular data? And probably more importantly, makes it easy for users to edit and add to that data. Please let me know either in the comments here, or by emailing readwriteweb AT gmail.com. Thanks!

]]>
http://www.readwriteweb.com/archives/question_about.php http://www.readwriteweb.com/archives/question_about.php Blogging Thu, 01 Sep 2005 15:57:35 -0800 Richard MacManus
Avoid Data Caps and Compress Your iPhone Data with Onavo onavo150.jpgIf you have to keep an eye on your iPhone data usage - whether it's due to data caps or roaming fees - then you might like Onavo, a new app that will monitor and compress your data so you can use (and pay for) less data.

Once you download the Onavo app and register your device, your data is routed through Ovaco's services, which compress your data and help you track your usage, identifying for example which apps are data-download-intensive.

]]> Once installed the app runs in the background of your iPhone and automatically compresses your download data (not your streaming data, it's worth pointing out). The app can be toggled on and off and will pause itself when you're using a WiFi connection. You can also opt to turn off email compression so that data is ignored.

onavoss.jpg

The trade-off of saving money on data here is, of course, running your data through a third-party service. Onavo does say "we take our users' privacy very seriously," storing the minimum amount of data - all aggregated and anonymized. Some metadata is kept to be able to generate the reports. Onavo says it doesn't story any data and can't read HTTPS traffic, with the exception of Exchange Mail authentication info, something that users will have to explicitly approve in order to set up.

The app works on iPhones and iPads and is coming to Android soon. It's currently free, but the startup indicates this is just a limited-time offer and it will eventually charge a fee to use the service. In the meantime, Onavo announced today that it has raised $3 million in funding from Sequora Capital and Magma Venture Partners.

]]> Discuss]]>
http://www.readwriteweb.com/archives/avoid_data_caps_and_compress_your_iphone_data_with.php http://www.readwriteweb.com/archives/avoid_data_caps_and_compress_your_iphone_data_with.php Apple Tue, 17 May 2011 14:33:27 -0800 Audrey Watters
Data.gov to Face a Challenger From Sunlight Labs Data.gov, the US federal government's new catalog of sets of public data for outside developers to mashup and analyze, now faces some friendly competition. The Sunlight Foundation, a non-partisan non-profit organization dedicated to government transparency, has announced that it will launch a National Data Catalog to go above and beyond what Data.gov offers.

We've been critical of the sparse offering from Data.gov but Sunlight's plan looks like it could surpass what even the most ambitious government program is likely capable of.

]]> Here's how Clay Johnson, head of Sunlight Labs, described the need for this project:

Because of politics and scale there's only so much the government is going to be able to do. There are legal hurdles and boundaries the government can't cross that we can. For instance: there's no legislative or judicial branch data inside Data.gov and while Data.gov links off to state data catalogs, entries aren't in the same place or format as the rest of the catalog. Community documentation and collaboration are virtual impossibilities because of the regulations that impact the way Government interacts with people on the web.

We think we can add value on top of things like Data.gov and the municipal data catalogs by autonomously bringing them into one system, manually curating and adding other data sources and providing features that, well, Government just can't do.

Johnson says the site, which is being developed with publicly visible open source code, will allow users to contribute sets of data, documentation to go along with data and links to places around the web that are using any set of data cataloged. It sounds like a ProgrammableWeb type site for public data.

In addition to government information, Sunlight will also be including derivative data sets, like information from the campaign contribution tracker OpenSecrets. That's something it's very hard to imagine a government website including.

Can Sunlight build a one-stop-shopping destination for public data, and will people make use of that? Time will tell, but it sounds like a very important project.

]]> Discuss]]>
http://www.readwriteweb.com/archives/datagov_to_face_a_challenger_from_sunlight_labs.php http://www.readwriteweb.com/archives/datagov_to_face_a_challenger_from_sunlight_labs.php Data Services Thu, 16 Jul 2009 11:10:17 -0800 Marshall Kirkpatrick
Visualize Your Data with Google Public Data Explorer

The Google Public Data Explorer takes large data sets and makes them palatable for public consumption, taking numbers, figures and other data and turning them into bar graphs, line graphs, maps and bubbles. When the explorer launched last year, it started out with 13 data sets. The number of data sets has more than doubled since then and it's about to get a lot larger.

Today, Google announced that it would be "opening the Public Data Explorer to your data," which means you can upload data and use Google's explorer tools to explore it visually.

]]> Omar Benjelloun, technical lead with the Google Public Data Team, writes that Google hopes "more datasets can come to life through Public Data Explorer visualisations and enable people to better understand the world around them and make more informed, data-driven decisions."

According to Benjelloun, the opening of Google Public Data Explorer comes with a new data format, called the Dataset Publishing Language (DSPL), an XML-based format, which data must be packaged in to use the explorer.

"Once imported," writes Benjelloun, "a dataset can be visualized, embedded in external websites, shared with others and published."

Google isn't the only one in the big data game, of course, and it's also not the only one attempting to make it simple for others to take part. There's a growing competition from sites like Factual, CKAN, InfoChimps and Amazon's Public Data Sets. Another thing to note with Google's implementation is that it is not a two-way road. Although uploading data to it's Public Data Explorer allows you to visualize that data, it isn't a way to share data sets. Google says that downloading data is something it's looking into, but that for now "you should be able to follow the links to data provider websites and download the data there."

If you're curious what Google Public Data Explorer is capable of, take a look at the embedded graph below. It's fully interactive and is created off of a public data set.

]]> Discuss]]>
http://www.readwriteweb.com/archives/visualize_your_data_with_google_public_data_explor.php http://www.readwriteweb.com/archives/visualize_your_data_with_google_public_data_explor.php Google Wed, 16 Feb 2011 12:12:47 -0800 Mike Melanson
Factual and SimpleGeo Team Up to Offer Developers a Better Geo-Data Toolkit factual_simplegeo.jpgDevelopers working on building location-aware applications need two at least two things: a robust set of tools and quality geo-data.

A new strategic partnership announced today between two of the leading geo-data startups, Factual and SimpleGeo, will help with just that, as SimpleGeo will now be incorporating Factual's global places data into its Places API.

]]> Factual's dataset contains more than 30 million places and local points of interest. The quantity of data is impressive, to be sure, but the key here is really the quality of that data. Factual has worked to "clean up" its location data and to standardize some of the naming conventions for the data fields. This is particularly important when it comes to handling data from multiple countries, for example, but questions of aggregating and normalizing data is a challenge that big data startups all face.

By making this partnership, SimpleGeo's CEO Jay Adelson says his company will be able to "focus on our core strengths" - its location-aware toolset that includes "Storage", its hosted database for location information and "Context," a tool that lets developers query for data relevant to a given location. The partnership will mean that SimpleGeo can continue to focus on the tools, while Factual will maintain the quality of the places data.

The timing of the announcement comes as interest in location-based technologies is rapidly expanding, and it's likely that both startups will benefit from a partnership that clearly delineates their areas of expertise. But developers too will benefit, says Factual CEO Gil Elbaz, as they'll have more support, better tools and better data.

It also means that geo-data now has, in Elbaz's words, a "sophisticated stack" upon which developers can build.

]]> Discuss]]>
http://www.readwriteweb.com/archives/factual_and_simplegeo_team_up_to_offer_developers.php http://www.readwriteweb.com/archives/factual_and_simplegeo_team_up_to_offer_developers.php Data Services Thu, 23 Jun 2011 13:00:00 -0800 Audrey Watters
"Siri, Why Are You Such a Data Hog?" With each new iteration of Apple's iPhone, we expect to see the addition of new features like speedier processors and better cameras. What isn't necessarily expected is that each subsequent device will consume way more data than its predecessor. But, in fact, this is the case.

The iPhone 4S uses about twice as much data as the iPhone 4 and three times the data than the iPhone 3G, according to a new study by Arieso. What causes the 4S to hog so much data? Just ask Siri.

]]> Every time you tap the button and ask Siri a question, it eats up about 63 KB on average, according to detailed testing done by Ars Technica. Sure, that's a modest amount of data, but if you use Siri frequently, it adds up over the course of a week.

Even local tasks like setting an alarm or adding a reminder use some data, since Siri bounces requests off of Apple's servers before executing them. Naturally, those types of queries take up less data. More Web-reliant requests like searches can use closer to 100 KB apiece. At the end of the month, it may not be enough to blow your data cap to smithereens, but it can add a good dozen or two megabytes.

The news that our smartphones are getting more data hungry comes at an inconvenient time, just as carriers are getting less generous about doling out those bytes. Sprint is now the only carrier offering unlimited data for smartphone users, and even that plan reportedly has its limitations. As smartphone usage has exploded, charging customers based on their data usage has become a way to both maximize revenue and keep congested networks under some control.

Regardless of the handset, data usage in general has been on the rise for the last few years thanks to the proliferation of devices, popularity of video and our increasing propensity to keep our digital stuff stored in the cloud.

]]> Discuss]]>
http://www.readwriteweb.com/archives/siri_why_are_you_such_a_data_hog.php http://www.readwriteweb.com/archives/siri_why_are_you_such_a_data_hog.php Apple Fri, 06 Jan 2012 07:00:19 -0800 John Paul Titlow
Idealware Releases Technical Guide to Data Portability Nonprofit tech analysts Idealware released a collection of resources today that anyone can use to evaluate APIs under consideration. Titled “Getting Your Systems Talking: A Framework to Evaluate APIs and Data Exchange Features,” the guide at its core is a worksheet that walks you through more than 30 different technical questions you should ask about any new data exchange technology you're evaluating. It's free to download.

While data portability is a hot topic of the day, there hasn't been a lot of tangible work done around the details yet. Idealware's guide could make implementation of these themes much more manageable. Readers may also be interested in this related discussion about data portability use cases over at the DataPortability.org public discussion.

]]>
http://www.readwriteweb.com/archives/idealware_guide.php http://www.readwriteweb.com/archives/idealware_guide.php Data Portability Thu, 17 Jan 2008 10:45:39 -0800 Marshall Kirkpatrick
Give Me My Data Helps Refill Blanked Facebook Profiles givememydata.jpgWe've reported that if you disconnect your Facebook account from the manufactured pages that now connect to things like hometown and interests, it will blank your profile.

The default response was a shrug. What can you do? Now an indie app, Give Me My Data, promises to replace that missing information on your profile.

]]> Owen Mundy, a Florida State art prof and the developer, explained the app and its use.

givememydatagraph.png

"Give Me My Data is a Facebook application designed to give users the ability to export their data out of Facebook for any purpose they see fit. This could include making artwork, archiving and deleting your account, or circumventing the interface Facebook provides. Data can be exported in CSV, XML, and other common formats."

The application does not export the data, then import it back into the profile section. Instead, a user can use the data to construct a new-old profile in a less restrictive part of their Facebook account.

]]> Discuss]]>
http://www.readwriteweb.com/archives/give_me_my_data_refills_blanked_facebook_profiles.php http://www.readwriteweb.com/archives/give_me_my_data_refills_blanked_facebook_profiles.php Facebook Sun, 02 May 2010 17:00:00 -0800 Curt Hopkins
Life in the Future, With Data: Livestreaming O'Reilly's Strata Conference mchui.jpg"Big data enables new ways to create value, it's going to change the basis of competition," Michael Chui of the McKinsey Global Institute said this morning to kick off O'Reilly's big data conference, the Strata Summit. The next two days are all about the rise of information that has to be dealt with on scale, big data, and its consequences. "It will change the way companies, sectors and economies compete," says Chui.

McKinsey published an exhaustive 150 page report on big data this Spring, which argued that data will soon become an economic input as important as labor and capital. It's not just about pure economics, though. As Edd Dumbill, chair of Strata, put it today, our relationship with big data needs to serve humans - not turn humans into the servants of machines and information overload. "We know that big data can help us, it may be the case that big data has to help us." Below, a live video stream of the next two days' proceedings addressing this mega-opportunity and trend.

]]> ]]> Discuss]]>
http://www.readwriteweb.com/archives/life_in_the_future_with_data_livestreaming_oreilly.php http://www.readwriteweb.com/archives/life_in_the_future_with_data_livestreaming_oreilly.php Data Services Tue, 20 Sep 2011 06:35:39 -0800 Marshall Kirkpatrick
UN Data: the Ultimate Research Tool The new site at UN Data allows anyone to access the United Nations Data Access System. This online, easy-to-use database was created by the UN in order to provide current, relevant, and reliable statistics to the whole world, for free. Using UN Data, you can access statistical information on populations, demographics, trade, commodities, agriculture, employment, the environment, industry, education, tourism, and much more.

]]> For many people, the first source for data is a Google search or a Wikipedia entry, however educational institutions and news agencies generally frown on using data from these sources, especially considering some of the inaccuracies found on Wikipedia not too long ago.

But now, students, journalists, and everyone else can access data straight from the source - the actual statistics published by the UN.

Using UN Data

Underneath the search box on the homepage are a few popular searches, based on what users are searching for the most. You can either click on those or enter in your own keywords to begin.

For example, to see data about a country, you just enter in the country's name and click "Search." The search result will take you to a landing page where some general information about the country is provided, like population, GDP, life expectancy, etc. You can click the link to "view full profile" to get the full details of that country's statistics.

To the left, is a box where you can apply filters to your search by specifying that you want to include or exclude certain information from your search.

Below the country's profile are all the search results for that country. Each result has a "Download" link to download the data, but there is also a handy "Preview" link which will open a small window displaying the data, so you don't have to navigate away from the page you're on to see if that data is what you were looking for. Another link, "Explore," allows you to delve into to related data sources around that topic.

You can do more than search for data on specific countries or regions, though. You can also search for data about global statistics, like greenhouse gas emissions or global solar production statistics (well, that's what I looked up).

What's great about the UN Data site is not just the ability to access this huge database of statistics (currently 55 million+ records), but the well-designed way that such massive amounts of data have been provided to the public. Anyone can use this database - school kids, techies, even mom and dad. This is one site that is definitely worth a look.

]]> Discuss]]>
http://www.readwriteweb.com/archives/un_data_the_ultimate_research_tool.php http://www.readwriteweb.com/archives/un_data_the_ultimate_research_tool.php Product Reviews Thu, 06 Mar 2008 14:42:29 -0800 Sarah Perez
Have Issues With Big Data? Check Out These 5 Data-as-a-Service Startups dealmaker-150x150.png

Data is big. By that, we mean that there's not only a lot of data, but handling all that data is a problem that everyone from small businesses on up to the big boys, like Google and Facebook, has to tackle in one way or another.

To help with the problem of handling Big Data, Dealmaker Media has announced a group of startups that it says are "providing answers to all your database problems in unique and innovative ways."

]]> Dealmaker Media, the company behind the Under The Radar conference that brings together a number of early stage, "bleeding-edge technology startups" each year, has put together a list of companies under the title of "Data-as-a-Service."

"Scaling databases is a hard problem," writes Dealmaker Media Community Manager Clare Jacobson. "In fact, both Google and Facebook have had to build their own solutions to avoid buckling under today's online and data-thirsty society."

Companies that can't build their own solutions, writes Jacobson, are left "looking to innovative startups to provide scaling solutions."

So, what companies did Dealmaker pick for their collection of data-as-a-service recommendations?

  1. Akiban 
  2. Clustrix
  3. Hadapt
  4. Nimbus
  5. RethinkDB

If you find yourself, like so many other companies, unable to handle the data deluge, then one of these five companies may be able to save you from developing your own solution. The full blog post offers more details on each of the companies.

]]> Discuss]]>
http://www.readwriteweb.com/archives/have_issues_with_big_data_check_out_these_5_data-a.php http://www.readwriteweb.com/archives/have_issues_with_big_data_check_out_these_5_data-a.php Data Services Mon, 04 Apr 2011 16:48:50 -0800 Mike Melanson
HTML5 Calendar Data Sharing Standard Released as Public Draft by W3C This morning, the World Wide Web Consortium (W3C) published the first public draft of its Calendar API spec, the technical standard by which it recommends applications and calendars share event data. It just so happens that it was exactly 5 years ago to the day today that Google Calendar released its API!

Such data standards make it easier to develop apps that use your calendar data and make more viable the development of new calendar software without fear that users will be locked out of application ecosystems.

]]> The W3C spec is made for an HTML5 web, requires granular permissioning (permission for an app to read your calendar is not permission to write to it) and lays out detailed if increasingly common data privacy practices.

]]> Discuss]]>
http://www.readwriteweb.com/archives/html5_calendar_data_sharing_standard_released_as_1.php http://www.readwriteweb.com/archives/html5_calendar_data_sharing_standard_released_as_1.php Data Services Tue, 19 Apr 2011 11:40:24 -0800 Marshall Kirkpatrick
Infochimps: Share and Sell Your Raw Data Invite code available at bottom of the article!

Although the data repository Infochimps has been open for a year or so now, the company is making a big announcement at the DEMOfall 09 conference today. Now, in addition to simply being one of the best sources for finding raw data online, you're able to share your data - or even list it for sale - through new site features which the company hopes will encourage businesses to open up their commercial datasets to the world.

]]> Infochimps originally caught our eye back in April of 2008 when it made our list of the best places to find open data on the web. The beauty of this site, which is essentially a specialized search engine for raw data, is that it makes finding relevant data much easier than if you tried to do so using a traditional search engine like Google. For example, a query for "music" returned, among other things, a listing for Last.fm Artist Tags from 2007. A similar search on Google wouldn't pull in that link until you hit the 42nd page of search results. In other words, you would never find it on Google.

At Infochimps, data can either be hosted on site in a standard format like CSV, XML, or YAML, or it can simply point to an external source. While the data itself cannot be manipulated on the site, the metadata like the description and tags can be edited by anyone who creates an account at Infochimps.org. The site founders have seeded the site with some data already - like the data from the comprehensive knowledge archive network - but the majority of the data is user-submitted. They've also partnered with Amazon to share Infochimps data with Amazon's Public Data Sets service. To date, one-third of Amazon's Public Data Sets were contributed by Infochimps.

New Announcements: Sell and Share Your Data

Today, the company is opening up and allowing anyone to upload their own datasets. The data can be any raw data that has an open license. To get started, users just upload it to the site and Infochimps will then handle the storage and distribution. Also, for any truly valuable data, be that commercial data a company wants to share or some sort of data manipulation - like editing awful census data into a format for use with MySQL databases - users can now charge for downloads, too. The price for the dataset can be set to any amount, however Infochimps makes its money by taking a 20% cut of all data sold.

There really isn't a company that's doing anything like Infochimps. Amazon's Public Data Sets comes close, but isn't nearly as extensive. And although other repositories of data exist, (Archive.org and the newly launched Data.gov come to mind), these resources focus on one particular type of data as opposed to providing a search engine for all data.

Those who have data to share or sell can now do so as of today: just visit Infochimps.org to get started. ReadWriteWeb readers who use this service can use the code RWWrocks to get in.

]]> Discuss]]>
http://www.readwriteweb.com/archives/infochimps_share_and_sell_your_raw_data.php http://www.readwriteweb.com/archives/infochimps_share_and_sell_your_raw_data.php Product Reviews Tue, 22 Sep 2009 16:30:00 -0800 Sarah Perez