Google Takeout was unveiled in summer 2011. It allows Google users to export all their Google data to disk or just data from individual services. It's all thanks to the Data Liberation Front team, which builds tools to give Google users control over their data.

"If you lock your users in, it makes you complacent," Google's Brian Fitzpatrick said at OSCON last summer. Contrast Google's per-product liberation with Facebook's feeble efforts at data portability. You can't even really delete a Facebook account; you can only suspend it. Google's Data Liberation Front makes user data portable back and forth between Google services and standard file types on the user's computer.

While Google Docs already had its own export option, its inclusion in Google Takeout is an important step. Takeout is the only place users have to go to get their information out of Google. Google's business is built around gathering data from its free services, but the Data Liberation Front lives up to its promise to give users final control. Especially for Docs, which has all kinds of applications for work and other sensitive uses, inclusion in Google Takeout is a great step for users.

Liberate your data at google.com/takeout.

Have you ever used Google Takeout to export your data?

Let's not get into whether Diaspora can take credit for features of Google Plus and Facebook. There are things about Diaspora that still are unique among its competitors. Not only is it open-source, it's decentralized and distributed. Users are encouraged to set up their own servers. But these are not features for normal human users. In that category, the social networking superpowers seem to have Diaspora cornered.

Circles and Aspects

Diaspora has been called the anti-Facebook for its strong privacy stance, and it had "aspects" before anyone knew about Google Plus and its circles. But Plus exists now, and Facebook is coming around to this whole "privacy" thing. And really, for Facebook, it was just an interface problem, anyway. Why does everybody forget that Facebook has had publishing to select friend lists for a really long time?

If Google Plus has taught us anything, it's that normal people don't feel like leaving the social networks where they already feel settled. Diaspora lets users post their updates to Facebook, Twitter and Tumblr, but it still wants to be a destination of its own. Now that all Diaspora's user-facing features are available in the dominant social networks, that looks like a tall order.

Social Networks are Sticky

Sometimes, Diaspora seems like it is only a dream; it's still in closed alpha, so nobody's home. It's real, though; I promise. I've had alpha access for about a year, although I've only convinced 10 of my friends to accept my invites. And some of them haven't added so much as a profile pic yet. And none of us, myself included, has posted more than once.

Social network inertia is real. Has Google Plus killed Facebook yet? No. Diaspora can take credit for Google Plus circles all it wants, but Google reached beta first. I'd be happy to show you more Diaspora screenshots, but it's really mostly white space and things that look like Google Plus.

Diaspora's Defense

Yosem Companys, Diaspora's chief evangelist, insists that we do need a new home on the social Web. "Facebook really was fun for a while there," he says. "People who showed up revealed and shared their authentic selves. ... Today, what we see is people using Facebook only to say the most innocuous personal things, to market their own services, or to forward content created by others. As a result, Facebook has become boring and even depressing. We still go there because it's part of our routine, and for fear of missing something, but it's like an old favorite TV show on its 8th season, when the original magic is gone."

That's anecdotal, though. I've been a Facebook user since 2005, and it got depressing for a while, but I think certain recent features, particularly the Groups overhaul, have made it as fun and fulfilling as ever in its current incarnation, if you can get over the ads. But that's all just opinion. The data, however, show that Facebook is absolutely crushing it as far as traffic, and it is offering new mobile extensions of its network into the real social world where people have fun. Diaspora use is a desk activity.

Regarding Google Plus, Companys hits the new kid on the block where it really hurts: the real names policy: "All kinds of people have good reasons to be inhibited by this policy, including people looking for work, women, people expressing views different from their parents', famous people, and pretty much anyone outside the mainstream of the community where they live." Diaspora has no such policy, and it offers progressive new ways for members to self-identify, such as an empty text field for gender.

But is this enough to convert the hundreds of millions of people who have already settled into social networking habits? Only if Companys' anecdotal evidence is true, and that remains to be seen.

Whither Diaspora?

Is there anything Diaspora can do? I think so, but it's a departure from it's current incarnation, which is an awful lot like Google Plus (or vice versa, or whatever). It's unrealistic to expect a mass exodus from one social network that works to another of which no one has ever heard. Diaspora's potential is in its ability to syndicate to our other services (currently Facebook, Twitter and Tumblr) while still allowing us to own our data. Companys notes that "our service already includes outbound syndication ... and it will include inbound syndication, too." So this use case sounds like it's in the cards.

If Diaspora is built as a publishing platform that lets us own our content and direct it to our existing networks - and especially if we can read from them, too - it would be an awesome, welcome tool that even Dave Winer could love. But if the launch of Google Plus wasn't splashy enough to start a mass Facebook exodus, a later launch of a service that looks the same is not going to do it.

What isn't working for you about the social Web? What would you like to see change?

Just before the weekend, Facebook announced that it would begin allowing third-party applications and websites to request that users share both their mobile phone number and address. Now, the company has said that it will be putting the new feature on hold while it makes changes to make sure that Facebook users are aware of the potential for data sharing.

Douglas Purdy, director of developer relations, just posted on the Facebook developer blog to explain that Facebook agrees with its critics that the feature could be better implemented and the company will be pulling it until changes are made.

"Over the weekend, we got some useful feedback that we could make people more clearly aware of when they are granting access to this data. We agree, and we are making changes to help ensure you only share this information when you intend to do so. We'll be working to launch these updates as soon as possible, and will be temporarily disabling this feature until those changes are ready. We look forward to re-enabling this improved feature in the next few weeks."

Purdy was just hired last November to "improve Facebook's relationship with the community." This move will not only help with Facebook's developer community, but potentially its user community too.

A primary complaint of many commenters, developers and members of the data portability community was that the permissions dialog design did little to convey to users that they were now sharing information that had previously been kept for use on Facebook, not third-party applications. Take a look at the permission dialog:

While the request for new information seems apparent here, when it becomes part of a much longer list of permissions, it may easily slip past without users noticing. Purdy was not specific in what changes would be made, but we hope that it has to do with at very least the design of the dialog, if not even allowing users to have granular control over what they share with who at the time of sharing, not in a separate settings page.

For an in-depth look at the new feature, give "Facebook & Identity: The Continued Push Toward Becoming Your One True Login" a read.

Facebook announced this weekend that it would begin offering third-party applications and websites a way to request that users share their "current address and mobile phone number." Simple enough, right?

While Facebook says that the move is intended to make it easier for users to take their information with them across the Web, members of the data portability community argue that its yet another move by Facebook to lock users in. At the same time, the question of how Facebook handles these permission requests and whether or not it's clear to the end user is a point of contention that further calls into question how a user's online identity should be handled.

We asked some experts in the field and here's what they had to say.

Facebook: It's All About Portability

First, Malorie Lucich, a Facebook spokesperson, told us that the impetus for the change comes down to enabling users to bring their information with them, rather than forcing them to type in the same information wherever they go:

We want to make it easy for people to take the information they've entered into Facebook with them across the web. This new permission gives people the ability to control and share their mobile phone number and address with the websites and apps they want to use for more efficient experiences. As always, no information will be shared with an app or website until a user explicitly chooses to share it.

David Recordon, a senior open programs manager at Facebook and member of the OpenID Foundation, commented yesterday that the feature was intended to offer precisely this portability. "Given that I trust Kickstarter enough to give them my credit card information, I also trust giving them with my address," wrote Recordon. "Why should I need to type my address in again versus them being able to ask me for it?"

Other members of the data portability community, however, see things differently and call into question Facebook's methods of data sharing.

A Central Hub or Cog in the Machine?

"The problem is not that the user can (and must be able to) choose to access their data from Facebook on a 3rd party site," said Chris Saad, co-founder of the Data Portability Project and VP of strategy at Echo. "The problem is that Facebook has architected the whole thing from the beginning to be an exclusive hub and spoke relationship with them rather than a peer to peer relationship on the open web."

According to Saad, Facebook's intentions are simple - to get third-party sites and apps to rely entirely on Facebook for user information. "It's like giving a taster," he said. "Paying comes later."

Alana Joy, an independent digital strategist, agreed with Saad that Facebook was vying to be the central hub, rather than another cog in the machine.

"Facebook seems to be more concerned with positioning themselves as the official global 'people registry', harvesting users' information for profit," said Joy, "than they are about providing a safe place for individuals to share their lives with only those they choose to."

Of course, Facebook founder Mark Zuckerberg has said rather plainly that the company wants to push people's boundaries and expand the idea of public versus private. A key point that the company stands behind is that the user is given the opportunity to decide whether or not they share this information, both in privacy settings and in the permissions dialog.

"We introduced the granular data permissions model last April to make it clear to users exactly what information an app or website is requesting, allowing them to make more informed decisions about whether or not they want to proceed," said Lucich. "That being said, we're always looking into ways to make the platform user experience as positive and clear as possible."

With Third-Party Apps, It's All or Nothing

This sort of granularity isn't enough for Elias Bizannes, a board member of the Data Portability Project.

"Something bugs me about the Facebook connect privacy options," said Bizannes. "When you connect, you see what permissions you have to give, but you don't have an option there to deny individual permissions."

According to Facebook, apps should only be requesting information they absolutely need and therefore, were a user to deny access, the app wouldn't be able to function anyway.

"The reason why people can't pick and choose what data to share is because the app needs the requested information in order to perform its core services, such as photo access for a photo app and birthday date access for a calendar app. Developers are required to only request the information it needs to provide a customized experience," wrote Lucich. "Ultimately, it's up to the user whether or not he or she wants to grant an app access to their information."

Bizannes argues that the current model of permission forces users to blindly accept and allow the sharing of their information, "because they know if they click 'no' they don't get any access to the app."

"Users should have the ability to decide upfront what data they permit, not after the handshake has been made where both Facebook and the app developer take advantage of the fact most users don't know how to manage application privacy or revoke individual permissions," argued Bizannes. "Data Portability is about privacy-respecting interoperability and Facebook has failed in this regard."

On The Other Hand...

Inside Facebook's Josh Constine takes a more tempered approach to the whole thing, suggesting that instead of kowtowing to fear, Facebook should "instead push forward while minimizing negative outcomes by helping users make [a] more informed decision."

Constine's article echoes Recordon's sentiments, reminding the fearful that Facebook has been diligent in other instances and made sure that third-parties were dealt with appropriately when they mismanaged user information. The new feature may ask users for more in terms of personal information, but "there are many benefits to allowing developers to ask users for their contact information," writes Constine. "Mobile phone number access could power apps that act as up-to-the-minute communication hubs between groups of friends, allowing members to be notified by SMS when friends are nearby, want to plan an event, or upload new content. Home address access could let ecommerce sites pre-populate delivery details during checkout, leveling the playing field so smaller merchants can compete with established giants like Amazon that have already forced users to type in their address manually."

Saad argues that Facebook continually ups the ante without properly notifying its users.

"It's obvious that most users don't understand the bargain they're making - and that's mostly because Facebook keeps changing it," said Saad. "The same prompt that once granted some basic permissions for authentication now grants permission to your whole life."

Pete Warden, a big data guru who recently found himself on the wrong side of Facebook's line between public and private, says that a big result of this new feature will be for third-party sites and applications to figure out who you really are.

"The most interesting part of this change is the ability it gives companies to connect Facebook accounts with information from offline databases," said Warden. "There's a treasure-trove of marketing data available for every household in the US that they'll now be able to use to profile their Facebook users for everything from buying habits to income, children and pets."

According to Warden, this sort of data will help sites connect your activities to your real identity in ways that were previously not possible.

While the word "privacy" has been a sore point for Facebook over the past year, is it that Facebook is entirely cavalier with user information or is it just that its scale provides an easy target? Warden said that Facebook isn't much different from many other sites. "They're just behaving like the rest of the marketing world, the only difference is that they're under a lot more scrutiny than everyone else."

To Share or Not to Share?

In the end, does this come down to the simple question of whether or not you should ever share this information if you don't want it to be public? This is a solution that many users advocate. On the other hand, should users be petitioning for a more granular control over their information, wherein they can make more individual decisions over when and where they share what information? The fact that Facebook wants to be your one true login is no secret and the company will continue to push the boundaries in this realm. How will most users react?

"Most people won't care until they personally are victimized by it," predicted Alana Joy. "Like the gentleman whose profile picture was used in a Facebook ad for singles when he is a married professional with a family. [...] It really does illustrate just how Facebook can and will do whatever they want with whatever you post there."

We spoke with a few members of the data portability community to see what they had to say about the debate between these two big companies and what it means for the rest of us.

Eve Maler (aka XMLgrrl), host of the User-Managed Access group among other things, started off by reminding us that "Facebook's end-users are not its customers; they're the product." While Google's relationship to its end-users is much the same, she said, "it has developed a strategic stance on privacy and data portability that accepts and promotes greater user control of the personal data it sees, and this allows Google to capture the high ground in this debate."

Forcing Facebook's Hand

Chris Saad, co-founder of the Data Portability Project and VP of strategy at Echo, agrees with Maler that Google has the upper hand in the debate.

"Facebook has been a one-way beneficiary of Data Portability for far too long," said Saad. "Google asking for reciprocity from equal peers on the internet is a perfectly reasonable, if not overdue, move."

Elias Bizannes, executive director of the Data Portability Project was a bit more ambivalent in his approach, framing the entire situation in terms of global trade negotiations. According to Bizannes, nobody is winning at this point.

"This tit-for-tat approach is what governments still practice with trade and people-travel restrictions," explained Bizannes. "The reality is, Facebook and Google are hurting the global information network."

When user data can be moved around at a user's discretion, then for a company to have the best product becomes more important than having the most data.

Both Saad and Maler, however, said that the dispute was moving things in the right direction.

"Battles like the current one," said Maler, "along with the mutually reinforcing effects of social pressure and media attention, are a key way to ratchet up data control across the board for end-users who aren't paying customers of these services."

Saad said that it was unfortunate that Google had to add the clause to their TOS, "but it is clear that there was no other way to force the issue. Many open standards on the Internet - including HTTP, HTML, DNS etc assume/force two way openness, and Google is just trying to re-establish some fair use."

In the end, Bizannes said that both Google and Facebook would benefit from working together rather than duking it out.

"These companies need to recognize what their true comparative advantage is and what they can do with that data," explained Bizannes.

What a site really wants is persistent access to a person so they can tap into the more recently updated data, for whatever they need. [...] Having a uniform way of transferring data between their information network silo's ensures privacy-respecting ways that minimize the risk for the consumer (which they claim to be protecting) and the liberalization of the data economy means they can in the long term focus on their comparative advantage with the same data.

What if No One is Right?

Steve Greenberg, lead author of the first Portability Policy and self-described "product manager nonpareil", pointed out that while Google may appear to hold the moral high-ground, neither company comes out ahead in the end.

There is no "good guy" here.  To me, it looks like Google wanted to pick a fight with Facebook and used data portability as an excuse.  This is a shame because, from a portability standpoint, Google had been among the best in the industry.  I would ask both Google and Facebook to allow people to move their data around freely.

Why do I doubt Google's sincerity, especially since they've been friendly in the past?  Unless they were just looking for a pretense to shut off portability, it's hard to see what they hope to get out of this.  Everyone who might care already knows that Facebook won't let you export data.  Why punish your users just to get back at Facebook?

What's Next?

According to Bizannes, companies can only survive by being competitive and, in this case, that comes in the form of data portability.

Maler, meanwhile, cites the User-Managed Access group she hosts, saying that it "is creating opportunities for users to offer novel 'personal data products' in a way that increases their ability to dictate requirements for privacy and data portability."

Saad reminds us that data portability isn't just for show and "'Having an API' is not openness. Having the right terms of service and an INTEROPERABLE data format and protocol is openness."

Maybe it's battles like this, between the Internet giants, that will settle the debate. Perhaps folks like Maler, Bizannes, Saad and Greenberg will finally convince them that openness is in their best interest. Or maybe, one day, the users will demand a way to do what they want with their data.

Now, Google has retaliated by asking any user that gets that far if you are "super sure you want to import your contact information for your friends into a service that won't let you get it out?" Take a look at Google's rather hilarious response after the jump.

(Click on image for full size view)

This is the page that Facebook users now see when they try to export their Google data to find their friends on Facebook. "You have been directed to this page from a site that doesn't allow you to re-export your data to other services, essentially locking up your contact data about your friends," warns Google. "So once you import your data there, you won't be able to get it out." The page even offers users the ability to "register a complaint", although it's unclear where this complaint will end up.

This is just the latest move in a week-long skirmish between the two companies. If you want to fully catch up on the whole affair, you can read up on Techmeme. For now, it looks like the ball is now in Facebook's court, as Google has tried to take the moral high ground by continuing to allow the export of users' data. That is, of course, with a bit of a "don't say I didn't warn you" before they click the button.

Here is the full text of Google's warning:

Hold on a second. Are you super sure you want to import your contact information for your friends into a service that won't let you get it out?

Here's the not-so-fine print. You have been directed to this page from a site that doesn't allow you to re-export your data to other services, essentially locking up your contact data about your friends. So once you import your data there, you won't be able to get it out. We think this is an important thing for you to know before you import your data there. Although we strongly disagree with this data protectionism, the choice is yours. Because, after all, you should have control over your data.

Of course, you are always free to download your contacts using the export feature in Google Contacts.

This public service announcement is brought to you on behalf of your friends in Google Contacts.

[__] Register a complaint over data protectionism. (Google will not record or display your name or email address.)

[__] Proceed with exporting this data. I recognize that once it's been imported to another service, that service may not allow me to export it back out.

Select one or more options. Cancel and go back

While the move is a significant step in the right direction for the social network and its users, a few significant players in the data portability discussion are here to remind us that we aren't there quite yet.

Data portability is the idea that users are, and should be, in control of their data, how its used, and have access to it at any time. Beyond this, data portability inherently implies data interoperability-- the ability for your identity and social graph data to be used across any site or service, as controlled by the end user, and therefore requires the use of open web standards. [...] Facebook continues to maintain, under their TOS, the last word on your data usage through an all-encompassing license to do what they wish with your data (including sub-license it to other entities).

Leonard goes on to point out that, while you may now be able to more easily download your data to manipulate separately, it will still remain on Facebook's servers. You cannot remove it.

The Electronic Frontier Foundation similarly lauded and chastised Facebook's efforts this morning, saying that the feature brought Facebook closer in line with its Bill of Privacy Rights for Social Networking. The EFF offered a series of recommendations for Facebook to make it easier for its users to switch services if they so desired. Part of those recommendations included the ability to export more of your social connections and contact information, the ability for users to opt-out of allowing their contact information to be exported, and even the possibility of contact information available as a separate file.

Both the EFF and Leonard seem to agree that Facebook has made a big step to data accessibility, but have yet another step (or two, or 10) to get to "data portability". Leonard urged the site to adopt the DataPortability Project's Portability Policy, which she calls "the first true step towards data portability, transparency and end user control."

So, for the new feature, we have a resounding "here, here!", but we're reserving the real applause for the day when users can actually download their data and delete it off of Facebook's servers.

As the one of the people who popularized that concept in relation to social networks, and as a founding member of the organization representing that cause, I'd like to call bullshit on that.

Until now I have stayed largely silent on the privacy hoopla because data portability and the open Web are not strictly related to privacy - at least in the sense that things don't need to be public for them to be portable or interoperable.

For example, just because the Web is based on open technologies (HTTP, HTML, SSL, JavaScript, etc.), it does not mean using your credit card on a properly configured website is public or unsafe. Sending email from one person to another does not mean third party websites can now suddenly "instantly personalize" their recommendations to you based on keywords found in your inbox.

Despite being based on interoperable technologies, these transactions remain private and secure.

Advocating Open Technologies Is Not Promoting the Death of Secrets

In the face of this, however, Mark Zuckerberg and Facebook continue to (deliberately?) confuse the idea of open technologies with "sharing in public." The attempt to correlate the two things is at best misinformed and at worst dishonest.

With his latest statement, Zuckerberg and Facebook are now going so far as to declare their privacy missteps as "data portability." Actually, Facebook's changes have nothing to do with data portability. In fact, the root of the user backlash has nothing to do with what the company is doing but rather how its are doing it.

Its problem is that, as a service, Facebook started as a place for people to share with friends and family in a private setting. Users expected privacy. This expectation is referred to as a "social compact." It is an implied agreement that has less to do with the terms of service and more to do with user expectations and ethics. When I give you my business card, for example, I expect (through our implied social compact) that you won't give it to spammers.

It turns out, however, that this compact was good for users but not great for Facebook's business. There are two broad reasons why Facebook has felt forced to make the service more public.

Mark Zuckerberg Facebook SXSWi 2008. Photo by deneyterrio.

First, it's hard, if not impossible, to monetize private communication. People don't use those kinds of service with the intent to buy, but rather with the intent to communicate. Intention is critical when it comes to advertising and e-commerce.

Second, competition from services like Twitter have made it cool to be public, and it's finding interesting ways to monetize this public information (the least of which is selling its inventory of Tweets for $15 million a pop).

Most of Facebook's very mainstream users, however, still just want a private place to keep up with their friends and family. In short, the economic interests of the service are not in line with the interests of its users. Despite this, Facebook has been forced to smashed big cracks in its privacy blanket and started forcing its users, en mass, to adopt more transparent and public online personas.

This (now public) data can be used by advertisers, publishers and other third parties to help Facebook attract even more users, more data and ultimately more dollars through targeted ads and micro-transactions.

The Wrong Social Compact

The problem, then, is not Mark Zuckerberg's stated goal of making the world a more open (read, less private) place, but rather that Facebook did not initially establish the right social compact - promise - to its users to justify its role in this vision of public sharing.

As a result, users feel (rightly) violated. Facebook broke its promise for business purposes. And this is not the first - or last I suspect - time it will do it. (Remember Beacon?)

Finally - in regards to actual data portability, interoperability and the Web - the technology choices Facebook makes are anything but open. It uses proprietary technologies, protocols and formats to capture value from the Web and lock it up in its hub.

In short, nothing about its cultural or technological approach is open or interoperable; it has nothing to do with interoperable data portability - the only kind that matters.

Facebook has every right to do whatever it likes with its service. The market will decide if it continues to like the service or not. Any backlash from the media, or demands for more fairness, are largely irrelevant unless users vote with their feet and stop using the service. Facebook knows this is unlikely, though, given its deep (and growing) integration with the rest of the Web.

But claiming that users love the changes because more and more of them are stumbling into the service by way of widgets on publisher pages is dishonest. There is a real fear amongst the user base (and their partners) about these changes.

When it comes right down to it, the lack of honesty and clarity from the company and its representatives about these issues, and the continued trend of taking established language - such as "open technology" or "data portability" - and corrupting it for its own marketing purposes, is far more disconcerting than the boundaries it's pushing with its technology choices.

What Are the Next Steps?

We as responsible members of the technology community and the open web must be clear and honest about what we see - and any threat it might pose to our industry or the wider world. While jumping on the bandwagon might be fun and easy (and even profitable), it is a abjection of our own responsibilities.

So what can Facebook do in the face of this criticism and push-back?

• Declare clearly and unequivocally that its service has changed from a private place for sharing to a tool for public publishing.
• Go beyond what it has already done to correct the issue and provide a giant status indicator on the top right of a user's profile page indicating if they are in one of three modes: Public, Private, or Friends and Family only.
• Alternatively, (although highly unlikely) it can change its business model from one based on ads and publishers and to one that's based on charging users for pro services in order to align its economic interest with those of its users.

What can others do to protect their privacy or capitalize on Facebook's faults?

• Right now: Recognize that Facebook has violated user trust over and over for the sake of its business model, and will do it again. Stop sharing private information with the service.
• Short term: Create a properly private sharing network where people can feel safe to be with their friends and family.
• Medium term: Recognize (or decide to ensure) that Facebook is only one service, and in order to maintain and encourage competition and respect in the marketplace, other smaller (and not-so-small) players must be supported when making technology decisions (i.e. publishers must choose cross-platform tools and technologies).
• Long term: Continue to create an open alternative to Facebook whereby the Web is the platform and users can choose the applications that make sense for them, which includes privacy.
• Forever: Understand the difference between an "interoperable, open Web" and "Death of Privacy" - they are not the same thing.

Next week The DataPortability Project will be announcing a new initiative that will improve communications between Web services and users - stay tuned.

Standardized legal documents for technical specifications may not seem like the sexiest thing in the the world - but this is actually pretty exciting news. Developments like this could be a key part of the foundation that online service providers need to move forward on a long list of great ideas for ways to serve their users.

People come up with crazy ideas for making the web work better all the time. This agreement aims to provide an easy way to make it safe to implement those ideas. The companies participating have spent large amounts of time and money negotiating the agreement, now anyone can take advantage of the fruits of that labor at no cost.

Existing specifications that will be placed under the Open Web Foundation Agreement, per the announcement today, include:

• Syndicated media delivery spec Media RSS (currently controlled by Yahoo!)


• Secure 3rd party authentication spec OAuth Core and Wrap (from Facebook, Google, Yahoo! and Microsoft)


• Real-time feed protocol PubSubHubbub (Google)


• Comment aggregation protocol Salmon (Google)


• Web Slice Format (Microsoft)


• And several others.

Users are required to give TrueSwitch (through a Gmail interface) the username and password for the old account, then import can take a few hours or days. I pulled in contacts from an old Hotmail account and am now waiting to have them arrive in my Gmail contacts list.

Dear Internet, please offer features like this at every website. The ability to pull in contacts and data from one service provider to another is the dream of data portability. It enables users to try new services, prevents them from being locked-in to old ones, promotes competition between service providers and generally makes the world a better place.

Not all Gmail users can see the new Import feature but over the next few weeks that will probably change.

As of today, several bloggers have reported seeing this new feature, which allows users to grab all their Google Docs and batch export them as a zip file. Files can be exported in a number of formats, including Microsoft Office and Open Office formats. Users can also choose to export only certain types of docs, e.g., spreadsheets and slide decks only.

If a user is particularly pressed for time or has a larger chunk of data to export, he can also choose to navigate away and receive an email when the export is finished.

Unfortunately, any folders a user may have created or data related to authorship or shared documents do not seem to be included once documents are exported.

Called the Data Liberation Front, the project team said in a Google blog post today that it has "liberated" more than half of the major Google services. "In the upcoming months," writes project lead Brian Fitzpatrick, "we also plan to liberate Google Sites and Google Docs (batch-export)."

Making sure the door isn't locked if users choose to leave a service is a required, if less exciting, part of the data portability movement. Just as important as a bulk dump of user data is the option for users to easily and securely port data online from service to service for immediate personalization based on past activity at a legacy site. Google is a market leader in that kind of data portability as well.

The information on DataLiberation.org does not include instructions for deleting your data from Google's servers. The project is taking suggestions for acts of liberation on a Google Moderator page and is publishing updates on Twitter.

Much like the original premise of Ning, Social Beans simplifies the creation of community websites.  However, since it is a portable format, a Social Beans site is not locked in to a single provider. In addition to the Grou.ps platform, the 0.1 version works with MediaWiki and WordPress. A Drupal plugin is also expected for October 2009. 

At this point, Social Beans is extremely experimental and while it's an interesting concept, the group's fate lies in 2 simple questions: Is it an easy enough template for non-technical users to adopt it? And perhaps more importantly, will developers build engines to run it? Let us know your thoughts in the comments below.

Author and consultant Doc Searls writes in a post memorializing Givotovsky that "Every encounter with Nick was engaging and mind-sharpening." London entrepreneur, Ian Henderson, offers the following quote from Givotovsky, exemplifying his contribution to the digital rights conversation.

There are indeed technologists fully qualified to architect the infrastructure to enable a better, more equitable, reciprocal, transparent and accountable digital realm, and they have to a large extent already built the tools and system. Now, the application of that prospective infrastructure to systems and services with the potential to change "the digital deal" from the user-centric perspective is what's needed, and I hope, what's next.

Going forward, the formulation, creation and assertion of binding identity rights agreements in the context of "leverage", that in turn drives change enabled in the market by market forces, is the most pragmatic, short path to something better than a-shrug-a-click-and-a-sigh privacy statements.

It's exactly the implementation of such use cases to which I think the most beneficial and productive (though not always the most immediately profitable) effort can, and should be devoted. We all need a better, fairer, more accountable and credible digital deal. If we are to be "digital citizens" should we not also know the real "digital deal"?

Givotovsky leaves behind a wife and two children.

At some point in conversation Hammer-Lahav realized that the problem went far beyond the Twitter implementation. The OAuth protocol had an inherent vulnerability; big companies like Google, Netflix and Yahoo had implemented OAuth and scores of tiny startups had too.

The Nature of The Problem

The problem was a vulnerability to something called a "Session Fixation Attack." The gist of it is this. Services supporting OAuth let their users pull data into other websites for reuse around the web. In order to do this securely, the 3rd party site has to ask the original site for permission. This might be a new little website asking permission to import your Gmail contacts or to post to Twitter through their site instead of Twitter.com. OAuth was born from the work that Flickr did to create a secure way that other applications could be granted permission to access your photos for printing, editing or posting elsewhere.

The problem arose if an attacker could convince you to complete their request for account permission with your login. At the end of the process they would have access to your account.

Hammer-Lahav explains how this works in detail and offers flow charts in his blog post explaining the technical nature of the problem. For another explanation of this kind of attack see Mitja Kolsek's paper titled Session Fixation Vulnerability in
Web-based Applications (PDF), which was published in 2002. In other words, this is not a new problem - it was just newly discovered to be an OAuth vulnerability.

How It All Went Down

Eran Hammer-Lahav was at FOOCamp when he realized this was a problem that extended far beyond Twitter's implementation. All 30 companies currently offering OAuth were vulnerable. MySpace, Yammer, PhotoBucket. Google, Netflix, Yahoo. Millions of peoples' accounts were at some risk.

If OAuth was software, a fix could be implemented and pushed out to everyone who was using it. But it's not, it's just a standard-based specification implemented out in the wild and no one party is in charge of it. Someone had to do something though, and they had to do it fast.

The first thing Hammer-Lahav decided to do was call up Alex Payne, API lead at Twitter. Though Twitter had done everything right, it was a particular Twitter implementation that revealed the whole problem and it had only been out for a few days. (We thought it a big enough deal that we wrote a whole post about that implementation.)

Twitter shut down the OAuth option for login within 30 seconds of his phone call, Hammer-Lahav says. They did it without explanation, because they were asked to keep quiet about the security problem for one week - in order for all the providers to get a chance to respond before the security problem went public and could be exploited.

Developers cried out that Twitter was shutting down technology essential to their business without warning - and not for the first time. Robin Wauters wrote a post on TechCrunch channeling developer anger over the shut-off. (Lest we imply too much criticism we'll note that we've written very similar stories ourselves.)

Twitter was widely criticized - and they kept their mouth shut, saying only that it was a temporary problem that would soon be resolved. "I can't stress enough how noble Twitter's behavior was yesterday," Hammer-Lahav told us. "Twitter bashing is a sport now and it's a sport that sells ads. Techcrunch wasn't aware of the security threat but it put Twitter in a position where if they were going to talk about it then they would put other companies at risk. We told Twitter that it was going to go public so do your own PR management and they did a good job. The emails sent by other providers to Twitter thanking them for taking that hit have been amazing."

After contacting Twitter, Hammer-Lahav started emailing all 30 companies listed as OAuth providers with Chris Messina's help. Half of them had representatives at FOOCamp, the event he was calling from. He explained the problem to them as he was able to reach people and asked them not to discuss it until next Thursday, one week later. He knew it would be a difficult secret to keep with so many parties involved, including the frustrated developers trying access all of those companies' OAuth APIs.

"At first it took me half an hour to explain the problem," he says. "By the next day I had the explanation down to 30 seconds." Within 12 hours the group discussing the problem knew there was no simple solution - it could require changes by OAuth providers and outside applications that consume OAuth permission in order for everything working again.

The group of OAuth providers formed an email list to discuss the problem and fifty people from 30 companies joined in. Deciding to focus on communicating with the initial service providers was a decision that had to be made. "You have to triage the parties involved," Hammer-Lahav says. Providers needed extra time to deal with the problem because they couldn't just plug the hole or pull the plug easily; FireEagle, for example, only has an OAuth API - there's no other way for the service to function.

OAuth is being advanced by a decentralized community of developers and other parties, but Eran Hammer-Lahav has been its most visible advocate. He's gained years of experience in the trenches fighting for a variety of open standards. He talked to every OAuth provider on the list and volunteered to act as the Community Threat Response Contact. Yahoo, his employer, told him to take as much time and do whatever he needed to deal with the problem. The company put Allen Tom in charge of Yahoo's response and donated Hammer-Lahav's paid time to the community effort. "If I was working for a different company this might not have been possible," he says. "Yahoo! had a whole team of people managing their own response to the situation."

All thirty companies sprung into action to neutralize the security risk and prepare their respective technical responses. Mashery co-founder Clay Loveless and team pushed back other work to pull all nighters and others pitched in as well. Everyone was an equal participant in working together, from single person startups to multibillion dollar companies. "Yahoo and Google put engineers on the line helping people with small startups to review solutions they were going to deploy," Hammer-Lahav says. "Usually the big guys figure it out amongst themselves and leave everyone else to their own devices. This felt like a real community. There was no liability because it was casual advice. Security people are expensive. Some startups don't even have in-house engineers, they are entirely outsourced."

One by one many of the providers shut down their APIs and one by one they implemented solutions.

By Wednesday, one day before the self-imposed period of silence was over, there had to be a lot of pressure built up behind the scenes. Alex Payne, the man in charge of the Twitter API and a guy who is much less grumpy than you'd probably be if you had his job, started getting visibly frustrated. "The view from under this bus is really something," he said on Twitter. "Nobody in the tech press has bothered to contact me for comment on the OAuth issue. Why bother with facts when speculation drives clicks?"

Just after noon on Wednesday, CNet's Caroline McCarthy reported that Twitter and others had pulled OAuth support because of a security problem in the spec. "In the interest of online safety," she wrote, "CNET News has chosen not to make the details of the security hole public." McCarthy was at FOOCamp as well and may have heard about the security issue then, but decided to more or less respect the wishes of the developer community and hold off writing about the issue at all until just before the deadline lifted. If that was the case then she both won accolades from involved parties for her discretion and got a lot of pageviews for jumping deftly on the story after the threat had mostly passed but before others wrote about it.

Minutes after the real story was out, Twitter posted about it on the company blog. Then the official OAuth blog posted about it, linking to McCarthy's post and publicly thanking Twitter for taking all the heat for days. Chris Messina worked fast to update the site and co-ordinate the community response. Then API service provider Mashery, the company that powers OAuth APIs for Netflix and many other companies, posted about it on its blog, assuring customers that the problem was small and under control and thanking Twitter as well. Finally Dave Winer, a web forefather and hardcore Twitter critic, made a post on his blog urging people to lay off Twitter and appreciate the way they were communicating with people about a number of intersecting and difficult technical problems.

One day later, one week after the community responding to the OAuth threat called for a week of silence to come up with a solution - Twitter announced that its OAuth API was back.

That was yesterday and by today almost all of the 30 OAuth providers have OAuth back up and running. There are two different long-term solutions in the works that are being debated on the email list as we speak. Hammer-Lahav says he expects a revised draft of the spec will be ready next week.

And that's how a decentralized community solved a security threat in an open identity spec, quickly. One company (Twitter) took a risk at implementing a new technology advocated by an employee of another company (Yahoo's Hammer-Lahav), then an engineer at yet another company found the beginning of the security hole, then news of the whole problem was sent out to contacts on a Wiki, an email list was formed, companies donated their employees' valuable time to aid in the effort, everyone more or less kept their mouths shut (including the unfairly criticized Twitter) and then everyone worked together to find a solution just in time. I think that's a pretty cool story.

Lessons for the Future

Hammer-Lahav took the lead in responding to this crisis and says he did it with the future of crisis response in open web communities in mind. Creating a template now for the future is only so possible, though. "In a year this same approach isn't going to work because too many businesses are going to depend on the providers," he says. "If we don't find a way to deal with this in the future then companies will remain very cautious about relying on multiple data sources." He says that people want to create a database listing all the parties involved in technologies like this, but prioritizing who gets talked to first will depend on the nature of the threat.

Finally, Hammer-Lahav says that more companies need to empower more employees to step up and take leadership in this kind of situation. The combination of technical, people and process skills is rare but those people need to be found. "It's not sufficient to have only Chris Messina and I as the two people who can do this," he told us. "We need other companies to step up and say there are people in their organization that can support the community. Yahoo said 'you're going to go do this for the community for as long as it takes,' Yahoo was paying me to manage the community threat in a way that was not purely in their self interest."

Can open communities advocating for an open web respond quickly and effectively to inevitable security issues? It sounds easier said than done, but for now we've got at least one very interesting story that says it is possible.