Ideas are rarely created through a "eureka" moment. It may seem like Doc Brown fell off his toilet and invented the flux capacitor, but really the idea for time travel and how to do it were converging in his brain for quite some time before the blow tothe head. Instead of an "aha!" moment, Johnson believes that ideas are born of a "slow hunch" that are made possible through periods of technological innovation and evolution. If you are creating a startup, where do you get your ideas from?

That sounds familiar - taking data and mapping it to suit your mission and illuminate theories or facts. Essentially, anybody can do that now with GIS software and an .xml data set. The Centers for Disease Control use mapping to determine outbreaks and recently Google released its Google Flu Trends that accumulates flu data throughout the season. Yet, it was not Snow's own genius that came up with the solution to the cholera outbreak (disputed at the time) but rather a set of groups in various sectors using their collective intelligence to solve a problem.

The Hive Mind & Collective Intelligence

"It is just this idea that if you diversify and have an electric range of interests and you are constantly getting interesting stories about things that you do not know that much about or are adjacent to your particular field of expertise you are much more likely to come up with innovative ideas," Johnson told ReadWriteWeb.

The same approach would work well for developers and innovators working on the next technology breakthrough. Startup founders should take step back from their project and ask what type of similar projects have been undertaken in a completely different field and see if those lessons can be applied to their project.

"The trick is to look at something different and borrow ideas. It is like saying 'this worked for that field, if we put it here what would it do in this new context?'" Johnson said.

In today's world, the ability to branch out of your field of expertise has been made much easier through social media. You can follow what is happening in your niche through a specifically created Twitter list, but it is also beneficial to create lists of people working in different sectors as well.

"The important thing is that this is not some kind of hive-mind wisdom of the crowds, collective intelligence network smarts," Johnson said. "The unit is still the individual or the small group. There are some examples of group intelligence. This is an example instead of taking individuals in small groups and making them smarter by connecting them to a wider range of influences."

Data visualization initiatives fared well at this year's challenge, as a second entry, Tilemapping, was granted $74,000 from the foundation. Other popular categories included projects aimed at finding new ways to engage readers, as well as those geared toward determining new methods for funding journalism at the local level (see video below). The foundation hopes that by funding these ideas they will help spur innovation and aid digital journalism take its first strides in the Internet age.

"The free flow of shared information is essential for communities to function in a democracy. More each day, that information flows through and because of digital technology," said Knight Foundation President Alberto Ibargüen. "Until someone figures out the next big thing [...] we can use the Knight News Challenge to experiment with ways to learn how to think in different ways about information sharing so we might discover the future of news."

The announcement of this year's winners was made at the Future of News and Civic Media conference at MIT - an event featuring previous challenge winners. Since beginning four years ago, the challenge has received over 10,000 applications and has doled out $23 million to 50 winning projects. Possibly the most well known past Knight News Challenge winner is Spot.us - a platform for community funded journalism.

Today is the 3rd annual international Data Privacy Day and a whole bunch of companies are listed on the organization's website as participants. Google, Microsoft, even Walmart. Facebook is not listed as a participant and has stirred up a lot of controversy with changes to its privacy policy lately.

Why are these corporations singing out loud about protecting our personal privacy? According to the website, "Data Privacy Day is an international celebration of the dignity of the individual expressed through personal information." More than dignity, this is about building trust with consumers so that these companies can do things with our personal data. Some of those are things we might like, a lot. Aggregate data analysis and personal recommendation could be the foundation of the next step of the internet. Unfortunately, Facebook's recent privacy policy changes put that future at risk by burning the trust of hundreds of millions of mainstream users.

Had Facebook opened up access to user data through users' consent - then access to that data would be a whole different story. As is, the privacy change was unclear and pushed-through without user choice concerning some key data, putting the whole concept of users sharing their data at risk.

See also: Facebook's round-up of other peoples' statements about privacy today on its blog.

How Facebook Changed

This past December, Facebook did an about-face on privacy. (Here's our extensive coverage of the changes and why they were made.) For years the company had based its core relationship with users on protecting their privacy, making sure the information they posted could only be viewed by trusted friends. Privacy control "is the vector around which Facebook operates," Zuckerberg told me in an interview two years ago. 350 million people around the world signed up for that system.

Facebook's obsession with privacy slowed down the work of people who wanted to build cool new features or find important social patterns on top of all the connections we users make between people, places and things on the site. (Marshall shared a link to The San Francisco Giants with Alex, for example.)

Those geeky cries in the wilderness to set the data free, for users to be allowed to take their data with them ("data portability") from one website to another? Not going to happen at Facebook, founder Mark Zuckerberg said, due to privacy concerns.

Aggregate data analysis? Facebook as a living census unlike any the world has ever seen? Back off, sociologists, you can't access aggregate Facebook user data due to...privacy concerns, the company said. Facebook staff did team with a few outside academics they knew and studied that Facebook data themselves. They published some charts about racial demographics on Facebook, concluding that everything was peachy-keen and only getting better on the social network. But if you thought an army of independent analysts could glean some objective insights into the contemporary human condition out of Facebook, you were wrong.

Then in December, things changed. Facebook began prompting users to re-evaluate their privacy settings. Public was the new default and some fields on a user's profile were suddenly and irrevocably made visible to the web at large. Your photo, your list of friends and your interests as expressed through fan page subscriptions could no longer be set to private.

Sorry, 350 million people who signed up for the old system. When Facebook said in the fine print that it reserved the right to change its policies, the company really meant it.

The changes were responded to with an international wave of confusion and indignation. News stories were written all around the world about Facebook's privacy changes - they're still being written today. Yesterday the Canadian government announced it was launching its second investigation in six months into Facebook's privacy policies.

Did Facebook Break the Future of the Internet?

Is it naive to think that things you post on the internet are really "private?" Many people say it is, but that was core to the value proposition that Facebook grew up on.

Presumably the companies working together on International Data Privacy Day don't believe that privacy online is a lost cause.

In fact, trusting that your private data will remain private could be a key requirement for everyday, mainstream users to be willing to input all the more of their personal data into systems that would build value on top of that data.

Facebook is the first system ever that allowed hundreds of millions of people around the world to input information about their most personal interests, no matter how minor.

Will that information serve as a platform for developers to build applications and for social observers to tell us things about ourselves that we never could have seen without a bird's eye view? That would be far more likely if more people trusted the systems they input their data into.

Think of Mint's analysis of your spending habits over time. Think of Amazon's product recommendations. Think of Facebook's friend recommender. Think of the mashup between US census data and mortgage loan data that exposed the racist practice of real-estate Redlining in the last century.

Personal recommendations and the other side of that same coin - large-scale understanding of social patterns - could be the trend that defines the next era of the internet just like easy publishing of content has defined this era.

Imagine this kind of future:

You say: "Dear iPad (or whatever), I'm considering inviting Jane to lunch at The Observatory on Thursday, what can you tell me about that? Give me the widest scope of information possible."

Then your Web 3.0-enabled iPad (or whatever) says to you: "Jane has not eaten Sushi in the past 6 weeks but has 2 times in the year so far. [Location data] The average calorie count of a lunch meal at that location is 250 calories, which would put you below your daily goal. [Nutrition data online.] Please note that there is a landmark within 100 yards of The Observatory for which the Wikipedia page is tagged with 3 keywords that match your recent newspaper reading interest-list and 4 of Jane's. Furthermore...

"People who like sushi and that landmark also tend to like the movie showing at a theatre down the street. Since you have race and class demographics turned on, though, I can also tell you that college-educated black people tend to give that director's movies unusually bad reviews. Click here to learn more."

That's what the future of the internet could look like. That sounds great to me. Think that vision of the future sounds crazy? How long ago was it that it sounded crazy to think a day would come when you typed little notes into your computer about how you felt and all your friends and family saw them?

But how many people will trust this new class of systems enough to contribute meaningfully to them, now that they've been burned by Facebook?

On International Data Privacy Day, it's good to consider the possible implications of Facebook's actions not just on users in the short term, but on the larger ecosystem of online development and innovation over time.

Photo: Mark Zuckerberg, by Andrew Feinberg

For the quarter ending this past June, we compared the findings of three research firms that reported on the money invested in Q2:

• July 21, MoneyTree (PricewaterhouseCoopers, with data from the National Venture Capital Association and Thomson Reuters): $3.7 billion, with 612 deals,
• July 18, VentureSource (DowJones): $5.27 billion, with 595 deals,
• July 14, ChubbyBrain (a New York City-based startup partnering with ReadWriteWeb): $5.329 billion, with 613 deals.

VentureSource and ChubbyBrain seem to agree on the top line number. But MoneyTree's number is what most people report, and that is about $1.5 billion different.

As the old saying goes, "A billion here, a billion there. Sooner or later it adds up."

Disclosure: Our VC Funding Report

ReadWriteWeb has an interest in this. We sell a report for $299 that has details on the 240 deals done this quarter in the Internet, mobile, and SaaS space (not clean tech or bio tech), and this is powered by data from ChubbyBrain. So we are biased. But it also means that we are engaged and have been looking at this fairly deeply.

Who Cares?

We also think that accuracy matters, and we are trying to figure whom accuracy matters to. We see three main types of participants in the industry:

1. VCs. They need accurate data for their own fund-raising. They have to be able to benchmark their own funds relative to the broader market.
2. Entrepreneurs. Data on what funding deals are being made, and why, helps them figure out how much to raise, when, and from which VC.
3. The startup "community." This is a catch-all for everyone else, who tend to align to either VCs or entrepreneurs. Journalists, the non-aligned fourth estate, want reliable data to key off interesting stories.

Why does this matter? The startup community matters to the health of the overall economy. As the National Venture Capital Association (NVCA, the trade association of VCs) likes to point out:

"Originally, venture-backed companies have created companies that accounted for 10.4 million jobs and over $2.3 trillion in revenue (based on 2006 data)."

So a headline like "VC Investments Falling Off Cliff in the US" really impacts a lot of people. That is the kind of headline that most journalists/bloggers wrote in April 2009, based on data reported by those trusted sources.

We wrote a really boring headline:

"VC Investment in Internet Deals Did Not Fall Off a Cliff."

That's a lousy headline for generating page views. It's a story about "the dog that did not bark."

The point is that headlines drive business behavior to wild excesses on both the down-cycle bust and the up-cycle boom.

Just good reliable data would help.

Innovation Is Global, But It Keys Off US Data

At ReadWriteWeb, we love to track innovation from far-flung corners of the world, and we see the globalization of innovation as a critical trend.

So we want to be able to report on financing trends for early-stage Web technology startups across Europe and Asia, in addition to the US. And we expect any research process to be able to scale to that challenge.

But the reality today is that, globally, entrepreneurs and VCs key off US data. If they were to key off bad data, that would matter to everyone.

Why This Matters

Driving with one's eyes in the rear-view mirror is dangerous. We take action based on what authoritative sources tell us is happening today, and we base our assumptions on what that means will happen next and plan accordingly.

In reality, these sources tell us what has happened in the past, and they may not even tell us that accurately.

When we at ReadWriteWeb look at the macro picture, we favor a contrarian view simply because the reality we see today is often not what the headlines trumpet. When the markets were in the late stage of a boom, we were sounding the warning signals.

When the markets were melting, we began to see surprising signs of life in the early-stage Web technology world we live in.

Whether you are an entrepreneur or an investor, knowing what the crowd is thinking -- and what the headlines are trumpeting -- is valuable. Even more valuable are the underlying facts and trends that may be missing from those headlines. In the disconnect between the two often lies a lot of opportunity.

We hope to ignite a debate that leads to greater accuracy and transparency of these numbers.

Steve Jurvetson, one of the best-known technology investors in the world, has posted an updated version of Kurzweil's visualization of the history of exponential growth in computing. In his thought provoking discussion of the phenomenon, Jurvetson calls this "the most important chart in technology business."

"What Moore observed in the belly of the early IC [integrated circuit] industry was a derivative metric," Jurvetson writes, "a refracted signal, from the bigger trend, the trend that begs various philosophical questions and predicts mind-bending futures."

What will be the next big leap in computing technology? From a hardware perspective I won't even venture to guess, but it sure is exciting to think about what those technologies will enable. Many things we'd love to see done today remain too processor intensive, from advanced Augmented Reality implementations to broader real time services and machine learning. Ultimately, we can only make clumsy guesses about the killer apps of an exponentially more powerful computing platform - that's the point.

When calculations per second increase exponentially, fundamentally new things become possible. It's one of those fascinating quantitative changes that cross the threshold of creating qualitative change.

For every frontier explored there is baggage brought from home, of course. About this chart, Union Square Ventures associate Andrew Parker asks "how much does a newer medium pull from an older medium in terms of design paradigms?" (That makes us want to cry for the poor limping US Patent Office, now facing layoffs of all things!)

Mind bending is right. What does this chart, that Jurvetson calls the most important in the technology business, bring to your mind - innovation loving readers of ReadWriteWeb?

You're not alone. And maybe they're right in thinking we have our priorities out of whack... but in the great scheme of things, there are worse things to lose your sense of proportion over than innovation, new possibilities and sheer fun.

More Noise to Signal

Note: If you want to live on the cutting edge, here are the instructions for enabling the Chrome Beta and Developer channels.

Some Highlights

Here are some of our favorite apps in the current Google Chrome Experiments line-up:

Social Collider

Social Collider might just be one of the coolest Twitter visualization tools we have seen in the recent past. Social Collider shows the connections between different Twitter users. You can use a user name or keyword to initiate Social Collider, but it can also be used to visualize current Twitter trends.

Note: Using Social Collider can be quite CPU intensive, but the results are definitely worth it.

Google Gravity

This is an utterly useless experiment, but it shows off some of the surprising possibilities of using JavaScript together with the Box2D Physics Engine. After you have seen gravity take its toll on the Google homepage, also try to perform some searches.

Smalltalk

Smalltalk is another Chrome Experiment that uses the Twitter API to visualize real-time chatter on the Internet. Specifically, Smalltalk looks at comments about the weather in the US (sunny, foggy, windy, etc.). Besides JavaScript, Smalltalk also makes use of the canvas element in HTML5 and the jQuery framework.

BallDroppings

Josh Nimoy's BallDroppings is a cool little musical toy that has already been implemented in a number of other languages. Here is the JavaScript version. Just draw a few lines on the screen and see what happens.

As avid users of FriendFeed will tell you, Facebook's implementation of FriendFeed's features are nothing but a pale imitation of the real thing. Still, there's a growing concern among the service's fans about its sustainability. Although FriendFeed's founders believe they can still innovate to profitability, we're no longer sure that's true.

FriendFeed is a web application that's very much like Facebook's News Feed, except that it incorporates far more services. Where Facebook lets you import content to your News Feed from a dozen social web services that range from YouTube to Flickr, FriendFeed offers nearly sixty..including Facebook status updates. That's not the only difference, either. In FriendFeed, commenting on and "liking" items causes them to "bubble up" to the top - that is, it brings popular content up to the top of the page. FriendFeed's "FOAF" (friend-of-a-friend) feature also integrates posts from your friends' friends into your activity stream which can expose you to more interesting people who you might want to follow.

Although on the surface, FriendFeed might appear to be just a more robust version of the Facebook News Feed - a News Feed on steroids - the differences between the two go far beyond a list of features. Where Facebook users track their real-life friends' activities, FriendFeeders tend to track news and topics they're interested in. Most have probably never even met half the people they're subscribed to - they just like what they have to say and the things they share.

Wait...Doesn't FriendFeed Need to Make Money?

What FriendFeed delivers is something that's more than just the sum of its parts. It doesn't have one single killer feature that defines it. It is simply a mashup of pure innovation. So what if Facebook rips off bits and pieces of FriendFeed's better qualities? Why shouldn't mainstream users enjoy this too? For what's innovation's worth if it doesn't spread?

Ah, but therein lies the root of all FriendFeed's problems. The innovation of the social "like," of aggregating your web activity and letting others comment on it - all of this, all of FriendFeed's innovation, is spreading off-site. It's becoming popularized on Facebook, where a good portion of the social network's users have never heard of FriendFeed and (possibly) never will.

That doesn't bother FriendFeed, though. Says co-founder Bret Taylor:

"The ability to comment on and like entries has always been popular on FriendFeed, so it is not surprising to see it appear in other places. We have always been focused on building a unique, but open sharing and communications product, and we think that it's great when users are able to share things in more places. While there will always be some overlap in functionality between FriendFeed and large social networks, we believe there is a lot of room for FriendFeed to grow. The problems of sharing and communication are large, and we don't think they will be solved by a single product or company."

While that's true to a point - we certainly don't think Facebook will solve all our communication problems either - there is a valid concern that if FriendFeed can't cross over into the mainstream, they may not make it, especially given our current economy. Businesses still need to make money...and for web startups to make money they need users. Yes, more users than web celeb Robert Scoble and his 25,000 followers. Unless FriendFeed can prove to us that they can, without a doubt, monetize the long tail of technology early adopters, then they need to grow their user base. Can they do this? How? These remain unanswered questions as of now.

FriendFeed's Real Value

But don't get us wrong, FriendFeed's financial success (or lack thereof) is only one way to measure its real value. Obviously it's the one that investors and business owners care about. If that describes you - if you only care about the bottom line and all the nickels and dimes - then seeing FriendFeed's features swallowed up by the social giant that is Facebook may be worrisome.

However, if you measure success not by money alone but by pure, unadulterated excitement, the feeling that you've witnessed the birth of something new -something different - then it doesn't matter how many features Facebook steals for their own. All that matters is that innovation happened. It happened on FriendFeed. And you liked it.

ReadWriteWeb will be reporting from the 15th Stanford Accel Symposium on February 25th. Hosted by Accel (the VC firm that funded Facebook, Etsy, and dozens of other companies we write about here) and Media X at Stanford, the event boasts some big name speakers, such as the CEOs of eBay and WPP.

Who Will Speak?

The keynote talks will be delivered by:

• John Donahoe, CEO, eBay Inc.
• Sir Martin Sorrell, Chairman and CEO, WPP

The speakers and panelists include:

• Jeremy Allaire, CEO and Founder, Brightcove, Inc.
• Adam Bain, President of Audience Network, Fox
• Jim Bankoff, CEO, SB Nation
• Matthew Barzun, National Finance Team for Barack Obama and former Chief Strategy Officer, CNET Networks, Inc
• John Battelle, Chairman and CEO, Federated Media
• Rob Bearden, former President, Jboss; President, Springsource
• Dave Berman, President of Worldwide Sales and Service, WebEx
• Tim Cadogan, CEO, OpenX
• Duke Chung, CEO, Parature
• Scott Dietzen, SVP of Communications and Products, Yahoo!
• Curt Hecht, President, Vivaki Nerve Center
• Chris Hughes, Facebook, and Architect of Obama's Digital Campaign Strategies
• Bob Muglia, President Server and Tools Business, Microsoft Corp.
• Raghu Ramakrishnan, Chief Scientist for Audience and Research Fellow at Yahoo, Inc.
• Mark Read, Strategy Director, WPP
• Mike Schroepfer, VP Engineering, Facebook
• Quincy Smith, CEO, CBS Interactive
• David Thompspn, CEO, Genius Inc., former CMO, WebEx
• Jayshree Ullal, President and CEO, Arista Networks
• Zack Urlocker, VP of Products, Sun Microsystems
• Maynard Webb, CEO, LiveOps; former COO, eBay, Inc.
• Jeff Weiner, President, LinkedIn; former EIR, Accel Partners

What Will They Talk About?

Here are just some of the topics these big names will cover:

1. The Obama Administration's Impact on Innovation Sectors
2. Digital Media Advertiser and Technology Priorities in a Downsizing World
3. Trends in Data Center Infrastructure and Cloud Computing
4. Will the Recession Strengthen Open Source or Destroy Profitability?
5. SaaS Crossing the Chasm: Are We There Yet?

What Questions Would You Ask if You Were There?

Tell us in the comments. We will select the best questions and ask them at the conference and then report on the answers. Please be specific: ask a question that relates to one of the above topics; if you want, choose which panelist you would like us to direct your question to.

As the worlds second most popular browser, and with their recent milestone of reaching 20 percent worldwide market share, the folks over at Mozilla certainly have some celebrating to do.

Well done Firefox, you've come a long way; a 329 percent market share increase over four years. So, theoretically, if Firefox continues at the same growth rate, in another four years it will hit 85.67 percent market share.

Interesting.

Isn't this pretty much where IE was when Firefox came onto the scene?

Browser market share December 2004

Browser market share December 2005

Browser market share December 2006

Browser market share December 2007

Browser market share October 2008

Today, we realized that the most popular posts from those blogs may be useful resources for many folk, so we created our own OPML file. Ah, the beauty of the Web.

Chris liked the list; didn't like that it wasn't offered as an OPML file, so he added the feeds and removed those he didn't like - a gift to himself he says, and a gift for you to download and share.

Using AideRSS PostRank, ReadWriteWeb has created an OPML file of the greatest hits from each of those 150 blogs; the top 20 percent of the most popular posts from each site. You can grab it here.

If you're not familiar with OPML, take a look at this OPML primer or read Marshall's how (and why) to create an OPML file.

Earlier this year we compiled a list of ReadWriteWeb OPML resources; take a look if you're interested in what we are reading, or come join us on FriendFeed, where incidentally we first noticed this story, and we can discuss other cool things we find on the Web.

But Twitter won't survive if it doesn't find a great revenue model. This matters to all of us.

If Twitter fails to find a revenue model and hits the deadpool, it will have a chilling effect on innovation. That matters to all of us in the innovation economy. Sure, you could live without Twitter, but what about the funding chances for that brilliant idea of yours?

No Rush But It Has To Be Right

Twitter has top tier VC backing and enough cash to wait till they have scale and the right revenue model. But when they launch it has to be just right. In this climate, Twitter would probably not recover from a Beacon like fiasco. Nor can Twitter do well with a smorgasboard of revenue models, where none really moves the needle and all irritate users just a bit. It has to be one absolutely compelling model that enables Twitter revenue to scale the same way their usage scales.

Adwords Is The Gold Standard, But This Is Tougher

Adwords is the monetization gold standard. The fact that Google did not invent it is irrelevant. But how many new revenue models like this have we seen in the last 10 (or even 100) years? That's right. This is tough to get right.

But Twitter's challenge is even tougher. Google was not creating an entirely new type of service. It was Yet Another Search Engine, just a better one. They could just use/improve a revenue model developed at a different search engine. Twitter has created an entirely new type of medium, which is what makes it so exciting. So there is no obvious place to borrow a revenue model from.

This requires serious creativity. Yes the words "revenue" and "creativity" were just used in the same sentence, welcome to the new world!

Why Did Adwords Work? Two Acid Tests

These are the two things that Twitter's Magic Revenue Model has to achieve that Adwords did so brilliantly:

1. Do not irritate/interrupt the user and even occasionally add value to the user.

2. Provide a value proposition that is so compelling that even conservative buyers give it a try.

Show That The Wisdom Of Experts Works

My horoscope today told me to "pick a controversial topic and ask someone smart what they think about it." Well, I am taking that to extreme and asking lots of smart people what they think.

Wisdom of crowds is "so last cycle". Wisdom of experts is what really works. That is what we have here on ReadWriteWeb, lots of really smart innovation experts. Give us your one best idea for a Twitter Revenue Model and show us how it meets the 2 acid tests defined above.

reCAPTCHA combines traditional OCR with an approach similar to Amazon's Mechanical Turk. Every text is analyzed by two different OCR programs and whenever those two program disagree on a word, it is marked as 'suspicious.' Those suspicious words are then fed into reCAPTCHA, which creates a CAPTCHA with both the suspicious word and a known control word. Once a certain number of users have solved the suspicious word with the same result, it becomes a control word itself.

Overall, reCAPTCHA achieves an accuracy of 99.1%, which is on par with the accuracy achieved by having two humans type the text and then verify the results.

While it is mostly a proof of concept right now, reCAPTCHA's developers calculate that the system can be used to transcribe the equivalent of 160 books a day.

The most fascinating aspect of this idea is that it turns mental energy, which would otherwise be wasted, into something useful. Other projects like fold.it, which turns protein folding into a game, or Google's Image Labeler take a similar approach, but the user has to actively decide to play a game. reCAPTCHA, on the other hand, turns a chore into a useful project.

Dot Coms 2.0? Say it Ain't So...

Maybe we are all just fooling ourselves. When the Internet bubble started to burst in March 2000, most people were saying "that's them crazy Dot Coms, not us". Gradually, it was all of us who were in any way associated with technology.

After 18 months, in the summer of 2002, everybody had capitulated. You could buy shares of Rational Corp (the leading supplier of software tools after Microsoft, bought by IBM) for a valuation of $1 billion when they had $1 billion in the bank and $1 billion in revenues (oh, yes, and a couple of bad quarters which made it obvious that nobody would buy software ever again). You could walk into a VC with a patented machine to turn mud into gold and be greeted with a sceptical "but what if gold falls in value?". You could prove that your $50k software would have $500k savings to a company within 6 months and the response was still "we will get back to you". The technology nuclear winter was very, very cold.

So maybe it is coming again in the technology business. No more funding, no more deals, no more parties. Maybe the hangover is coming.

Innovation Economy Still Thriving

But it does not look that way from what I am seeing. VCs are saying that their companies are doing well (and not all are hyping their portfolio). It also coincides with what I am seeing from companies that I know well. Companies are either growing revenues or getting more funding or doing both. I have interviewed two founders at the Web 2.0 Expo in New York that have broken into profitability. Even if VCs run for the hills they will be fine. These upstarts are taking business from higher cost alternatives.

I have also heard first hand of big deals from large companies awarded to small, young ventures. And I have seen large enterprises that are working on large social media rollouts.

This is not good for big tech companies. But it does look good for small, low cost, agile upstarts. The smart companies have worked out how to reduce risk for clients. What's the risk of implementing Basecamp or Zoho?

I call this the "innovation economy" and that is a tad worrying. It sounds like "New Economy" and we all know where that ended up. I did not want to say the "technology industry" as huge parts of the tech industry now simply follow economic cycles. The fortunes of Microsoft, Oracle, Dell, HP, IBM, Cisco tend to rise and fall in line with global GDP. Bigco is not the heart of the innovation economy.

Shift in Power to Smallcos

It is more likely to do with a fundamental secular shift in power from large business to small business. The Internet and Coase's law would be the theoretical underpinning for that. Something dramatic may have quietly happened, making the playing field not just level for start-ups vs incumbents, but tilted in their favor. It might have something to do with the tools that enable you to run a global business with all virtual operations and almost no infrastructure cost. You can simply scale faster and cheaper than your incumbents.

Conclusion

The market mayhem this week has been unprecedented, far worse than even my worst imaginings and I was thinking it was going to be bad. So this is bad. It will spill over into everything. Many parts of the Web 2.0 industry will be in trouble, specifically those with dependency on consumer advertising or financial services.

But the gritty entrepreneurs are building value and getting profitable and have better opportunities than ever before to get their case heard. VCs who keep their nerve will do enormously well, just as they did from deals done in the 2002/2003 era.

What do you think? Is it tough times for all? Or tough times for the slow and good times for the fast?

Image credit: Thomas Hawk

That's a part of what's going on, economics. This is an information economy, after all, and user data is clearly one of the most important currencies in circulation.

The Importance of Privacy

Facebook holds a growing amount of user data and tries to hold on to it tightly in the name of user privacy. Founding CEO Mark Zuckerberg told us in an interview at SXSW that he agrees with the principals of data portability but believes that Facebook has to solve a number of problems about privacy as its contribution to the data portability discussion. At the time many people were skeptical of Zuckerberg's claim (some still are) - but nothing illustrated the validity of his argument better than Scoblegate.

High profile blogger Robert Scoble teamed up with spamtastic startup Plaxo to scrape the emails from his contacts' profiles in Facebook, turn them from protected images into machine readable text with OCR technology and then export them outside of Facebook. Facebook shut Scoble's account down for a couple of days and a huge brouhaha erupted. Some said that Scoble had every right to the email of his contacts and others said they only gave him a right to see their contact info in Facebook and not to export it elsewhere.

Can users have access to data about their past activities and their social connections without violating the privacy of the people they are connected to? That is the question and no solution has been found yet. There needs to be though and it ought to be a solution that's standards-based and thus reproducible everywhere.

Protection of user privacy is a precondition to meaningful data portability, but vendor control over data is a stopgap measure at most - no more than a short term solution. In the long run, there needs to be a way for users to designate some of their data as being not for export or use outside of its original context. The data that is made available for public sharing should be accessible through a standards-based system of authentication, so any new vendor can show up and make use of it.

Long Term Interests

In the long term, it's in everyone's best interests for data to be as portable as possible. For users, data portability means that we can invest time and resources into new platforms on the web without the fear that the work we create will be locked in to that network or otherwise lost to us. It also offers the possibility that we can take our compiled work in one place and let another service process that data to create new kinds of value for our benefit. For example, being able to export our reading history from one service would enable other services to immediately recommend new experiences they can offer based on our tastes elsewhere. The music website Idiomag, for example, can look at our public history on Last.fm and build from that history a customized music "magazine" about artists we would likely enjoy. That's just one kind of service that could be enabled for users by data portability.

Most importantly for the purposes of this post, vendors too have an interest in data portability. Allowing your users to port their data elsewhere means that you'll be able to import their data from other platforms enabling export as well. When your users take their history with you to another site, they will be able to make faster, better friends and content connections in that new place, which should lead to their having a better developed social network to bring back to your site.

If you can add value to user data, and thus help grow the aggregate information economy, then there should be far more information for you to monetize (advertise against) than you could keep within your grasp alone. Add value, let it go and focus on offering a compelling enough user experience that users will bring their data back to you, freshly grown from their experiences in other environments. Everybody wins.

Google search has huge value not because it owns anything but because it touches everything. The value it ads by enabling discoverability lets the web at large grow, meaning that there's more web for Google to advertise on. Google Friend Connect, on the other hand, keeps users' social and activity data on Google servers - barring the participating websites from gaining read/write access to that data. What a huge loss for everyone!

Recall the economic theory of comparative advantage. International development thinker and all around Renaissance man Jed Sundwall says the old "I'm better at making wine, you're better at making cheese, let's trade" logic could well apply to social web data portability as well. Facebook, for example, has a great video mail system. It doesn't do microblogging well at all. Allowing a user to bring their video mail from Facebook with them to Twitter and their Twitter history with them to Facebook would only make both services stronger. Blogger and economist in a previous life Bob Uva says it's a matter of enabling "greater efficiency by all points of production." There's no need for one service to reinvent the wheel or spend resources building data extraction technology if there is standards based data portability. Likewise, users wouldn't have to start anew building their social networks and personal profile/history in every new service they join. What that increased efficiency means is more innovation, both in terms of service features and personal creativity.

Data Neutrality

Perhaps what we need is not just data portability but data neutrality, a paradigm emphasizing that only users can control where their data passes from one location to another. Just like supporters of net neutrality argue, allowing vendors to limit passage of data allows them to stifle upstart competitors and hurts the whole web's need for continuous innovation.

Good Faith

Privacy is really important in order for data portability to be real. Working to assure that privacy is important, but we also need to see vendors making consistent progress towards a user-centric economy of open data. That means that building iframes and widgets to send the social around the web is just a short term solution. More important news for long term progress would be vendors accepting inbound OpenID, offering oAuth APIs for passing user data along from site to site, marking up user data with semantic and microformat code to make that data machine readable elsewhere, enabling easy behavioral data export and perhaps most importantly building the machinery that will process portable data as the world moves in that direction.

Everyone says they support data portability but it's most exciting to see vendors who are developing methods of deriving value for users and for themselves from the free passage of data that ought to be the defining characteristic of the web, once user control and privacy are workably solved. We want it, you want it, we're all going to get more value out of it if service providers offer a place to put our data and make some magic with it.

As Chris Saad, founder of the Data Portability Working Group said today, data portability is the new web and vendor apps are like the browsers that allow you to view and remix that data. That's where the innovation that will fuel the future growth of the information economy will come from. Users have to have a reasonable expectation that they will be safe, secure and in control of their own data assets - and then it's a competition to see which vendors can add the most value to the free flow of data. In just a few short years, vendors should win the hearts of users by providing superior service - not in any part by locking in data. Let's have heated debates about who's innovating the fastest using the secure, free flow of user data - not about whether that needs to happen at all.

Horse race photo by Ian Ransley. Special thanks to the friends who helped me with this post on Twitter, UStream, the telephone and a wiki - we used each of those media to put this together and it was a lot of fun.