For now, Factual obviously only offers a relatively small repository of databases, though the company's current focus is on getting more developers to use its service and on bringing as much data as possible into the system.

Getting Data into Factual.

To enter data, users could obviously tediously enter the data field by field, or upload spreadsheets in most of the standard formats. The service also provides a number of easier ways to import data. You can, for example, give Factual a URL of any website or Wikipedia page that includes tables and the service will automatically create a new table based on this data. We tried this with tables from a number of sites and it generally worked well and only required a few edits. For advanced users, Factual also includes a number of more advanced extraction tools.

Once the data is available on Factual, developers can obviously use the API to read, write and mash this data up in any form they like. Users can also edit tables directly on the site or through an embedded table. In addition, users can mash up and combine existing tables.

Currently, Factual only offers one relatively basic embeddable widget that can only display the table without any graphical embellishments. The company plans to rely on developers to create other ways to access and display the data available on the service.

Not a Wiki

While Factual allows any user to make changes to the database, Factual's model is slightly different from the standard wiki approach where only the last edit is generally visible to the public. Changes made to a fact in a Factual database are more like votes for a certain entry. If three users or data sources say a restaurant doesn't offer vegetarian food, for example, and one user says it does, then the table will display the fact that the majority of users entered. Factual, however, will also display a question mark next to this disputed entry. Users can click on this question mark to see all the editors and data sources.

Factual will obviously try to weed out spam here as well, though given how new the service is, it's hard to evaluate how effective Factual's spam filters are.

License

Users who enter data into a Factual database do not automatically give up their copyright - though given that Factual focuses on facts, which typically can't be copyrighted anyway, this shouldn't be too much of a problem. Users can, however, choose an open license for their work, which might be necessary if the table they used to seed their database was licensed under a Creative Commons license, for example. Factual's FAQ explains this issue in greater detail.

Would You Use an Open Data Service?

With regards to the question of why businesses would open up their data, Gilad Elbaz told us yesterday that he believes open data could eventually go the way of open source, which also had a hard time to get acceptance among businesses. While open source software is a tool that a lot of companies now use, data is usually what is at the heart of a company's products and it remains to be seen how many companies would really want to put their data into an open database. For now, we mostly expect non-profits and government organizations to make use of this service.

Like the creation of railroads and highways fostered economic development in the past, giving software developers access to a landscape of municipal data could be the beginning of a foundation for a new era of innovation.

The full text of the resolution has been posted as text (from a PDF) on the same blog.

Portland joins San Francisco, Chicago and Vancouver, British Columbia as cities with major initiatives to offer municipal data in formats that will enable independent developers to build new applications leveraging that data. Making municipal data openly available for developers could be the contemporary economic equivalent of paving roads and installing electricity that can be used to open new businesses and better serve the people living in that city.

Portland, Oregon isn't new to tech innovation, of course. It's a place where the city bus system has its own app store, it's home to red-hot mobile development shops like Small Society (built iPhone apps for Starbucks, WholeFoods etc.) and Urban Airship (iPhone push infrastructure) and it's the home of Linux creator Linus Torvalds, wiki inventor Ward Cunningham and one third of the staff of ReadWriteWeb - amongst other geekery.

What could come next? How about more cities getting on board, a national or international standard for municipal data and delivery of that data in real time? One Prefecture in Japan has announced that it will promote the mobile Augmented Reality app Sekai Camera to display historical data about locations in the area. Seeing individual cities move in this direction is a great start.

What US city will move in favor of open source and open, structured data next? Seattle? New York? Someplace in the Mid West? Place your bets now as these are unlikely to be isolated developments.

Photo: "Max" Creative Commons by Stu Seeger.

Web Apps

Google App Engine: History's Next Step or Monopolistic Boondoggle?

The big news this week was the launch of Google App Engine, "a developer tool that enables you to run your web applications on Google's infrastructure." This will allow startups to use Google's web servers, APIs, and other developer tools to build a web app on top of. Google clearly has the scale and smarts to provide this platform service to developers. However, it begs the question: why would a startup want to hand over that much control and dependence to a big Internet company? Check out Marshall's analysis.

A new feature at ReadWriteWeb is an interactive game helping you to understand and contextualize web tech news in a fun way. This week we posted, via a new app from Impact Games, an interactive game that will let you model Google App Engine's impact in the marketplace. You can play the game here.

Our other coverage: Google App Engine: Cloud Control to Major Tom; HuddleChat: Did Google Just Rip Off 37Signals? and Google Takes Down HuddleChat After Complaints About 37Signals Ripoff

Related: Red Dog: Microsoft's Answer to App Engine and AWS?

How to Get Customer Service via Twitter

There has been a lot of talk lately of companies monitoring social media, be it Twitter, blogs, or social networking sites, for mentions of their company name and responding to customer service issues. Some of this interaction has been in the Twitter community, with Comcast being one of the more active participants as of late. Although in some cases, customers twittered their frustration after failing to receive the support they needed through traditional methods, in many cases, Twitter was the first place the customers vented their frustration, and then were surprised when they received a response from a support rep or company spokesperson.

Related: 5 Ways to Find More Friends on Twitter and Twittermethis Is A Brilliant Marketing Experiment

Seven Tips for Making the Most of Your RSS Reader

RSS is a big deal, as anyone who's subscribed to even a few feeds probably knows. Once you get past just a few feeds, though, it can quickly get overwhelming. RSS can leave you feeling inadequate, brain-dead and uninspired. Trying out new things will help you discover new, magical experiences, though. Letting go of the stress caused by any obligation to read everything will go a long way.

Here are seven tips for making the most of your RSS reader, from simple to more complex.

SEE MORE WEB APPS COVERAGE IN OUR WEB APPS CATEGORY

Web Trends

How Important is Offline Access, Anyway?

In today's world, you're never too far from an internet connection. In developed countries, broadband access is available in more places than ever, and even poorer countries have internet cafes sprouting up left and right. Modern web workers and business travelers even take extra precautions to maintain always-on connectivity - packing air cards in their laptop bags or buying laptops that already have built-in EVDO access.

Despite the broad availability of internet access, it's the dead spots that have been pushing forward the need for offline access to web apps. For how can a web office suite like Google Docs or Zoho compete with desktop software if they become unusable when the internet connection disappears?

Why We Need Web Apps on the Desktop

Sarah Perez conclued in the above post that offline access is important now, but not as important as it once was. And that with the increasing ubiquity of Internet access, it is growing less important every day. However Josh Catone thinks there is an important distinction to be made between offline access to web apps (as Google Gears provides) and desktop access to web apps (as Mozilla's Prism and Adobe's AIR provide). The latter is a very important step in the evolution of web apps.

Where to Find Open Data on the Web

This week there was a story on Techmeme entitled "We Need a Wikipedia for data". The article, written by X-Googler Bret Taylor, discussed the difficulty of finding open data sets on the internet, something which could spur innovation, allowing programmers to build new applications the likes of which have never been seen before. What was interesting about this story, in addition to, obviously, the concept of a Data Wiki itself, was the amazing and insightful commentary around this concept, not just on the blog, but all over the net, something which led to the discovery of some pretty good data sources that are already available.

A Guide to Business Development 2.0

At least once each day I get a call from someone trying to sell me outsourced development services. It's difficult to not be frustrated with these calls and it is increasingly hard to be polite, because they come so frequently. Yet, more than frustrated, I am just puzzled. Does this tactic still work? Who in this day and age would give business based on a cold call? These companies could definitely use a dose of business development 2.0.

Related: Marketing 2.0: Can Meebo Make it Real?

SEE MORE WEB TRENDS COVERAGE IN OUR TRENDS CATEGORY

That's a wrap for another week! Enjoy your weekend everyone.

In addition, as CNet and Ryan Stewart's blog spread the story, more people chimed in with suggestions. And of course, the Hacker News guys had some more ideas themselves.

So what did everyone come up with? A lot of data sources are already freely available on the net, as it turns out, if you just know where to look. Here's a summary, do you have anything to add?

CKAN (Comprehensive Knowledge Archive Network)

The CKAN site is a registry of open knowledge packages and projects. Here, you can find open knowledge resources or register one of your own. What kind of stuff can you find at CKAN? They mention a set of Shakespeare's works, a global population density database, the voting records of MPs, or 30 years of US patents as some examples, but they also point you to some useful URLs, like flickr's Creative Commons page, where photos can be searched by license type.

CKAN

Infochimps.org

This project is attempting to assemble and interconnect the world's best repository for raw data - like a giant, free, open almanac. The best way to describe it comes from MetaFilter, where the project was spotted recently: "Just as Wikipedia will help you find out something about everything, infochimps.org will help you find out everything about something." What can you find there? Every wikipedia infobox, each infobox type in its own table, 50 years of global hourly weather data, all the tables from the US Census Statistical Abstract, oh and 100,000 official crossword words, too.

Infochimps.org

OpenStreetMap

Not a data set in the traditional sense, but definitely a useful tool, OpenStreetMap is a free, editable map of the world where you can view, edit, and use your own geographical data. The project was started because most maps actually have legal or technical restrictions on their use.

OpenStreetMap

MusicBrainz

Musicbrainz

Jigsaw

Dismissed by the blogosphere as a bad idea, if not downright evil, Jigsaw, the marketplace that pays you to give up other people's contact info now boasts 7 million complete contacts for the taking.

DBpedia

This site is a community effort to extract structured info from Wikipedia and make that data publicly available on the web, essentially turning Wikipedia into a database you can query. Is this the beginnings of a semantic web? Check out their downloads section for the datasets and then scroll to the bottom for even more links to data sources on the web.

DBpedia

flickr wrappr

Where DBpedia takes Wikipedia and makes it semantic, flickr wrappr extends DBpedia with RDF links to photos posted on flickr. Here's an example. Here's another. This is pure geek hotness.

Freebase

Freebase, an open, shared database of the world's knowledge, received a lot of mentions in the comments, so this must be a good one. Community built and maintained, it pulls from open data sources like Wikipedia, MusicBrainz, and the SEC archives to create structured information on many topics, including more popular ones like movies, music, people, and locations. The site, unlike some of the others in this list, is also easy to navigate and well-designed, which makes it that much better to use.

Freebase

Opentick

Perhaps one of the less interesting items due to its dry subject matter - financial data - it's certainly worth a mention because a free database of real-time and historical market data for trading systems and platforms is the kind of thing that really floats some people's boats.

ThingISBN

Thanks to LibraryThing, ThingISBN is the site's first API, and even though its competitor became a paid service, ThingISBN is still free for non-commercial use. The API doesn't just return the usual book data, but also something called "edition disambiguation," meaning it also returns a list of "related" ISBNs—other editions, other media, and translations.

Numbrary

Like the title suggests, Numbrary is a library for numbers. This free service helps you find, use, and share numbers from public record data sets, like census data or the CIA World Factbook.

Numbrary

theinfo.org

This site isn't just a place to build or collect data sets, of which they have quite a nice list, but a place where you can interact with other number-lovin' folks like yourself.

theinfo.org

The Data Wrangling blog

This blog post lists a bunch, and I mean a bunch, of open datasets on the web, which just goes to show how much of a cursory list my post really is.

hCard: Providing Your Contact Information

Users are tired of repeatedly entering profile information over and over again. This problem is solved by the microformat hCard. Leslie Chicoine, an Experience Designer at Get Satisfaction, talked about how her company had created a sign up process for their web application using hCard. (see screen shot below)

XFN & FOAF: Who are your contacts

Another microformat, XFN, and the FOAF project are techniques for embedding relationships in links. This allows social networks to recommend contacts that should be shared, without scraping web based email clients. Recently, Google introduced a Social Graph API, which "index[es] the public Web for XHTML Friends Network (XFN), Friend of a Friend (FOAF) markup and other publicly declared connections".

Something very interesting that I wasn't aware of until today's panel was that both Plaxo & Six Apart were working on something similar before Google announced OpenSocial, according to Joe Smarr and David Recordon. However, once Google started focusing on this they were happy to hand it over to them - because Google "has the web on a hard drive", so it makes the crawling component of this far less difficult. For a good overview on Google's Social Graph API, check out the following introductory video:

OpenID: Authenticating Individuals

OpenID is a decentralized framework for allowing social networks (and other web applications) to authenticate users. In other words, it lets users login using shared credentials across different services. It also allows individuals to decide what information they want to share with each application. For example, a user might decide not to provide their postal or email address.

OAuth: Authorizing Access

The final protocol discussed was OAuth. It is a protocol that is less about authentication (OpenID) and more about authorization. The protocol has been developed over the last year. The specification was released in December 2007 and modeled off a number of authorization protocols, including the Flickr Authorization protocol. According to Chris Messina, a number of services have already started using it including:

• Fireeagle
• Open Social
• Pownce
• Get Satisfcation, and
• Magnolia
• (and Twitter support will be coming soon)

Chris also pointed to a comment in a recent post of ours about email passwords, that highlighted the need for tools like these. Also there was a comment on RWW from Oren Michels at Mashery, indicating it is the most requested feature for them right now.

Conclusion

Securely moving your data around the web has increasingly become an important concept on the web. Arguably, it was the most discussed meme at this year's SXSW. While not an application, you could say it has been 'this year's Twitter'.

The Data Portability group deserves credit for educating the market. Beyond that, it is also an idea whose time has clearly come. It is interesting to think what applications will be built on top of these portability standards - they might be popular by next year's SXSW!

While startup churn is inevitable in any industry (thank goodness we're not restaurant founders!) I think this forecast is selling the future short. There are some big trends I'm really excited about for the web in 2008. Whatever happens to the economy, there's at least a whole lot of innovation to be inspired by right now. Ultimately, I think that will end up brightening the picture for all of us around the world.

Let's Build Some Stuff

For each of the 5 big topical trends described below, I've assembled some resources I think will be useful for anyone who wants to keep up with cutting edge developments in these fields in the next year.

* An OPML file of top blogs on each subject. This is a bundle of feeds you can import into your reader.
* A filtered RSS feed of just the most popular items regarding each topic (using AideRSS). Remember, whenever you subscribe to new RSS feeds - some of the magic won't be visible until you mark all the initial items as read and new ones come in again.
* A Custom Search Engine that you can bookmark and use to search inside the top news and reference sites regarding each topic.

If you'd like to recommend any additional feeds or sites to add to these resources just let me know and I'll check them out. Please do, in fact. I hardly have the energy to make these lists exhaustive by myself. That's kind of the point of this whole web endeavor, isn't it?

All of these resources are dynamic; sites added at a later date will be automatically delivered to everyone who subscribes to these OPML files today.

So let's get to some trends that are shaping up to make a big impact on 2008...

Open Data

Data silos and walled gardens are a huge loss of opportunity and more people are figuring that out every day. The developments in this direction seen just in the last half of 2007 are too numerous to list here but some of the subthemes include the following.

• Data portability - taking your archives and friends from one site to another.


• The portable identity of OpenID


• The Google-led OpenSocial


• Google's Android mobile OS


• The "by hell or high water" rise of data-centric startups


• The personal data aggregation and publishing tools called Lifestreaming apps like Tumblr, named one of Time's Top 50 Websites of the Year, or the bleeding edge Onaswarm, Lifestrea.ms and Soup.


• The burgeoning Attention standard APML and various other efforts you can learn more about at sites like DataPortability.org.

Open Data Resources:
*RWW Open Data Feed Favorites OPML file (save link)
*RWW Open Data Feeds - Best of Feed (copy and paste to your reader)
*Preview the above feeds before subscribing (pop-up window)
*RWW Open Data Sites Search (Visit and Bookmark)

Recommendation

As is aptly demonstrated from the category above, the future is likely to be even more swamped in data, social and content options than the web is today. From Google Reader's recent incorporation of both feed recommendations and shared items in Reader from your contacts in GMail to the ascendancy of services like Last.fm, Pandora and StumbleUpon - recommendation is beginning to make a big splash already.

Dr. Rick Hangartner, Chief Scientist at recommendation engine MyStrands, posited the following about the relationship between search engines and vendor-specific recommendation engines in a recent guest post on mobile search blog MSearchGroove:

In the near term, search engines will increasingly incorporate simple recommender technologies to handle approximate queries (e.g. “You asked for this, and based on similar queries/behavior by others, you might be looking for this.”). But in the long term, the recommender industry will be larger, and recommender technologies will be more pervasive than the search industry and search technology as we know it. [Because there will be recommendation going on all over the web.]

Recommendation Resources:
*RWW Recommendation Industry Feed Favorites OPML file (save link)
*RWW Recommendation Industry Feeds - Best of Feed (copy and paste to your reader)
*Click to preview the above feeds before subscribing (pop-up window)
*RWW Open Data Sites Search (Visit and Bookmark)

Semantic Web

A Semantic Web has been in the works for a long time but is just starting to hit the scene for real. The idea is that semantic web technologies are able to derive meaning from online content and determine connections where none have been made explicitly.

If I'm looking at a web page about Assata Shakur, for example, then SemWeb tech should be able to determine that she's the subject of the page I'm looking at and that it's a biographical page. Once that's been determined, semantic technology can leverage the two trends discussed above (data openness and recommendation) to do all kinds of interesting things.

As I wrote in coverage of an excellent interview with Semantic Web scientist Yihong Ding - once our software is capable of deriving meaning from web pages it looks at for us, then there's a whole lot of work that's already been done, allowing our creative human minds to reach new heights. By pre-processing online content for us, Semantic Web technology lets us start from a point of higher abstraction.

Richard MacManus called Semantic Web application Twine possibly the first mainstream consumer semantic web app, but there's a whole lot of innovation going on in this space. Major companies are starting to leverage Semantic Web technology under the covers of existing websites as well.

Semantic Web Resources:
*RWW Semantic Web Feed Favorites OPML file (save link)
*RWW SemWeb Feeds - Best of Feed (copy and paste to your reader)
*Click to preview the above feeds before subscribing (pop-up window)
*RWW SemWeb Sites Search (Visit and Bookmark)

Mobile

While Michael Arrington says the release of the iPhone relieved him of any pressure to build a mobile version of TechCrunch, I think there's still going to be a whole lot of innovation in the mobile space well into the future.

Most of the people online in this world access the web through a tiny little computer they carry in their pocket and also use as a phone.

Mobile means more than just small, though. It also means portable, fast, location-aware and tied to voice, media and the meat-space.

Mobile is already a great analogy for data portability in general - people are thrilled in the US that we can now switch cell phone carriers and keep the same phone number. Imagine if we lost our contacts when we switched phones. The same type of expectations are totally reasonable for services online.

Once mobile really gets tied into open data on the web, to recommendation engines and to the semantic web - then we'll be cooking with gas.

Mobile Industry Resources:
*RWW Mobile Feed Favorites OPML file (save link)
*RWW Mobile Feeds - Best of Feed (copy and paste to your reader)
*Click to preview the above feeds before subscribing (pop-up window)
*RWW Top Mobile Sites Search Engine (Visit and Bookmark)

Visualization

We're only going to get so far if we just tell the world, "trust us, all this ephemeral crap is going to change your life!." A big part of why there isn't widespread consumer demand for OpenID is that the benefits of it haven't been clearly communicated. The concept is gaining steam almost in spite of the communication of its advocates, many people believe. The future of OpenID and many other key technical innovations, lies in communicating with people about what they can do with the tools. That is not easy to do with things that are complicated or new.

Just as video has changed the web forever because visual communication is infinitely more evocative than text - so too do I expect the perceived value of visualization to grow by leaps and bounds in 2008.

I wrote a post about 3 methods of visualizing best practices in social software design over Thanksgiving, highlighting the work of Thomas Vander Wal, Chris Messina and the Google OpenSocial team. To that list I'd like to add Dave McClure's SlideShare archive, where you'll find images like the one below. This stuff is pure gold. Powerpoint is the future? Well, effective visual communication of complex data-based concepts is going to be an invaluable part of the future.

Visualization Resources:
*RWW Visualization Feed Favorites OPML file (save link)
*RWW Visualization Feeds - Best of Feed (copy and paste to your reader)
*Click to preview the above feeds before subscribing
*RWW Top Mobile Sites Search Engine (Visit and Bookmark)

That's It! Post Suggestions Below, Please!

I hope these resources will prove useful for our readers in the coming year. Please let us know about any sites that ought to be included here - or let us know if you think I'm barking up the wrong tree and these won't in fact be hugely influential trends in 2008. Thanks for getting all the way through this long post!