A top level domain is a core part of how the Internet organizes and parses the names of websites. The most common, of course, is .com, but other TLDs are .net, .org and country domains like .CO or .UK. ICANN's new gTLDs will allow companies, governments and other organizations to register unique strings. For instance, are we about to enter the era of .pepsi? See below for everything you need to know about the new domain name system.

ICANN believes that the new gTLD system will be a boon for the Internet economy. Startups, business, entrepreneurs and governments will all be allowed to own and manage their own little portion of the Internet, if they so choose.

Here is ICANN's reasoning:

"One of ICANN's key commitments is to promote competition in the domain name market while ensuring Internet security and stability. New generic Top-Level Domains (gTLDs) help achieve that commitment by paving the way for increased consumer choice by facilitating competition among registry service providers. Soon entrepreneurs, businesses, governments and communities around the world will be able to apply to operate a Top-Level Domain registry of their own choosing."

Many people think that the new gTLD system will start a gold rush for new domains. To a certain extent this is true. A lot of companies will be bidding big money to retain their trademarks as a gTLD. ICANN will not hold a trademarked name for a specific gTLD just because that company owns the trademark. On the other hand, owning a gTLD is a big organizational and financial responsibility that will be prohibitive for many brands and enterprises.

For example, the base financial commitment for entry for a gTLD is $185,000. If you thought you would waltz in and grab your last name as a TLD, you are probably not going to be able to. It is unlikely that we are going to start seeing individuals with personal URLs like "joe.smith." In this case, Mr. Smith would need to pay for the gTLD and prove to ICANN that he and his organization can support the strict requirements of owning a gTLD.

"Please note that applying for a new gTLD is not the same as buying a domain name. An applicant for a new gTLD is, in fact, applying to create and operate a registry business supporting the Internet's domain name system. This involves a number of significant responsibilities, as the operator of a new gTLD is running a piece of visible Internet infrastructure."

The financial commitment is more than just $185,000 that serves as an evaluation fee. A deposit of $5,000 is required with the application. As a gTLD owner, an entity is required to be the keeper of that domain. That means the company will, in one way or another, be responsible for every other URL that pops up using the new name. In the Smith scenario, whoever owns the Smith gTLD would be responsible for the organization, security and infrastructure of the domain name. After a domain is approved, there is a $6,250 monthly fee and a $0.25 per transaction fee after the first 50,000 transactions in a calendar year.

ICANN does not know how many applications it will receive in this first round of new gTLDs. Entities can apply for domains from today (Jan. 12) until April 12, 2012. This round will contain a maximum of 500 new gTLDs applications and subsequent batches will be limited to 400.

The application and review process is extensive. We are not going to see new gTLDs crop up tomorrow or even next week or next month. The review process for each application can take anywhere between nine and 20 months.

"There are several stages that an application may pass through prior to a final determination being rendered. Those stages are Administrative Check, Initial Evaluation, Extended Evaluation, String Contention, Dispute Resolution and Pre-delegation. The shortest path for a successful application is to pass Administrative Check (lasting 2 months), Initial Evaluation (lasting 5 months) and then move to Pre-delegation (lasting approximately 2 months) without any Objections filed or String Contention concerns. In this case the evaluation process could take as little as 9 months to complete. On the other hand if an application does not pass Initial Evaluation and elects Extended Evaluation and/or is in the Dispute Resolution or String Contention stages then the evaluation process could take up to 20 months to complete (or longer in the event that unforeseen circumstances arise)."

New applications will be assessed by independent third-party expert panels.

Next Page: What gTLDs will look like and other issues and concerns . . .

What Will New gTLDs Look Like?

New domains will be required to be at least three characters in length and contain only alphabetic characters from A to Z. Hence, no domains will be issued that have numbers, like .c0m or .1234.

Country codes are not included in this sale of gTLDs and are a completely separate part of ICANN's TLD standards. Part of the three letter requirement is to protect current and future country level domains like .CO (Colombia) or .UK (United Kingdom).

Not included in ICANN's gTLDs are second and third string domain names. Consider maps.google.com. In this case, the TLD is .com while the second string is .google and the third .maps. Operators of new gTLDs will be the ones to validate any second and third string domain names.

Multiple languages will be supported in the new gTLD system, and non-Latin writing systems, such as the Arabic alphabet and Chinese characters. When an applicant applies for a gTLD, it will not own the translation of the domain. Hence, if you are applying for .thing, you will not also receive the Spanish .cosa or the equivalent in Arabic letters or Chinese characters.

There are two options for new owners of gTLDs to operate them: open or closed. One will be that a company or a brand owns its own name, Coca Cola for instance, and keeps its second and third string URLs within the company. Like, news.coke or offers.coke. Coca Cola would not sell URLs to outside entities and maintain the entire gTLD in the corporate environment. This would be a closed example.

An open example would be if some entity purchases the aforementioned .smith gTLD. The organization could then start selling DNS registrations to individuals, like joe.smith or betty.smith. This is where the true money will be made in the new ICANN infrastructure.

Owners of new gTLDs will likely not be able to turn around and sell the domain. The extensive application and review process contains many layers of objections, comments and evaluation on how well an organization can conduct the domain. So, buying .sex and thinking that it can be flipped for millions of dollars later would not be a feasible model. Buying .sex and selling registrations to the domain would function much more effectively in the new system.

There are two types of applications: standard and community. A community could function as a group of like-minded people working towards a common goal with a reasonable shared infrastructure. A community could be, say, a large group of Silicon Valley startups that all want to use the gTLD .startup. The companies could create a partnership with a central body that would support the gTLD. A standard application would be to a company or an organization that merits consideration.

An organization cannot, however, apply for a gTLD on anothers behalf. If you buy a gTLD, it is yours. You cannot turn around and flip it to GoDaddy or Namecheap for management. On the other hand, there is nothing stopping the domain registrars from applying for gTLDs on their own.

"ICANN will only enter into an agreement with the applicant. There's no provision for Party X to enter a registry agreement with ICANN designating Party Y as the registry operator.

Other Issues

ICANN makes it a point to say that "no, gTLDs are not going to break the Internet."

"The increase in number of gTLDs into the root is not expected to affect the way the Internet operates, but it will, for example, potentially change the way people find information on the Internet or how businesses plan and structure their online presence."

The application and review process, not to mention the cost of supporting a gTLD, is prohibitive to cybersquatters. Some companies may buy their .brand as a defensive strategy but the fact of the matter is that it will be extremely difficult for squatters to get through the process without a plan of action and support. Some companies that can support might just buy the name so it will not have to deal with headaches later.

Forrester's Jeff Ernst has this to say on companies making decisions of whether or not to apply for a gTLD:

"Keep in mind that this is much bigger than just moving your brand from the left to the right of the dot. I'm not a big fan of submitting a defensive registration. Get some of your smartest people from marketing, finance, legal, distribution, service, and strategy together. Examine some of the biggest challenges you have today in any of those areas. Think about your company's strategy and priorities over the next 4 years. See if you can find a strategic application of a registry that can differentiate your company, contribute to growth plans, or help with one or more of your biggest challenges. And if so, go forward with a strategic application. If not, read my latest reports and understand the risks and actions you should take when you stay on the sidelines."

So we, for example, are not worried in the slightest about some organization buy the .writeweb gTLD and blackmailing us to sign up for read.writeweb. There are bigger fish in the sea.

From a security perspective, it is not likely that spammers will buy their own domains and use them as a launching point for spam attacks. As Blue Coat's Chris Larsen pointed out to me recently, it is fairly easy to block spam and malware once you know where it is coming from. So, if spam is coming primarily from a gTLD owned by a botnet operator, it would be simple to just block that gTLD. Again, given the cost and review process, that is not an efficient use of funds for malware makers.

For additional information, see ICANN's FAQ and the Application Guidebook.

What are the concerns regarding the new ICANN gTLD ecosystem? It will definitely dilute the TLD infrastructure and fragment how the Web is organized. Is that a bad thing though? ICANN is a not-for-profit organization so it is not motivated by making money hand over fist. How excess money from the application process will be used will be put to a vote. Overall, is this a positive or negative move for the Web? Let us know in the comments.

Update: GoDaddy has responded to Namecheap's accusations. See the statement below.

We wanted to give our customers a quick update on the status of domain transfers associated with one of our competitors, GoDaddy.

First, we're very sorry that some of you in the past 24 hours have experienced delays in transferring domains over to us.

As many customers have recently complained of transfer issues, we suspect that this competitor is thwarting efforts to transfer domains away from them.

Specifically, GoDaddy appears to be returning incomplete WHOIS information to Namecheap, delaying the transfer process. This practice is against ICANN rules.

We at Namecheap believe that this action speaks volumes about the impact that informed customers are having on GoDaddy's business.

It's a shame that GoDaddy feels they have to block their (former) customers from voting with their dollars. We can only guess that at GoDaddy, desperate times call for desperate measures.

Don't worry - each and every transfer request will be processed manually by our team. Every request will go through. We won't rest until everyone who wants to join the Namecheap family can do so!

Note: Italics emphasis is ours, bold by Namecheap.

The Verge reports that GoDaddy lost 21,054 domains on Dec. 23, 2011 yet gained 20,034 domains. While it has been widely reported that there has been an exodus from GoDaddy, it appears that much of the churn has been business as usual for the domain registrar.

Users began to boycott GoDaddy and transfer their domains away from the service after GoDaddy was revealed to be one of the official supporters of the Stop Online Piracy Act that has become very unpopular among technology circles. GoDaddy has since rescinded its support of SOPA but the damage has been done and the fact of the matter is that GoDaddy still supports SOPA in theory, if not in an official capacity.

Here is a screen shot from a user trying to transfer his website:

Reports surfaced last week that GoDaddy was calling clients with large numbers of registries that were trying to transfer away, begging them to stay. GoDaddy realizes that the cost of the SOPA blunder and PR nightmare could mean millions of dollars lost in the short term and less public trust in the longer run. Namecheap wants to position itself as the go-to destination for GoDaddy refugees. Hence, whatever Namecheap can do to make GoDaddy look bad in the process is good for business.

Are you transferring out of GoDaddy? Has the process been easy? Have you experienced delays? Let us know in the comments.

Update 5:18 EST, Dec. 26:

GoDaddy has responded to Namecheap's accusations in an email to tech blog TechCrunch. In the statement, Richard Merdinger, senior director of product development at GoDaddy, says that Namecheap never contacted GoDaddy about normal rate limit blocking for transferring of domains. According to Merdinger, the block has been removed and GoDaddy is not hindering domain transfers to Namecheap. See the statement sent to tech blog TechCrunch below:

Namecheap posted their accusations in a blog, but to the best our of knowledge, has yet to contact Go Daddy directly, which would be common practice for situations like this. Normally, the fellow registrar would make a request for us to remove the normal rate limiting block which is a standard practice used by Go Daddy, and many other registrars, to rate limit Whois queries to combat WhoIs abuse.

Because some registrars (and other data gathering, analyzing and reporting entities) have legitimate need for heavy port 43 access, we routinely grant requests for expanded access per an SOP we've had in place for many years. Should we make contact with Namecheap, and learn they need similar access, we would treat that request similarly.

As a side note, we have seen some nefarious activity this weekend which came from non-registrar sources. But, that is not unusual for a holiday weekend, nor would it cause legitimate requests to be rejected. Nevertheless, we have now proactively removed the rate limit for Namecheap, as a courtesy, but it is important to point out, there still may be back-end IP addresses affiliated with Namecheap of which we are unaware. For complete resolution, we should be talking to each other -- an effort we are initiating since they have not done so themselves.

-Rich Merdinger
Sr. Director of Product Development - Domains
Go Daddy

Several commenters and people on Twitter have noted that of the several domains they transferred off of GoDaddy, the last few have taken a lot longer than the first. What has been your experience? Let us know in the comments.

If you want a Web that degrades gracefully onto older systems, works with any browser on any screen, is accessible to all regardless of abilities, and doesn't load like trying to squeeze football down a garden hose, join the movement and don your blue beanies.

As we learned at ReadWriteWeb 2Way this year, the World Wide Web Consortium (W3C) is working to pull us together around Web standards. It has developer and community groups working on HTML5, device APIs, standards for privacy and security, and other essential building blocks of the future Web that will keep us, our computers and our sites and services from talking past each other.

Some decisions are hashed out quietly by experts. Others happen loudly in public. Web standards are sustained by conversation, and Blue Beanie Day is a chance to start some.

Don Your Blue Beanies!

To participate, take a self-portrait wearing a blue beanie and upload it to the Blue Beanie Day 2011 Flickr pool. You can also share it on Instagram and Twitter with the hashtag #bbd11. Chairman Zeldman also offers a downloadable blue beanie you can add to your social network avatars.

You can also just share the link to the Blue Beanie Day page.

Want to learn more about Web standards? Here's our infographic on the history of Web standards.

Check out our coverage of past Blue Beanie Days.

The Massachusetts Institute of Technology may be the birthplace of the American geek. Within MIT, its Media Lab drills down to the heart of the next wave of technology from creating buildings with 3D printing to prosthetic limbs to gesture-based user interfaces. For instance, the MIT Media Lab was where the idea for the technology seen in the movie Minority Report originated.

The unofficial motto of the MIT Media Lab is "demo or die." It is akin to the classic academic model of "publish or perish," except that students and faculty at the Media Lab are encouraged to actually create the products they are thinking up, as opposed to pontificating upon them in research papers. See below to check out some of the amazing waves of technology that will be bursting out of the Media Lab in the future.

There is a little orange robot called Combusto. It can show emotion and has a wide array of movement. Think Wall-E with fur. Combusto is powered with an Android smartphone that has a unique chipboard in it that helps determines Combusto's functions.

Right next to the robot lab, there is a team working on holographic 3D imaging. Another line from Star Wars comes to mind: "Help me Obi-Wan Kenobi, you are my only hope." One of the goals for the holographic imaging team would be to make it feasible to make those Star Wars-style holographs a reality.

Throughout the Media Lab, students, researchers and faculty are using the Microsoft Kinect for gesture-based moving and tracking. The Kinect API is one of the most innovative consumer technologies to hit the market in a long time and its affordability makes for perfect use in research settings.

Tangible Media

If you put the ideas of holographic images together with the notion of tangible media, you get the computer that Tom Cruise uses in Minority Report. The company that is trying to make that vision a reality is called Oblong, and it originated at the Media Lab.

Tangible media is the idea of being able to manipulate a user interface either through gesture or interacting with a physical object connected through the Internet. This can also be done through Kinect, although the department also has a series of infrared cameras that can track movement and intention with more precision. The cameras are used to localize an item (say, a person) in a 3D space.

Tangible media is described as where the input is also the output. Say I have a gesture-based camera watching my movements. If I am using building blocks in physical space, those same building blocks should move in the digital space as well.

Capacitive touch, which is perhaps the biggest breakthrough in the smartphone revolution, also has origins with projects in the media labs. Indirectly, the iPhone and touch-based devices have roots at MIT.

Robotic Opera, 3D Printing & More

It is hard to keep track of everything that is happening at the MIT Media Lab without being an insider. There are teams working on prosthetic limbs that reduce the impact of missing legs and let a person move around normally. There is a storytelling center looking at new ways to present digital media. There is a camera center that is working on how best to utilize the powerful cameras now attached to every person's hip via cellphone. There are teams working on creating different densities of material coming through a 3D printer that can make the building blocks of structures with concrete. The innovators behind Rock Band came from the MIT Media Lab. These days, the music group is working on robotic operas.

The MIT Media Lab is majority funded by sponsors or "members" that are companies working on specific functions. For instance, DirecTV helps fund some of the interactive television projects that are being worked on. The building is designed to be an open center of collaboration where everybody can see what the others are working on. The idea is to get designers and scientists together to create technology that will have an impact on the world.

Google is doing this through a program called WebRTC, a project that was spawned from the company's acquisition of Global IP Positions (GIPS) in May 2010. Google is now ready to begin implementing WebRTC into its Chrome browser. It will augment its current video chat capabilities within Gmail and put a dent into real-time, unified communications aspirations of companies like Microsoft (Skype), Apple and Cisco.

Secondly, and on a more practical basis, Google's three biggest initiatives are in developing Web applications outside of the desktop (see Chrome OS), mobile functionality and the cloud. Those three are tightly intertwined in Google's vision. WebRTC fits within all three of those by creating system where people can connect through video or audio anywhere on the Web, on any device, through the browser, in the cloud.

WebRTC makes this possible through Javascript APIs that are free and open source. Hence, not limited to Google. Mozilla and Opera have been invited to the initiative as well. The immediate goal is to collaborate with the open source community to create a universal set of Javascript APIs that enable real-time communication through the browser across the Web.

Google announced last week that WebRTC is about to land in Chromium, the code source for the Chrome browser.

"We are working hard to complete the remaining steps for a full integration in Chromium and Chrome including JavaScript APIs in WebKit and handling of the native audio and video capture and rendering," Jan Linded wrote in a blog post at WebRTC.org. "When we are done, any web developer shall be able to create RTC applications, like the Google Talk client in Gmail, without using any plugins but only WebRTC components that runs in the sandbox."

Generic Top-Level Domains (gTLDs) are essentially specific destinations for brands. Companies will be able to buy their brand and attach it to a URL. So instead of seeing Pepsi.com, the soda manufacturer could have Pepsi.soda or something similar. It will not be cheap to get your own TLD, with an $185,000 application fee and $25,000 a year to run the registry. Yet, some Internet advocates are crying foul, saying that gTLDs will create new headaches in cybersquatting, trademark issues and excessive spam.

The price tag for a gTLD may cut down on the overall instances of cybersquatting, except for the most affluent spam networks and domain registries. ICANN will be accepting applications for new top-level domains between January 12 and April 12, 2012.

ICANN is providing safeguards to ward off mass cybersquatting. The Applicant Guidebook has gone through seven significant revisions since 2008 that incorporated 1,000 or so comments from the public. The evaluation procedures provide for background screening of pre-applicants that measure business history and look for history of cybersquatting. It will conduct string-similarity reviews to determine if the domain is like anything else currently on the Internet and assess the potential security risks of creating a new TLD.

There are currently 22 TLDs that range from the original .com to .org and .net. ICANN's final approval of the gTLDs will certainly make that number skyrocket but the question is what the final affect on the Internet will be. Are gTLDs the first salvo in a new Web land rush or will it be a source of new innovation for the next decade of Internet development?

The Biggest Thing to Happen to the Internet Since .Com?

Forrester analyst Jeff Ernst believes this is one of the biggest things to happen to the Internet.

"I think this is probably the biggest change to the the Internet since we have had it," Ernst said over the phone. "A lot of the biggest brands are figuring out the requirements. It makes a lot of sense as a brand owner to have as much control over your brand as possible. Why be stuck behind .com when you can own your own primary domain and control the secondary domains you issue within your domain?"

Ernst points out that there are stringent technical guidelines to obtaining a gTLDs and the ability to administer it. There is a nine-month application process and brands must have the ability to effectively administer secondary domains. This could increase corporate IT spending as brands feel the need to get their own TLDs but then must adhere to ICANN's policies.

"Many of the biggest brands are planning to apply for their .brand TLD, but many marketing leaders I've talked with look at this as a nuisance and are skeptical about whether Internet users will embrace them," Ernst said in a blog post.

Lauren Weinstein, the co-founder of People for Internet Responsibility lambasts ICANN in a blog post, calling ICANN and the domain registries the "Domain Industrial Complex" and calling shame to all that helped approve this decision.

"Has the horrific economic saga of the last few years taught us nothing?" Weinstein wrote. "Is there no sense of ethical or moral outrage among those persons who are truly concerned about creating the best possible future for the entire Internet and Internet community, not just for a comparatively few "domain exploitation" tycoons and would-be tycoons?"

This has led more technologists including myself to start thinking about how the Web needs to evolve to keep up with user expectations. People hold companies to impossibly high expectations to deliver extremely personalized experiences as they browse, shop, learn and play on the Web.

What we need is to build a smarter approach that allows companies to adapt to their customers' needs in real-time.

The concept of collective intelligence, which I'll address a little further down below, will be critical to achieving this vision--something I like to think of as an "adaptive Web." That is, a digital experience that is always relevant and based on users' current intent and interests. It also must be device-agnostic, especially important given the increased mobility of the online experience -- a challenge analyst firm Forrester calls "the Splinternet."

In an adaptive Web scenario, the Web will truly come to life and become self-learning. We'll see the Google way of determining linkages between sites and content go by the wayside in favor of an approach that's based entirely on what the Web community at large and like-minded users within it found useful.

An adaptive Web experience would represent a dramatic shift in how we interact with the Web (or more accurately, how the Web interacts with us). Let me explain.

What An Adaptive Web Would Look Like:

Let's say you're planning a trip to Vail with some old college buddies. So you hop on to Expedia to buy plane and hotel tickets. On checkout, you're suddenly alerted to a winter jacket sale at REI.com based on the Web community's affinities between Vail and buying ski jackets. No hard coding necessary.

After you get your jacket you then visit the Vail Ski Resort homepage, which is automatically optimized to show you the best trails because it knows you're already headed there (and thus need no persuading why Vail is the best place on Earth to visit!). Later that night you search for "Sushi in Vail" on your iPhone and the top search results are all restaurants by your Vail hotel that are open on the days of your trip.

This is just the tip of the iceberg. At some point, we could see a time when not only your Web experience is adaptive, but also your social connections - based merely on your current intent and interests. For example, in the scenario above you could also be temporally linked with people who are going to Vail at the same time or who live in your area. This would have numerous benefits: you could share travel trips, ask them to carpool with you to save gas, or meet up for sushi at that restaurant you just discovered.

But creating an experience like this requires much more than machine learning and advanced algorithms. It requires a deeper understanding of how the human brain works.

Next page: Changing the Status Quo: No Easy Task

Changing the Status Quo: No Easy Task

To no surprise, there are numerous technical and psychological challenges for building an adaptive Web. Namely, I see three primary roadblocks:

1. Privacy Issues: To deliver adaptive experiences, we have to pay attention to what people are doing online in the first place. Different users have varying levels of comfort. We'll have to find some sort of middle ground where the value of an adaptive experience greatly outweighs users' privacy concerns.
2. Deciding on the Method: Second, there's determining the approach itself. Do we need a "metalayer" over the web? Some sort of toolbar or plugin that could connect users' entire Web experiences across devices? Do ISPs need to get involved at the network level to watch every site users' visit and how they engage with it? These are all options to consider - some more realistic than others - but the path is murky at best at this point.
3. Determining Users' Intents: The third obstacle is the biggest obstacle of all: pure science. It's not a trivial problem to automatically pinpoint and serve up an experience based on a user's current intent and context. As someone who has devoted his life's work to studying human/computer interaction, I can't emphasize this enough. Predicting what people want and need, and adapting their Web experience in real-time is perhaps one of the remaining "big picture" challenges facing technologists.

The good news is that we're not starting from scratch. In many ways, Amazon and recommendation engines were really the beginning of the adaptive Web, but because they were completely relegated to sidebar widgets on individual sites and leveraged only a small subset of users' implicit behavioral signals, they stopped short of delivering a truly end-to-end personalized and relevant Web experience.

The DiggBar hinted at the creation of browser/machine-agnostic Web "metalayer" but as we know that did not end well. The behavioral targeting guys of today have the right spirit, but their focus is on individuals' previous behaviors and the often-hardcoded needs of the seller rather than the dynamic needs of the buyer, which make it unlikely they'll be the ones rising to the challenge.

The Power of the Collective

Let's revisit collective intelligence and its role in making the adaptive Web a reality. Collective intelligence refers to the process of gathering insight from a group of like-minded individuals online, often implicitly based on their shared navigation and engagement patterns. A central concept of collective intelligence is to aggregate behaviors of the silent majority of website visitors, augment that information with the expertise of super-users and provide the most relevant information that meets every individual user's goals.

An obvious benefit to using collective intelligence to power the adaptive Web is one of mere scale: it enables machines to draw conclusions about an individuals' current intent based on the knowledge and experiences of the larger community. It also gives us the power to efficiently deliver automated and real-time experiences to users. This would be very difficult within any user-by-user scenario, which again poses enormous difficulties in matters of scale.

The Adaptive Web: Not If But When

Despite the obstacles I discussed earlier, I would argue that we're on the path to moving toward an adaptive Web - and at a rapid rate. The days of programmers organizing the Web and creating linkages and associations for us are ending, and the age of the self-organizing, adaptive Web is quickly approaching.

In many ways, it's in step with the democratization of the Internet: In the first phase of the web, only people "in the know" could publish content. In web. 2.0, everyone could explicitly contribute to the community via social networking and self-publishing tools.

Naturally, the next step is for all users to be able to contribute to building and organizing the Web in its entirety, organically shaping our individual relationship with it based on our collective experiences.

Do you agree? What do you think will be the biggest technical, if not cultural challenges in realizing an adaptive web?

Photo by waterlamd

A report by analyst firm Mintel released this week claims that "Wi-Fi pirating" could be a main reason for the slow growth of broadband adoption over recent years.

"Home Internet penetration barely moved from 2006 to 2009. The slow growth in the era of Facebook, Pandora and YouTube shows that people are accessing the Internet from home through different methods, even if they haven't paid for access themselves," said Billy Hulkower, a senior analyst at Mintel. "Younger consumers appear especially likely to use a neighbor's Wi-Fi signal instead of subscribing at home as they are more likely to know how to find and connect to their neighbors' service."

Beyond these "young consumer", guess who else appear most likely to steal Wi-Fi - those households bringing in more than $75,000 a year.

As part of the same report, Mintel says it found that cell phones and mobile devices have more than doubled from 2005 to 2009 as methods for home Internet access, while dial-up still hangs on with just under 10% of the respondents.

Today, Google is announcing one such "white space" use - a broadband network.

Microsoft has done the same on its 500-acre Redmond, Washington campus, a feat that it says would have taken thousands of wi-fi routers and other equipment to replicate. These "white spaces" are the unused part of the broadcasting spectrum that were made available by the changeover to a digital signal. The Microsoft white spaces project used two transmitters to cover the entire campus.

According to Alder, the project is operating on an "experimental white spaces license". Next week, the Federal Communications Commission will be voting on "governing the white spaces - a vote that could pave the way for unlicensed white spaces deployments across the country."

Many people have worried that the inaccessibility of historical Twitter search results might mean that the messages weren't being saved at all. Company co-founder Biz Stone told us otherwise by email today, though. Twitter is in fact saving all the tweets. You just can't access them through search "right now."

Scrubbed geolocation data after two weeks and no way to access historical information at all? That sounded like a pretty bum deal for a world-changing new communication platform. So we emailed to ask.

This was the reply we received from Biz Stone: "We definitely save all the tweets although you're right in noting that our search focuses more on newer content right now. And yes, the plan is to drop the coordinates after 14 days."

There you go. Now that Twitter is putting $100 million more into the bank, it would be great to see some of those resources dedicated to making information retrieval on the service a first class function. The future would be thankful.

Twitter has offered two explanations for the change. First, that very few users were choosing to receive these kinds of messages anyway and that it was confusing. Then, this morning, the company put up a blog post saying simply that "there were serious technical reasons why that setting had to go or be entirely rebuilt--it wouldn't have lasted long even if we thought it was the best thing ever." So what's the story? Here's our best guess.

First, it is clear that most users have not chosen to receive public replies sent by their friends to people they themselves aren't following. Non-early-adopters in particular are quite likely to consider these kinds of messages noise. Many of us early adopters believed such messages were a part of the magic of Twitter; it's a great way to discover people your friends find interesting. Information overload can be dealt with by forming groups in a third party twitter client like Tweetdeck or Seesmic Desktop. Getting over our antiquated sense of guilt and obligation concerning reading every message we receive would help too. (We've argued that online noise is good for you.)

The fact is though that most Twitter users now probably only receive replies sent to people they know. That's been the default setting for several months and we assume the system has thus been architected for this use case.

Scalability

What does that mean? Probably this: when you publish a Tweet that begins with @username, the computer that all your user files lives on in Twitter HQ probably already knows not only who your friends are, but also who their friends are. Everyone who is your friend but is not friends with the person you are replying to won't even have the message sent to them. Making that determination locally and limiting the tweets broadcast to your friends is probably much more efficient for Twitter than sending that message to all your friends.

Even updating the "friends of friends" files for all your friends every time you make a new friend - is more efficient than sending out replies to all your friends who aren't following the recipient.

Sending Tweets from one user to another is traffic-expensive and if most people don't want replies directed at people they don't know, then none of us are going to get that kind of message.

Most people don't want a noisy public conversation? Most people don't want serendipitous introduction to new people? That sounds like most people don't want a key part of what makes Twitter most magical. That magic is expensive and if millions of Oprah followers don't even want it, then a few thousand @marshallk followers don't get to have it either.

Who can blame them? Everyone wants Twitter to scale. Maybe it has to be neutered in order to do so. Architecting the social graph can't be easy. We can be sad about the decisions that the company makes in order to try to do it, but that probably isn't going to change things.

Instead of strolling through the social graph to discover new users, Twitter users will likely be given a new set of recommendation features pointing them to new people to follow. It may be a "people you follow also follow these people" type of feature, hopefully it won't just be more recommendations based on who Twitter's established leaders like. Or perhaps those of us who want all messages published by our friends can pay for a premium account. I'm ready to do that right now.

Thanks to the super-smart Alex Iskold, who we trust a lot and who helped talk us through this technical speculation.

Update: Hours after we put up this post, Twitter has reversed this policy. The story just keeps getting more interesting!

Even the app's developer agrees that the app crossed the line, overstepping Facebook's much celebrated privacy controls. We're still disappointed though, and we wish that this rich source of data could be opened up for developers and users to build value on top of. What kind of publishing system doesn't offer an RSS feed? A fundamentally closed one.

The argument is that the News Feed RSS made it too easy to violate privacy conditions put on some users' Facebook data.

Facebook hasn't responded yet to our request for an official comment, but Facebook software engineer, Ari Steinberg, explained in an unofficial comment on the Activity Streams discussion group, "We're certainly not opposed to enabling you to export your own content (in fact, we're always trying to work on ways to make that easier), but exporting all your friends' content to a totally public place without their permission isn't cool."

But it was really useful. Sometimes privacy is at odds with innovation, and while we would never want to say that privacy is illegitimate - we're not happy to see it shut down major potential avenues for innovation either.

If the app published an authenticated feed (meaning you had to log in to view it) and if the apps around the web had better support for authenticated feeds, then the story would probably be different. That's not where we're at, though; even the very popular Google Reader can't handle password protected RSS feeds.

App developer, Teck Chia, says privacy issues and copyright violation in the use of the phrase "News Feed" were both cited by Facebook when the app was shut down. Chia understands the privacy concern but hopes to be able to find a solution shortly. One option may be to publish only a user's own items in a feed, perhaps folding in the updates of friends who have added the app as well and specifically opted-in, and perhaps sending items through the feed that say merely that "Your friend John updated his staus, click here to log in to Facebook and read it."

We're not sure how useful those options sound. It's not surprising but it is a real disappointment that Facebook shut the app down. The wall that keeps Facebook user data in and private by default feels too contrary to the fundamental nature of the internet for it to last. In a poll we performed last week, 40% of our readers said they wanted Facebook to open their data either a little or a lot. We're sure the percentage of all Facebook users who feel that way would be smaller, but a closed pocket of the web seems to us to be something that will be worn away in time.

There may not be an RSS feed for your Facebook News Feed today, but it sure seems like only a matter of time until there is.

We're big fans of web standards here at ReadWriteWeb and we'll tell you why.

This year's been a big one for web standards; the President Elect just enabled users to login to comment on his website using the standard authentication protocol OpenID, for example.

So get your blue beanie or similar hat on and make yourself a photo. Your friends will wonder why you and others are wearing them online today and when they ask either you or Google - they'll end up thinking about the importance of web standards as a result.

For more info, visit Jeffrey Zeldman's blog and see if there's a group relevant to you participating, like the South African Web Standards and Accessibility Group, by whom we were reminded of today's event.

During this transition, it's not "old with the old and in with the new" - several technologies from the prior period of innovation are still important. These include the following:

• Service-oriented Architecture (SOA) - middleware that enables new component apps
• Business Process Management (BPM) - user-driven automation of manual tasks
• Mobile - beyond laptops to cell phones and PDAs
• X Internet - RFID and sensors at the edge the net connects physical objects to the internet

New Trends CIO Should Care About

Keeping in mind that the above trends are not going away anytime soon, CIOs still need to be aware of the upcoming trends that will define the future of enterprise IT. In the "IT everywhere" wave, business technology (BT) is the driving factor. The control of this technology is being shifted away from IT and is increasingly under the control of the business organizations and the users themselves. Simply put, BT is the future of IT. When looking ahead to the future, Forrester recommends CIOs keep the following trends in focus:

1. Technology Populism: Web 2.0 and social networking in the enterprise. Workers are provisioning their own tools, especially when IT can't provide. IT had best look towards integrating Enterprise 2.0 into their organization - if they don't, the end users will simply go find their own apps to use. The risks of ignoring this trend include compromised security, comprised privacy, and poor control of intellectual property. (We discussed this concept in more detail here).
2. The Information Workplace: The information workplace is a term describing a next-gen platform that consists of numerous parts such as unified communications, portals, enterprise content management apps, office productivity apps, collaborative technologies, business intelligence, data warehousing, and more. However, the information workplace isn't about each of these technologies individually, but how they all seamlessly come together as a whole. Today's information workplace is role-based, individualized, and thanks to the Web 2.0 invasion, it's also often "social" and "quick," as Web 2.0 tools tend to be.
3. Dynamic Business Applications: These are component apps that target certain roles but change easily. Over the next five years, IT's goal is to develop enterprise software that adapts to the business and that's capable of evolving as the business grows. These apps are designed with a focus on the people who use them, but are also highly adaptable as the business changes over time.
4. Digital Business Architecture: This includes SOA, unified communications, and virtual computing among other things as a top-level conceptual model for planning the future of both technology and architecture. A digital business architecture means the design of your business is accurately reflected in your technology.
5. IT Ecosystems: By 2012, there will be a shift in the dominant form of IT delivery from buyers self-integrating technology to having outside providers assemble and manage it. Those with the strongest delivery capabilities will lead the way. This trend will also include a shift away from software investments based on ownership to those based on subscription as well as an increase in new IP sourced from open communities.
6. Enterprise Master Data Management: MDM focuses on delivering trusted data throughout the enterprise. Today the focus is on addressing cross-application data use and management while also considering MDM's multiyear and multiphase business capabilities. In 2008, information and knowledge management professional will work on overcoming the organizational, process, and business case challenges to bringing this data to the enterprise.

I believe I have finally seen the light as to how the Internet Topic Exchange (ITE) could be made simpler to use and more viral at once, taking one cue from my colleague Stephen's recent ridiculously easy shibboleth strategy for putting together conference aggregators and another from Joshua Shachter's now-defunct reversible.org

1. Right now submitting a post to a channel requires people to either fill in a form on the channel page of their choice or send a TrackBack ping to the channel. Both are simple, but still harder than they should. How about adding a third option: simply link to the channel in your post. To make it easier, the top of each channel page could even provide some standard boilerplate chunk of the requisite HTML.

2. In order for this to work the ITE needs to watch the participating weblogs. It already watches for weblog updates, so just let people register their blog or feed once and let the ITE pick out the posts that link to it as they appear thereafter.

Note that this new option automagically generates visibility for the channels on participating blogs with every post submitted; this is one of the key elements that were missing for effective word-of-mouth propagation of awareness of the Exchange. I'm kicking myself for having taken so long to find a simpler way.

Of course, along with ease of use comes more spam. I've been thinking about this too - more later.

My Reply

Here's what I posted in Seb's comments (tided up a bit):

This is pretty much what I was getting at a month or so ago, only I wanted Topic Exchange and K-Collector to play together more. Ref: Proposed Solution for ENT Topic-Sharing Community and the follow-ups here and here. And this is what I ended up with - K-Collector picks up my posts via my ENT-enabled RSS feed. And my weblog automatically pings Topic Exchange every time I post, via my use of MT categories (which match up with Topic Exchange names).

You propose people include the TE link in their post, which is similar to what I do - only I include the link on a reference tacked onto the end of the post (so it doesn't actually form part of the post itself). This way works for me, because it means I don't have to manually enter the link - MT does it automagically for me. The downside of my method is that I use pre-defined categories, which does limit the topic selection somewhat. But I balanced this by knowing that my topics/categories match now with *both* Topic Exchange and K-Collector, which was what I wanted.

Anywho, good luck with your proposal. If it helps more people adopt TE then it's a good thing.