Efficient XML Interchange, or EXI, is described as a very compact representation of information in XML (extensible markup language). EXI is so efficient that the W3C says it has been found to improve up to 100-fold the performance, network efficiency and power consumption of applications that use XML, including but not limited to consumer mobile apps. It is particularly useful on devices with low memory or low bandwidth.

EXI has been used in commercial contexts for more than seven years, but today's adoption of the format as a formal standard is the culmination years of collaboration between the W3C and 23 different corporate and academic institutions from around the world, including Oracle, IBM, Adobe, Chevron, Stanford University, Boeing, Cannon, France Telecom, Intel, the Web3D Consortium and others.

It's an amazing world where the transmission of large sets of data is costly enough relative to their creation, storage and processing (the price of those has fallen so much already) that industries have a strong incentive to work together to use standards to reduce those data transmission costs substantially.

The creation of a new data transmission standard format is an event of historic importance; it's like a new trans-continental railroad network has been unveiled, but in this case with standard rail-widths primed to make the delivery of all kinds of goods up to 100 times faster and cheaper than ever before. Florida oranges are going to make it to Minnesota for the first time, you might say, but in this century that will be a metaphor for massive sets of real-time data jumping from device to device around the world, enabling the creation and delivery of previously unimaginable products and services made of that data.

Foundational Platforms vs. Market Fragmentation

XML is a relatively open-ended data format that supports the creation of new fields of data, or namespaces, in a standardized and predictable format. The W3C says XML standards are "omnipresent in enterprise computing and are a part of the foundation of the Web." RSS (Really Simple Syndication) is one of many forms of XML, as is XLRB, the XML data format for business data. The standardization of these formats allows data to be shared easily across different applications and devices, without the challenges of translating data from proprietary formats.

The W3C says it's been clear for years that in low memory or low bandwidth situations, basic XML carried too high a cost for data transmission. "Market demand led to the proliferation of application-specific approaches," the Consortium said today, "but most were neither efficient nor general enough, and they sacrificed the interoperability that makes XML so valuable."

To adress that fragmentation of standards, the W3C brought together a wide variety of organizations seeking advances upon XML in industries ranging from smart electrical grids to defense technology to consumer devices.

The editing of the EXI standard has been lead by John Schneier, CTO of a company called AgileDelta, which has been offering EXI-based products for seven years. "They've achieved over 100-fold performance improvements and expanded their data networks to high speed aircraft, automobiles, mobile devices and sensor networks," Schneier says of technologies leveraging EXI. "At the same time, they've achieved dramatic cost savings by using open Web standards and off-the-shelf products in place of the custom protocols, gateways and applications previously required by these applications."

You can learn more about EXI at the W3C's website.

On May 1st South Carolina's Senator, Jim DeMint, officially asked the Senate Rules Committee to make the data available and just four days later the feed is here. Not everyone is happy about about the information being made publicly available like this, however.

John Wonderlich, policy director for the Sunlight Foundation, told Politico that the reason he's been given for the lack of XML feeds is this: "the secretary of the Senate has cited a general standing policy ... that they're not supposed to present votes in a comparative format, that senators have the right to present their votes however they want to...it's pretty bad."

Dave Lundy, acting executive director of the Chicago-based Better Government Association, told Politico again that: "It's a strategy to make information hard to find and hard to digest and hard to analyze...Call me a cynic, but I don't ... think [government entities] deserve the benefit of the doubt. We have ample experience to know that people try to hide information, even in plain sight."

Apparently, those problems were washed away this week by the tides of open technology. The Washington Post has offered something similar to what's now available for some time, but there's something to be said for what we hope will be a big, fat, official pipe of data.

We learned of the news this morning when New York Times technologist, Derek Willis, celebrated mention of the news by Rob Pierson, who yesterday began a new job leading new media initiatives for the House Democratic Caucus. The Sunlight Foundation said last week that neither the House nor the senate "maintain any reasonable database of lawmaker votes." The House of Representatives does release their votes in structured format, though.

Willis points out that the new Senate data feeds aren't perfect; the absence of Bioguide ID information linking Senators' names to their online profiles creates an unnecessary additional step for developers, for example.

It's exciting news none-the-less. "It's good to see high profile senators from both parties behind this," says John Musser, founder of the web's leading mashup and API directory, Programmable Web. "Those first steps are often the hardest. That is, just getting understanding of the value, getting buy-in and then having the data accessible in a developer friendly format. The next logical step is to wrap it in an API; having the XML is closer to having an RSS feed, there's not a lot of developer control of what data to retrieve. An API typically gives much more control over what data gets retrieved. Like 'give me all roll call votes for January 2009', versus 'here's the last 20 roll call votes.' Or all roll call votes by a specific senator, etc."

Musser says that he's seeing a broad movement towards increased access to government data. That work is being done by both official sources like this new Senate feed and the data-centric Recovery.org and by outside organizations like the Sunlight Foundation and the New York Times, work Musser is tracking closely.

What's left to open up? Check out, for example, this list of the 8 most desirable but unavailable government data sets, per Willis from the NYT. As of today, one of those can be checked off the list.