data visualization - ReadWriteWeb http://www.readwriteweb.com/feeds/tag/data visualization en Copyright 2012 Richard MacManus readwriteweb@gmail.com Tue, 14 Feb 2012 18:04:00 -0800 http://www.sixapart.com/movabletype/?v=4.35-en http://blogs.law.harvard.edu/tech/rss Visualizing Famine in the Horn of Africa [Infographic] wfp.pngThe famine eating up northeast Africa and threatening 13 million people is probably something you've seen out of the corner of your eye. A terrible thing, to be sure, but life goes on. Well, for some. Now, the World Food Programme has pulled open data from the United Nations, USAID and their own food distribution program and used mapping technology to enable us to visualize the data involved; to turn it, in fact, from data into knowledge, from data points to human beings and from what to so what. The resulting map is dynamic and easy to understand, if hard to digest.

"In the map you can see what areas are most affected by the famine, where food is being distributed, and how much more funding is needed to meet the demand," said Bonnie Bogle, of WFP's partners, Development Seed, by email. "For example, you see that the most affected areas have limited humanitarian access, as they are in the al Shabab controlled sections of Somalia."

]]> ]]> Discuss]]>
http://www.readwriteweb.com/archives/visualizing_famine_in_the_horn_of_africa_infograph.php http://www.readwriteweb.com/archives/visualizing_famine_in_the_horn_of_africa_infograph.php International Fri, 19 Aug 2011 13:00:00 -0800 Curt Hopkins
Who Uses Google Plus Now? Yep, Male Students & Geeks From the US [Infographic] A few weeks ago, we reported on some demographic information about the first wave of Google Plus adopters. Bime, the data visualization firm who conducted the study, found that early Google Plus users were mostly young American men working in technology (surprise!). The Bime study used profile data from Find People on Plus, a third-party directory of Google Plus users, except for the age numbers, which were pulled from comScore numbers.

Bime has just put out an updated visualization that breaks down Google Plus demographics including the month of August, now that the service has had some time to grow. This survey covered 10 million users, more than twice the size of the previous one, and some things haven't changed. About 70% of Google Plus users still identify as men, and the vast bulk of them are American. One major shift has taken place, though: While the updated post doesn't have the age numbers (which came from a different dataset last time), the occupation data show that students have overwhelmingly displaced tech workers, though all the same tech jobs as before dominate the rest of the top spots.

]]> gplus_gender-1.pngBime's findings about the student takeover conflict with some research we've covered since the first Bime study, but the methodology of this Experian Hitwise study was a little strange. It tracked 10 million people by rather different measures, creating "word clouds" to identify demographics by the descriptor words they use for themselves. It found that Google Plus usage in its "Colleges and Cafes" demographic has declined significantly since launch, while "Seeking Singles" and "Kids and Cabernet" -- defined as "Prosperous, middle-aged married couples living child-focused lives in affluent suburbs." -- have increased.

gplus_occupation-1.pngThat would seem to totally contradict what Bime found. Bime uses self-reported data from user profiles, though, and this Experian Hitwise study, to the extent it makes sense at all, uses some highly advanced semantic analysis where simple profile fields would seem to suffice. So let's assume that the huge growth in people calling themselves "students" is more believable than the decline in people using a cloud of words some algorithm defines as "student-y."

gplus_inactive_updated.pngTwo other trends jump out from the new data. User growth is steady and more or less linear week over week, but 83% of users in this study are classified as inactive. Bime doesn't know how this is defined, so we've asked Find People on Plus for comment and will update with the response. Find People on Plus has admitted that its directory can only see public posts, which makes this measurement unjustifiable, and it is renaming this data field as a result. If it's true, though, that means that only 2.5 million people from the sample are really using Google Plus. Furthermore, Bime admits that its own interpretation of the data was unclear. Out of this voluntary sample of 10 million, only 2.5 million are using the service. Neither of these studies seems dependable enough to draw conclusions about overall activity on the service. It was a voluntary sample, though it was still a large one, and the respondents were mostly male students and geeks (which, for the record, is a term of affection at ReadWriteWeb).

Are you on Google Plus? Has your use declined since you joined? Let us know in the comments.

gplus_users-1.png

]]> Discuss]]>
http://www.readwriteweb.com/archives/who_uses_google_plus_now_yep_male_students_geeks_f.php http://www.readwriteweb.com/archives/who_uses_google_plus_now_yep_male_students_geeks_f.php Google Fri, 19 Aug 2011 00:00:00 -0800 Jon Mitchell
Who Used Google Plus First? Male Geeks From the US [Infographic] Many words have been expended covering user demographics on Google Plus, mostly regarding whether or not the newborn social network is dominated by men. The data visualization wizards at Bime have just posted an interactive dashboard of Google Plus data that gives us a much more granular picture.

]]> gender.jpg

The data seem to confirm the popular conception that Google Plus users overwhelmingly identify as male (71.24%). The dominant age bracket (35%) is 25-34. The U.S. is by far the most represented country on Google Plus, with roughly three times the user base as India. The top 10 most common occupations of Google Plus users are dominated by tech jobs, with engineers (1.77%), developers (1.02%), designers (0.82%) and software engineers (0.72%) taking the top spots. And, by a wide margin, the top employer of Google Plus users is Google itself.

jobs.jpg

Bime pulled most of their data from Find People on Plus, a third-party member directory. They analyzed a sample of 4,412,227 users, which, according to estimates, is probably over one fifth of the entire user base. The age distribution data is from comScore, not from Find People on Plus, so the sample is not the same.

Bime admits that the data is "a few weeks old," so demographics may have shifted. As we noted last week, user behavior on Google Plus appears to be changing. But Bime's dashboard is still probably a reliable picture of who the early Google Plus adopters are.

google_plus_dashboard.png

]]> Discuss]]>
http://www.readwriteweb.com/archives/who_uses_google_plus_male_geeks_from_the_us_infogr.php http://www.readwriteweb.com/archives/who_uses_google_plus_male_geeks_from_the_us_infogr.php Google Mon, 01 Aug 2011 12:43:24 -0800 Jon Mitchell
Infographics: Anticipation Builds For New Visualization Tool & Community visuallylogo.jpgPeople love data visualization; when done well, it communicates new knowledge about otherwise inaccessible information in new and pleasing ways. Good visualizations can be efficient and effective; bad visualizations can be seductive and deceptive. That's why visually designed data-based content, both good and bad, is so popular online. People love infographics.

Today a startup called Visually drew back the curtains a little bit further on its data visualization technology and community. The company unveiled an index of 2,000 data visualizations, a cute Twitter visualization creation tool and promises to help anyone create their own visualizations with a series of self-service tools to be released throughout the year. Interest in the service is so heated that the Visually website melted early this morning.

]]> visuallyscreen.jpgThere are a number of services like this online already, but two things make Visually stand out. First, it appears that in the phrase data visualization, the company may focus more on the visualization than on the data. The gallery of infographics is beautiful. It appears heavy on color, text, fonts and design - more than being heavy on graphs or more complex ways of communicating deeply nuanced insights into information. That's what some other competitors focus on.

That's consistent with where the founders of the company came from: Mint.com. That wildly successful personal finance service gathered consumer finance data and then drew big picture conclusions about the state of the economy and consumer behavior. The resulting data visualizations were beautiful and a key part of driving consumer interest in the company's services. That consumer interest helped the company get acquired by finance giant Intuit in 2009.

Visually's co-founders are both from Mint; CEO Stew Langille was a marketing exec there, and Lee Sherman was the company blog's editor. (Disclosure: I actually recruited Sherman for that position as a part of the small consulting practice I do on the side, though before I was here at ReadWriteWeb.)

Mint reportedly didn't create its own infographics in-house; it gathered the data sets, then hired independent designers to visualize them. Then those infographics blew up on sites like then-hot social news site Digg and now-hot Reddit.

That strategy was parlayed into a great reputation for data visualization on the part of Langille and Sherman. Now they are working on a tool that will allow anyone to create such creative work for themselves. Visually also promises a search engine that will index "all existing Web-based data visualizations" (how they will be ranked and sorted, we don't know) and data warehouse that will help seed the visualizations of their customers.

The company has told press that it will charge power users $100 to $300 per month to use its tool and has said that it could "easily" be billing $300,000 per month just from client work it has secured, without launching any public tool at all.

If that's the case, it's not clear to me why the company is going to mess around with a long tail of customers at a low price point. Are there tens of thousands of organizations that are in a position to create valuable infographics, even with the help of the easy and effective tool that Visually is promising? I'm not sure. I'm also not sure that a demonstration app at launch that makes cute Twitter avatars based on the contents of your tweets are the most effective content marketing for a B2B service, but maybe it doesn't matter: the lust for infographic power is very, very strong.

As a result of that, there is no longer a shortage of other services online that promise to make it easy to create data visualizations. How Visually will stack up with ManyEyes or Tableau, much less visualization specialists for hire like JESS3 (apparently a Visually launch partner) or Bloom remains of course to be seen. Visually hasn't launched much so far, though 2,000 examples of infographic inspiration are certainly valuable.

It's widely believed that a tidal wave of value-laden data is just about to crash over the top of the Web and the world, though. As it does, huge pockets of opportunity should appear for new service providers to offer new forms of value. Data visualization is going to be a big area of opportunity for services like Visually and for their customers.

]]> Discuss]]>
http://www.readwriteweb.com/archives/infographics_anticipation_builds_for_new_visualiza.php http://www.readwriteweb.com/archives/infographics_anticipation_builds_for_new_visualiza.php Publishing Services Wed, 13 Jul 2011 05:56:07 -0800 Marshall Kirkpatrick
Your Neighborhood Data Visualized: Startup Builds Census Map Block by Block moonshadowlogo.jpgThe 2010 US Census has begun publishing its detailed demographic data state by state and the race now begins to see which data geeks can do the coolest things with the information. Remember when large-scale social data was only collected once a decade? When terms like "social graph" and "interest graph" didn't even exist?

It turns out that old fashioned data still has a lot to teach us. One company has already launched its first block-by-block, state-wide data visualization site for New Jersey census data. Want to see hyper-localization and personalization in action? Check out this map from MoonShadow Mobile.

]]> moonshadowcensusmap2.jpg

MoonShadow Mobile specializes in working with large datasets and making them easy to navigate around in quickly. The company's service allows anyone to check out race, age, gender and other information from eight different data sets about any particular block in New Jersey. New states will be added as Census Data is released to the public. The interface is a little challenging to learn, but it may make up for that with its feature set and customizability.

MoonShadow makes its living selling similar software to public agencies and others interested in drawing redistricting lines or in municipal planning. Want to know if a location might be a good place to put a new park? A service like this can help you determine how many children live in the area.

Other data visualization projects built on the US Census include a less granular but more visually dazzling effort from well-known data desingers Stamen Design. The New York TImes has made a go at it as well; the Times does every block nation-wide, using estimated numbers.

Thanks to last week's release of the Google Public Data Explorer, anyone can now do many things like this with big sets of public data.

Bring on the census visualizations, folks. The visualization, cross-referencing and subsequent insights enabled by this latest census are likely to far surpass what the tech world was able to do with the 2000 US Census. Just imagine what magic will be performed with the census data of forthcoming decades.

If, that is, Facebook and Twitter data don't outshine anything the governement has been able to collect before. Data collection every ten years does seem a little old-fashioned, doesn't it? None the less, let's see who can top the quality of Census data. This data clearly remains very valuable.

]]> Discuss]]>
http://www.readwriteweb.com/archives/your_neighborhood_data_visualized_startup_builds_c.php http://www.readwriteweb.com/archives/your_neighborhood_data_visualized_startup_builds_c.php Data Services Tue, 22 Feb 2011 12:27:52 -0800 Marshall Kirkpatrick
Visualize Your Data with Google Public Data Explorer

The Google Public Data Explorer takes large data sets and makes them palatable for public consumption, taking numbers, figures and other data and turning them into bar graphs, line graphs, maps and bubbles. When the explorer launched last year, it started out with 13 data sets. The number of data sets has more than doubled since then and it's about to get a lot larger.

Today, Google announced that it would be "opening the Public Data Explorer to your data," which means you can upload data and use Google's explorer tools to explore it visually.

]]> Omar Benjelloun, technical lead with the Google Public Data Team, writes that Google hopes "more datasets can come to life through Public Data Explorer visualisations and enable people to better understand the world around them and make more informed, data-driven decisions."

According to Benjelloun, the opening of Google Public Data Explorer comes with a new data format, called the Dataset Publishing Language (DSPL), an XML-based format, which data must be packaged in to use the explorer.

"Once imported," writes Benjelloun, "a dataset can be visualized, embedded in external websites, shared with others and published."

Google isn't the only one in the big data game, of course, and it's also not the only one attempting to make it simple for others to take part. There's a growing competition from sites like Factual, CKAN, InfoChimps and Amazon's Public Data Sets. Another thing to note with Google's implementation is that it is not a two-way road. Although uploading data to it's Public Data Explorer allows you to visualize that data, it isn't a way to share data sets. Google says that downloading data is something it's looking into, but that for now "you should be able to follow the links to data provider websites and download the data there."

If you're curious what Google Public Data Explorer is capable of, take a look at the embedded graph below. It's fully interactive and is created off of a public data set.

]]> Discuss]]>
http://www.readwriteweb.com/archives/visualize_your_data_with_google_public_data_explor.php http://www.readwriteweb.com/archives/visualize_your_data_with_google_public_data_explor.php Google Wed, 16 Feb 2011 12:12:47 -0800 Mike Melanson
Word Cloud 2.0: The Spatial Dimension Word clouds: No doubt you've seen these graphic representations of the most commonly used words in a body of text, floating around the internet. They are especially popular after big political speeches. Thanks to IBM researcher Jonathan Feinberg's web site Wordle.net, word clouds are easy for anyone to create. To draw a loose analogy: Wordle has been to text analysis what Blogger or Facebook has been to online publishing - great tools to democratize what used to be an elite skill. Are there substantial limitations to this word cloud format that need to be taken into consideration, though?

New York University PhD student of political science Drew Conway thinks there are. Conway hosted an interesting debate on his blog this week about one of the key concerns about word clouds and he offered an alternate model for understanding bodies of text. He calls word clouds "spatial visualization wherein space is meaningless." That's hard to argue with. Check out one of the models he proposes as a possible next step of the word cloud's evolution.

]]> wordcloudvdeux.jpg

Above, Conway's visualization of words used by both President Obama and Sarah Palin in their speeches about the shooting of Congresswoman Giffords in Tucson. On the left, words used by both but more by Palin, on the right, words used by both but more by Obama. Click to view full size.

The big picture here is that space, both the x and the y axes, are opportunities to convey something important.

Conway explains what he did:

To understand how these speeches compared I first needed to create a term-frequency matrix, which contained only words used in both speeches. After removing common English stop words and the word 'applause' (Obama's speech was in front of a live audience), and retaining only words contained in both speeches at least once, I was left with 103 words to visualize.

To show how the two speeches contrasted, I decided to use the x-axis position to pull words used more by one politician closer to either the left or right of the plot. Words used more by Palin are to the left, and likewise words to the right were used more by Obama. The color reinforces this information, making words Palin words darker red, and Obama darker blue...
While this is a very simple extension of the traditional word cloud, much more can be learned from it. For example, both politicians used the words "congresswomen," and "america" equally but also frequently. While the word "tragedy" is used often in both speeches, but slightly more by Obama. The edges are most interesting. Palin repeated the shared terms "ideas," "debate," "victims," "values," and "strength," while Obama focused on "people," "lives," and "life."

Conway acknowledges that this visualization has its own limitations, specifically that the words not intersecting between both speeches aren't represented here. He posts the Wordle charts for both speeches as well in his blog post.

See also: Mapping Our Friendships Over Time and Space: The Future of Social Network AnalysisI like Conway's thinking, both in general and with the specific way he's made use of space in his version of the word cloud. What do you think? Are you familiar with other efforts to take space into consideration when visualizing word frequency in bodies of text?

Nielsen's Keith Stewart points, for example, to that company's word association visualizations using concentric circles to represent associations across Twitter, blogs and Google search results. Stewart says that accurate sentiment analysis is a painstaking process with high overhead. This kind of association analysis seems much more scalable to me, though, if cruder.

What are your thoughts, readers, on how visualization of text analysis could evolve to become more effective?

Disclosure: IBM, the company where Wordle was in large part developed, is a ReadWriteWeb sponsor. We think that's pretty cool.


]]> Discuss]]>
http://www.readwriteweb.com/archives/word_cloud_20_the_spatial_dimension.php http://www.readwriteweb.com/archives/word_cloud_20_the_spatial_dimension.php Analysis Fri, 28 Jan 2011 08:36:39 -0800 Marshall Kirkpatrick
Beautiful Data Visualization Special Coming to BBC, Too Bad It's UK Only (Video Preview) rollingvid.jpgThe BBC will broadcast an hour-long special next Tuesday called The Joy of Stats and it looks great. Hosted by Hans Rosling, Professor of International Health at Karolinska Institute and Director of the Gapminder Foundation, the show will use some beautiful computer generated graphics to demonstrate the importance of data and data visualization.

]]> Unfortunately, the show is expected to be shown only in the UK, due to BBC licensing practices. (It's on BBC 4, UK only.) That's a real shame, because as is so amply demonstrated by the preview video of the show - statistics, data and data visualization are matters of global importance. As the world produces more and more data than ever before, every day, that will only grow more true.

Thanks to Nathan Yau, author of the great blog Flowing Data, for posting this video in the first place we saw it.

See also Rosling's widely acclaimed TED Talk video and Emily Cunningham's in-depth review of Gapminder's DIY data desktop data visualization tool here at ReadWriteWeb from August.

I think this video also points to arguments similar to those I make in favor of aggregate social web user data being made available for programmatic access and analysis: data about our world offers new opportunities to unearth injustices, shine light on opportunities and gain a new level of social self-awareness. Let's set that data free!

]]> Discuss]]>
http://www.readwriteweb.com/archives/beautiful_data_visualization_special_coming_to_bbc.php http://www.readwriteweb.com/archives/beautiful_data_visualization_special_coming_to_bbc.php News Wed, 01 Dec 2010 00:40:49 -0800 Marshall Kirkpatrick
Setting Data Free With Gapminder gapminder_logo.pngLast month Hans Rosling, the Swedish global health professor, statician and sword swallower released a desktop version of Gapminder World, his mesmerizing data visualization tool. Named one of Foreign Policy's top 100 global thinkers in 2009, the information design visionary co-founded Gapminder.org with his son and daughter-in-law aiming to make the world's most important trends accessible and digestible to global leaders, policy makers and the general public.

The software they developed, Trendalyzer, (acquired by Google in 2007) translates static numbers into dynamic, interactive bubbles moving through time. The desktop version of Gapminder, which is still in beta, allows you to create and present graphs without an Internet connection.

]]> Emily Cunningham is a research intern at ReadWriteWeb and a design and user experience intern at OpenMRS.org. She is pursuing a Master in Information Management at University of Washington in Seattle, WA. Emily is continually fascinated by the way social technologies are enabling collective action in new forms and on a scale we've never seen before. You can contact her at emilykcunningham@gmail.com or via Twitter: @emahlee.

What is Awesome

Many of same things that make the Web-based version of Gapminder a great tool applies to the desktop version:

  • Graphs are highly dynamic and yet easy to understand and create. Presenting data as animated colored dots allows you to show the relationship between sets of information at a level of complexity that's impossible in your typical graphing program.
  • The interactive nature of the graphs lets users' curiosity lead them to more discoveries about the data. You might, for instance, click on one or two dots representing countries and all the other country dots fade in the background. You can then compare the life expectancy and income per person of say, India and Vietnam.
  • Accessing the raw data for the graph is as easy as clicking a button.
  • The example graphs are fun to browse and play with. For instance, "USA or China, who emits the most CO2?" and "Who has the most Internet users?"

Benefits specific to Gapminder Desktop include:

  • For presentation purposes, having the ability to bookmark graphs for easy reference is invaluable. (And with automatic software you never have to worry if your data is out of date.)
  • Obviously not needing a Internet connection is a big bonus. Especially if you're giving a lecture in a resource-constrained part of the world with unreliable Internet connectivity.

Definitely don't miss Rosling's screencast (below) explaining how to use the application. Bonus: He gives five good tips on how to give a successful presentation with the software. Double bonus: Rosling is an endearing, quirky guy and is entertaining to listen to even in a "how to" screencast. (See his answer to "What's it like knowing so many on reedit (sic) have intense nerd crushes on you?" in this screencast for more Rosling goodness.)


What Needs Work

In a word: social. I would love to see the most shared and viewed graphs as well as new graphs and the people who made them. Peering in on conversations to see how people are using the graphs in their own work would be interesting and might help spark more ideas on how to use the content. I'm thinking of the way Wordnik displays the most recent tweets of a given looked-up word to give users more context to its meaning. I can also imagine people discovering one another as well as important discussions and collaborations coming out of a more social Gapminder.

Right now, Gapminder World has a "share" button, but all it does is give users a shortened bit.ly URL instructing them to "copy the short link below and paste it in a blog or send it by e-mail." (Though not necessarily in real-time, you can imagine the desktop version of the app getting the same information as the website version.)

Another benefit to having more social information displayed: It gives users more hooks to play with the tool. Yes, the data is very interesting. But with over 600 indicators, I found myself not going as deep into the data as I probably could have; it just got overwhelming at some point. Seeing the titles of other people's graphs probably would have kept me tinkering around a bit longer. I played around with the pre-made "example" graphs more than making my own graphs, in fact.

The Larger Picture

You won't comprehend the full power of Gapminder unless you watch at least one of Rosling's TED talks. His passion and style makes him a joy to watch. As TED notes, "With the drama and urgency of a sportscaster, statistics guru Hans Rosling debunks myths about the so-called 'developing world.'"

Rosling is a master at understanding global statistics and communicating the underlining trends they reveal. But what I find most admirable about Rosling is his commitment to making important information and insights understandable by everyone. Freeing public data from database hell so that it can be used by experts and laypersons alike is one of his goals. It was also the impetus behind creating Gapminder.

In a Gapcast titled "Free Statistics for Democracy," Rosling explains in his typical humorous style that his target audience consists of "children below 12 and heads of state. Because they both, they don't want people to show them boring graphs, you know and pretend. They want to see for themselves."

While children and global leaders are on his target list, his real audience is you and me: "The main decision makers we have aren't government, it's not policy makers; it's the public... Decision making, policy setting is done by the public in a democracy. So we have to change. Statistics means bookkeeping of the state. Serving the decision makers within the state."

]]> Discuss]]>
http://www.readwriteweb.com/archives/setting_data_free_with_gapminder.php http://www.readwriteweb.com/archives/setting_data_free_with_gapminder.php Data Services Sun, 22 Aug 2010 13:00:00 -0800 Guest Author
Paper.li Gets Investment for Its "Twitter Newspapers" paperli-logo.pngOpportunities to visualize data can turn the theoretical into the actual. Even so, many tools and services that do so are useful mostly to professionals - academics, economists, business people. Paper.li, a young Swiss company that turns a user's Twitter links into a newspaper-like Web page has been spectacularly successful at doing so for the lay person.

We're not the only entity to think so, apparently. A scrum of investors lined up behind the company, including Kima Ventures, whose co-founder, Xavier Niel, recently bought what is, arguably, France's best-known newspaper, Le Monde.

]]> Like TwitterTim.es, Paper.li harvests a user's Twitter account for links, then presents hints of what they link to in an interactive broadsheet format. New "editions" are generated every few days. It seems to be a remarkably able expression of the concerns and interests of a user, a kind of intellectual and social snapshot of what the user is doing and thinking. (Apparently this reporter is a reasonably steady reader of his own news site. But he also digs comedians, Berlin, business and is keeping an eye on the Gulf oil spill.)

The investing team is filled out be German group Econa, Jeremie Berrebi and a handful of angels.

The company says it was caught flatfooted by what it called an immediate and sudden uptake in users that it has spent time on stabilizing the back-end. It will now turn to "generic new functionalities" and to supporting new languages.

paper.li_screenshot.png

]]> Discuss]]>
http://www.readwriteweb.com/archives/paperli_gets_investment_for_its_twitter_newspapers.php http://www.readwriteweb.com/archives/paperli_gets_investment_for_its_twitter_newspapers.php Microcontent Wed, 30 Jun 2010 16:54:00 -0800 Curt Hopkins
Data Explosion: Analytics Software Must Adapt or Die In my previous few articles, I've explored the potential impact of sensors on the Internet. Soon there will be a trillion sensors connected to the Web, which will result in an explosion of online data. How will this mass of new and mostly real-time data be processed and analyzed? Will current data analytics software be able to cope? The short answer is, no it won't. New types of analytics software will be required, together with much more powerful computers.

During my visit to HP Labs last month, I sat down with Meichun Hsu - director of the Intelligent Information Management Lab at Hewlett Packard - to discuss this issue. Hsu has been researching new real-time, sensor analytics solutions for the coming Internet of Things era.

]]> Hsu told me that sensor data, along with other Web data such as feeds, should be managed as enterprise assets. There is a large amount of data, she said, and it needs "integrated management."

She also thinks that analytics and the compute function should be brought "closer to the data." By this she means breaking down the walls between data and analytics, barriers such as CRM systems and similar structures traditionally used in enterprise data management. She noted that enterprise information management and business intelligence systems have not changed much in the last 20 years.

In terms of actual analytics, we will see changes such as new methods of visualizing data and correlating patterns in it.

Processing needs to evolve too. The emerging era of the Web, said Meichun Hsu, will bring not just an increasing amount of data - but an increased rate of data. HP expects the data rate to increase by a factor of about 10,000 times in the near future, due largely to sensors. This will require massively parallel processors to analyze, Hsu said. HP is looking for a 10,000 times increase in the ability to crunch data.

As for some of the potential commercial applications that might arise out of this advanced data analytics, Hsu talked about the impact on retail as one example. A consumer might be able to scan a tag on a product and not only get information back about the product, but combine that with data about who you are and your preferences. Retailers will be able to do advanced targeting, real-time micro campaigns, and more. This type of functionality requires real-time analytics on a massive scale.

Let us know in the comments about the new forms of data analytics that you are seeing and how you think data analytics will evolve.

]]> Discuss]]>
http://www.readwriteweb.com/archives/data_analytics_software_must_adapt.php http://www.readwriteweb.com/archives/data_analytics_software_must_adapt.php Real-Time Web Wed, 02 Jun 2010 00:30:29 -0800 Richard MacManus
Current: Meme Tracker With Data Visualizations While in New York earlier this month, I attended New York University's annual ITP Spring Show. ITP is a graduate program for communications studies and the Spring Show is a chance for students to showcase their interactive projects. I saw everything from Matrix-like interactive squiddies, to a woman on stilts powered by an iPhone app, to a paint brush that made music.

Probably the most impressive thing I saw, though, was a media project by a student named Zoe Fraade-Blanar. Current: A News Project is a prototype meme tracker using data visualization.

]]> Current tracks the lifecycle of internet memes over 24 hours, using data visualization of U.S. news media coverage. It makes it easy to identify when a meme is starting, trending, at its peak, or dying. The tool is aimed at newspaper editors and writers, as it tells them which stories have the best chance of success.

The underlying data of Current comes from Google Trends, in particular the Hot Searches section. So it is ultimately powered by Google search queries. That data is cross-referenced with the five largest newspapers in the United States, using Google News.


"...it is not the placement but the thickness and volume of each line that represents the fluctuating level of interest."

Creator Zoe Fraade-Blanar has a noble goal for the project. In an explanatory paper, she explains that the goal is to "spotlight missed opportunities in news coverage, and, potentially, recover news readership that has been lost to more sensational sources." Current is Fraade-Blanar's third attempt to use data visualization "to tackle the question of what topics news media should be covering."

Fraade-Blanar did a stint with the New York Times Analytics Group in the summer of 2009, helping to analyze incoming traffic behavior. She discovered then that it often wasn't 'hard news' like politics or economics which drew the most traffic, but 'soft news' such as celebrity gossip or entertainment news. Here at ReadWriteWeb, we're well aware of this issue. Some of our best posts are ones that our writers sweat over for hours, yet probably won't hit the Digg frontpage because of the topic. Whereas a top 10 list about YouTube videos can draw hundreds of thousands of visitors!


"Each of these circles is a news item. So every time you see a circle inside this information stream, that's a successful news item." - Zoe in an On The Media interview.

While Current has a noble aim, curiously it's the exact same goal that motivates content farms like Demand Media and the newly Yahoo-acquired Associated Content. In the words of Fraade-Blanar's white paper: "Each highlighted meme represents untapped traffic that could be channeled to a specific news site if only an article existed on the topic."

As we've noted before on ReadWriteWeb, content farms aim to supply content to meet demand - usually perceived to be what people are searching for on Google. Demand Media, for example, has a sophisticated analytic engine that identifies topics that will be most attractive to Google.

So although Current aims to mediate between a writer's "need to drive traffic to [their] website and the need to cover important, albeit less sensational topics," in reality it would most likely be used to identify high traffic topics (the ones that will make money).

Regardless of how it might be deployed in the real world, Current is a beautiful tool that nicely illustrates trending stories on the Web. Although currently it is based on Google Trends, it may add other sources like Yahoo, Technorati, Twitter and Bing.

Congratulations to Zoe Fraade-Blanar for a smart and well-designed project! There is definite potential for Current to become a useful commercial product. Check it out for yourself on the project website.

]]> Discuss]]>
http://www.readwriteweb.com/archives/current_meme_tracker_with_data_visualizations.php http://www.readwriteweb.com/archives/current_meme_tracker_with_data_visualizations.php New Media Wed, 19 May 2010 23:00:03 -0800 Richard MacManus
ReadWriteWeb and Tableau Announce Winner of Data Visualization Contest tableau logoReadWriteWeb and Tableau are pleased to announce the winner of the Tableau User Generated Graph Contest: Rina Bongsu-Petersen and her interpretation of U.S. obesity data (see below).

The judges - Marshall Kirkpatrick, ReadWriteWeb's co-editor; Stephen Few, a leading data visualization expert; and Jock Mackinlay, Tableau's director of visual analysis - found the entry to be not just a powerful tool, but also an indicator of how easy-to-use data visualization is changing the world.

]]> "This entry was able to provide strong analysis with a view of the data that fits the subject, and the result is an incredible story anyone can discover," Mackinlay said. "People will look at it, immediately select their state and see relevant results."

Kirkpatrick sees the contest in a broader context:

"Judging this event, seeing data visualization projects from around the world, was a whole lot of fun. I believe that data is a key platform for the future, and stories drawn from data could become one of the next big forms of DIY publishing. Just like blogging changed the world by making text publishing easier than at any other point in history, then YouTube enabled almost anyone to become a video publisher, and then social networks made it simple to put all kinds of content online - so too will other types of content get brought to life by simple publishing tools that will change the world.

"It was an honor to get to judge what I'm sure will be just the first of many of these kinds of contests. Look out Internet, data visualization is leaving the confines of experts and becoming another tool that any of us can use to change the world."

Rina received more than $3,500 in prizes, including a free trip to Web 2.0 in San Francisco from May 3-6.

Editors Note: This post is part of a series ReadWriteWeb produced in partnership with Tableau Software where we examined interesting data sets relevant to technology trends today. Tableau Public is a free service that lets anyone publish interactive data to the Web in interesting and compelling graphs. Download Tableau Public and you can create interactive graphs, dashboards, maps and tables from virtually any data and embed them on your website or blog in minutes. Once on the Web, anyone can interact with your graph and the data. They can re-embed your work, download the data, or create their own visualizations. Check out Tableau's gallery to see some of the cool graphs bloggers have created. Or learn how to do it yourself in this five minute video.
]]> Discuss]]>
http://www.readwriteweb.com/archives/readwriteweb_and_tableau_announce_winner_of_data_visualization_contest.php http://www.readwriteweb.com/archives/readwriteweb_and_tableau_announce_winner_of_data_visualization_contest.php Sponsors Mon, 26 Apr 2010 13:48:00 -0800 Admin
More Twitter Analysis: Influencers Don't Retweet twitter logoAs part of our ongoing series in conjunction with Tableau Software and TweetStats, we've been looking at some of Twitter's most influential users. What it means to be a "top" user on Twitter has changed a lot since 2007. Recently we found out that a high number of followers, which most people use to judge the popularity of an account, doesn't actually really mean anything. However, Twitter lists - where we identify our favorites as "most influential" or "essential" - are still revealing. Today we're using lists to unscientifically analyze what we think are some of the some of the most influential people and entities on Twitter.

]]> Editors Note: This post is part of a series ReadWriteWeb is producing in partnership with Tableau Software, where we examine interesting data sets relevant to technology trends today. You can use Tableau Public to create interactive visualizations like this and publish them to your own blog or website or anywhere online. This is the last week to enter Tableau's User Generated Graph Contest. Winner will receive a free trip to Web 2.0 and $500. Sign up before March 26.

We used a variety of lists to identify 485 accounts and then ran those names through Tableau's data visualization tools. What did we find? Surprisingly, the majority of influencers retweet fewer than 30% of the time.

Play With the Influencers

The top graph shows the distribution of total tweets from all of our influencers. You can play with the data yourself. When you select an account on the right you can see they fall in the larger @, retweet and hashtag trends. The data is from January and February.

The Data

Thanks to TweetStats for providing the Twitter data.

]]> Discuss]]>
http://www.readwriteweb.com/archives/more_twitter_analysis_influencers_dont_retweet.php http://www.readwriteweb.com/archives/more_twitter_analysis_influencers_dont_retweet.php Analysis Tue, 23 Mar 2010 06:00:00 -0800 Abraham Hyatt
Widgenie Graph Maker is Fast, Free and Easy widgenielogo.jpgMaking charts and graphs might not seem like an exciting way to spend your time, but new service Widgenie manages to make this common task relatively enjoyable. The service allows you to upload spreadsheets and create nice looking visualization widgets that can be embedded on web pages.

You might not make graphs very often, but the next time you do - this service is worth a look. There are a lot of graph making tools online, but few are as easy to use and attractive as this one.

]]> Data Sources

Currently, Widgenie users are limited to uploading Excel or CSV files. In the near future the company says it will support graph creation from 3rd party data sources, including dynamic ones like Google Spreadsheets. The company says this functionality is likely to become available in the final months of this year.

Obviously my little sample spreadsheet isn't particularly well suited for a pie chart, but the chart is attractive enough, is it not?

Output

The service offers six different types of widgets: Tabular, Animated Line Chart, Animated Bar Chart, Animated Pie Chart, Animated Area Chart and a Text Cloud Widget. The widgets are very easy to configure and are embedded using Javascript. Further customization options would be good to have, the aesthetics of the graphs are relatively unchangeable.

We had some file uploading problems the first few times we tried the service, but it appears to be working now. The embed code for the primary widget is broken, we used the code for Blogger, but the company says that's a bug that will be fixed later this week. If everything about the service worked like it ought to then it wouldn't be a startup, would it?

Analytics

One of the most charming details here is the provision of viewership analytics. It's pretty simple, but Widgenie shows you how many viewers your widgets have had, from what sites and when. That's a pretty handy and fun feature for a free service.

Context

There is so much data flying around on the web that good tools for visualizing it are important. That's why we said at the start of the year that Data Visualization was going to be a key trend in 2008. We're not sure how well that prediction has played out, but it does help us appreciate the need for good services like Widgenie.

]]> Discuss]]>
http://www.readwriteweb.com/archives/widgenie_graph_maker_is_fast_f.php http://www.readwriteweb.com/archives/widgenie_graph_maker_is_fast_f.php Product Reviews Tue, 12 Aug 2008 11:43:39 -0800 Marshall Kirkpatrick