The Future of Social Media Monitoring
Ten years ago the ClueTrain manifesto said that "markets are conversations" but today a more pertinent statement could be that conversations are becoming markets - or that there's a market for monitoring conversations. A whole class of technologies are emerging to help companies keep track of the conversations exploding online.
The web moves fast enough that we may as well start looking at what comes next. Easy to use and affordable tools like Radian6 and ScoutLabs that track blog and twitter mentions are a given - but what kinds of crazy innovations can we hope for in the future?
To be sure some of the future will be frighteningly intrusive and creepy; we've argued that the present state of the art in social media monitoring already is. As lovers of technology, though, we're also excited to see what companies like this enable in the future. If web 2.0 was all about democratizing publishing, then the next stage of the web may well be based on democratizing data mining of all that content that's getting published.
Here's what we're thinking that might look like. We base these predictions in part on a conversation with Chris Golda of social media discussion search engine Backtype. Backtype is used under the hood at Radian6 and the company just announced a new partnership with Filterbox today. For an in-depth look at some of these kinds of services, see Social Media Monitoring Grudge Match: Radian6 vs. Scout Labs by Jason Falls. In house we use FriendFeed search a lot.
None of our predictions for the future are terribly shocking; to some degree they are just further developments of the same core value propositions these services already focus on. Hopefully this discussion will help spur some original thinking on your part, readers, and you can throw some suggestions for the future of the industry out in comments.
More Sources, Especially Facebook
The most obvious place for social media monitoring tools to go next is into Facebook. Twitter is the primary target for analysis right now because it's wide open and programmatically accessible. It's also 10% the size of Facebook. We're not sure whether Facebook is going to open up further or if monitoring tools are going to find a way to get around the fundamentally closed nature of the site to at least sample the conversations going on there. Can you imagine a monitoring company paying a wildly popular Facebook user to open up access to their newsfeed to monitoring? We can; it's a surprise that it's not already happening.
There's also a lot of multimedia to monitor, including television and radio content that is increasingly easy to find online. We expect that speech-to-text search services like Everyzing will find a place in the social media monitoring technology toolkit in the near future.
Source Analysis
Backtype's Chris Golda says that his company is working on more sophisticated analysis of participants in discovered conversations. If Robert Scoble says something bad about your company in comments, that's probably going to go further than if someone with a relatively small circle of connections does.
This is something that all the monitoring companies do now but it's relatively crude. We can imagine a much more sophisticated analysis. For every person online - primary circles of friends, semantic analysis of areas of interest and personal background information are all readily available in tools like Mailana, Twazzup, Calais (disclosure: RWW sponsor) and Headup. Imagine that information served up in an interface like Apture, but for every person and conversation on the web.
This is the kind of thing that data portability makes easier - when people travel around the web with their profiles and data with them, they are easier to get to know in a hurry. That's true for both the technologies that would serve up personalized content and the companies that would monitor what we do.
Conversation Discovery
If it's all about "joining the conversation," one of the biggest challenges for organizations jumping into social media is finding and prioritizing all the conversations available. Golda says that Backtype is working on becoming "like Compete for communities." Postrank is doing something like this already; it's discovering the most "engaging" blogs on a wide variety of topics. It's still ridiculously difficult to identify the most important sites of conversation on a given topic though. That's a problem ripe for solving.
We'd also like to see some real-time threshold monitoring. Let me know the moment that conversation about me or a given topic hits a certain level of intensity.
Those are some of the things we expect to see come to the world of social media monitoring in the near future. What else do you expect - or what would you really like to see?
Share
Facebook
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
Do you think the value of what's being said about brands online will diminish over time? The reason I ask is because I've been thinking recently about the backlash over Get Satisfaction and Yelp and the like where businesses are essentially forced into protecting their brands' identity on hundreds of different places. The only option is to monitor obsessively and respond as they can. However, perhaps the noise will become so great - imagine a future where every customer grievance is aired on Twitter - that it simply ceases to be important anymore because no one is listening.
To be honest, this is a landscape that's still pretty murky to me. I've tried Radian6 before, and am checking out Filtrbox at the moment, but haven't been sufficiently convinced by either of them.
When I compare them to free solutions (Google + Yahoo Pipes hackery + whatever), I just don't perceive the value. And neither do my clients, among them a sizable PR agency.
I can imagine that there's value for brands who are so big that they can only measure activity in aggregate. But for now they're priced too high for marginal additional value they offer.
Darren, thanks for that comment - that's just the kind of customer testimonial we love to hear. That said, it's a shame the state of the art commercially isn't topping your hackery. Way to go with that hackery though!
Some great idea Marshall! I'm hearing many from our customers too.
Facebook is slowly opening up. We're able to collect information that people have made public. The conversations in Facebook groups & LinkedIn Questions/Answers are interesting.
The textual analysis of the content provides a lot of direction for brands beyond just joining the conversation.
At Techrigy we're excited to be a part of this quickly evolving space. Many thousands are using our Freemium version & we're adding customers at an amazing rate. So I think that agencies & brands are definitely finding value.
Connie
Community Strategist, Techrigy
@cbensen
I have worked at several agencies working to put together budding social media practices, and have been pitched by many of the social media monitoring tools. I totally buy the concept-- the ability to "listen-in" on consumers in their native space. The challenge I find is that none of the offerings provide the total package. Some key things on my list:
-Efficient filtering of queries (most important, and often very poorly done)
-Sentiment analysis
-Sentiment plus (point in time, trend over time, compared with competitor, overlay with another issue etc.)
-Identification and targeting of influencers
-Ranking of "influential" or popular posts
-Comparison to competitive information
-Identification of "themes" within the conversation
-Easy, dynamic ability to chart and graph analysis of queries
-Self-service set up of queries (not a week or more lead time with "Customer Service")
- Backward trending (and not just for 30 days!)
These are just a few of the things I look for. My biggest pet peeve is that most of these tools still return a high volume of JUNK that takes a long time to sift through to get to the golden nuggets.
I have reviewed or used Radian6, Collective Intellect, BuzzMetrics, PropheSee, TruCast, and ScoutLabs, BuzzLogic and Radian6 among others. Many have some interesting functionality. I haven't found one yet that is the total package.
External monitoring of social media sites for business marketing or advertising purposes is pointless. Social media sites give a branded market the ability to present a specific look, an edge, to appear as forward-thinking, technology-literate, & adaptable. It does not at all give more marketability, just a more enhanced vision to a market - the ability to foresee change, & therefore, the ability to change with the scheme, or grow into the vision it has prescribed as a company-wide philosophy. All this is an internal PR strategy, & an external appearance of such strategy... gives the world a glimpse to a corporate maxim. A corporate marketing structure which includes social media, as it should, should very definitely keep control of all aspects of PR, including social media sites, which have the ability to be very damaging, if employees, or anyone within the corporate structure should speak out of turn; ie, not 'for the company' but come across as speaking 'for the company'... So, an external monitoring service could very well be 'amassing data' incorrectly, in an attempt to place a brand, or decide whether a company or service is viable in today's market, when in actuality, the company may well be suited to its market, but just getting bad press. The monitoring of a corporate release of information of any kind should always be an internal process, & include a plan for damage control dictated by the filtering of information from such external monitoring services.
Although I agree with the general direction of social media monitoring and its purpose, it's still quite early in the game. In essence, the existing technologies are based on keyword-match type approaches and offer little over manual searches.
The game changing apps will be the ones that can understand what is being said, infer things and take actions and notify accordingly. We still haven't made much headway in that direction.
I have also used the OpenCalais API and it's far from being truly useful. The 'entities' are far from complete. There are small things missing. For example, case sensitivity breaks results in many cases.
So, I'd say social media monitoring is probably an important part of the future but we have a long way to go.
Marshall,
What's most surprising is that some companies have yet to realize the power of a single voice online. No matter who it is, one person can make/break/affect a brand significantly. While no monitoring solution is perfect, that is by no means an argument for not listening.
Before Web 2.0 there were ways consumers could voice their concerns, dissatisfaction, etc. It was mostly done by letter or phone and the responses very often came back with coupons and/or other freebies.
Web 2.0 allows companies/brands to engage and respond immediately, preventing a domino effect of brand negativity (as we most recently saw with Motrin). Web 2.0 also gives a louder voice and more power to the consumer. Web 2.0 does not give absolute power to consumers or brands, and we can be thankful for that, as it corrupts absolutely.
Great post and I hope this starts some more meaty conversations!
DW
Anna,
While I will agree with you that no one provider has built the absolute perfect solution - yet, the one thing I have never heard is any of our TruCast clients call us to complain about junk data in their accounts. In fact, there isn't anything in your list of issues we can't do, and I would argue most of our competitors would also disagree. Many firms do great work for clients in this emerging field every day.
Mike Spataro
Visible Technologies
Marshall,
Great discussion you've got going on here.
Indeed, we're in an emerging industry that's rapidly evolving, and we're continually tweaking and improving our platform based on the feedback we get from users. Anna, I'd echo Mike's sentiments and say that we can do a great deal of what you're asking (and the stuff we can't is on the horizon), so I'd be happy to get you a fresh look anytime, and welcome your detailed feedback (my email is below).
At the speed of social communications, we're focusing hard on not just delivering data, but providing ways to integrate that data with other business functions and drive efficiency and insight for companies. It's no longer a matter of whether or not you should be listening, but what you do with all of the information you collect to build your business.
Thanks for continuing the discussion!
Cheers,
Amber Naslund
Director of Community | Radian6
@AmberCadabra
amber.naslund@radian6.com
Glad to know about backtype. It's interesting. When I search for my name in the People tab in backtype, I am able to see all my comments which i made in the past on different sites.
Nice feature.
Thanks
Sankar Datti
Hi Marshall,
I'm with Amber Naslund from Radian6 - This is a great discussion you've sparked here and one that will become increasingly relevant as the realities of pan-web brand transparency become more evident to marketing departments and advertising agencies.
Even today, long before we've reached the point where constant and full online disclosure becomes a prerequisite
for brand relevancy, there is already great value in the ability to aggregate, trace, match and collect peoples’ profiles from multiple social networks.
The beneficial uses of this new wealth of available info may be as prosaic as getting better acquainted with a business associate via their disparate online persona, or as complex as the creation of unified coherent consumer databases that are constantly updated in real-time with information collected from a multitude of disparate services.
We believe that being able to stay up-to-date with the online activities of your customers, partners & competitors is a valuable proposition already, especially if the effort required is minimal.
Search is quickly becoming to over-crowded to allow for efficient quick inquiries about the people & companies you do business with.
It is no wonder that other tools are filling the vacuum and helping users get a quick yet holistic sense of a companies online presence and activities (it's photo uploads, youtube interviews, screencasts, product reviews, stock data, etc.).
Another angle worth considering is that bringing users social and business persona into play in unison enablesw those that wish to discover how their network of contacts can be leveraged to benefit their business.
Finally, as was rightfully stated by the commentators before me, this is an emerging industry - one in which players must perform constant sanity checks, based on frequent rapport with their users, in order to check that they are indeed developing tools that people NEED and will actually appreciate and use.
I'll definitely be reaching out to some of the agency-background people who wrote before me in order to learn more about what they would consider a boon in terms of future feature developments for http://Headup.com and other products SemantiNet is working on.
Take care all,
Mike
"I tweet @headup"
Good piece and I believe Anna B makes a good point. As a PR professional, monitoring online conversations is critical. Twitter shows great promise as an early warning system to address what you said: "We'd also like to see some real-time threshold monitoring. Let me know the moment that conversation about me or a given topic hits a certain level of intensity."
But monitoring key words is not enough. Measuring influence is not enough. The real challenge is understanding sentiment. Where it gets tough is helping to differentiate between product X sucks and product X SUCKS!!
Creating a tool that measures all three (key words, influence, and sentiment) in real time is a killer app.
Has anyone checked out Social Radar - http://www.infegy.com/socialradar.php ?
I haven't had the occasion to use it yet but was quite impressed by the demo webinar. It certainly includes the sentiment analysis feature and nice visualisation of key influencers.
It is something I would like to try on a suitable client's campaign.
Truly interesting discussion here, and certainly a timely one with one of the busiest weeks so far on the topic of social media and brand protection - from the Amazon 'filtering' blunder to Domino's video PR disaster (see today's NY Times article. With brand protection being only one element of the Crisis Management life-cycle, you would expect corporate security and crisis management organizations to be on the forefront of the social media monitoring. But from our experience, the wheels have not even been put in motion yet for most MNCs. The Early Warning potential in areas ranging from workplace violence prevention to competitive intelligence are enormous and bound to be explored and exploited heavily in the years to come. But as many already pointed out already herein, most tools available today would have to be considered 'experimental'. It was almost two years ago that the news of what some referred to as the "Applegate" prompt me to start blogging about the subject and I must say that several social-media crisis later (as well as tens of millions of new social media participants later) I am amazed by the lack of maturity of solutions and awareness of the matter among the security industry. Less than six months ago I gave a presentation called 'Crisis Management 2.0' to a roomful of senior corporate security executives and asked how many had heard of Twitter: only one raised a hand. Your article was truly welcome and highly informative, and the ensuing discussions within and outside of this forum are what is needed to create a more educated public that can drive the demand for features, technology, and innovation, as well as establish privacy boundaries and best practices.
Cheers,
Maybe the search for the ultimate "black box" solution is a fruitless one. After all we are already running into difficulties and we are only at the start of this whole social media thing. Perhaps the clue is in the name - "social" media and thus the solution is likely to be a social one. Monitoring the converstaion in all its micro detail is all well and good, but if you become involved in the the right conversations you tend to find that the relevant information has a way of finding its way to you without having to search for it.
Perhaps the web itself is going to become the "black box".
See http://tinyurl.com/cey3s8 for similarish thoughts on monitoring's awkward sibling - measurement.
Marshall,
There's no doubt the social media analytics/monitoring market is extremely interesting and active as companies and people try to get a better sense of the conversations going on.
A company you should check out is Sysomos (www.sysomos.com), which has built a robust and feature-rich social media analytics service that's gaining a lot of traction.
Among the features helping Sysomos stand out from the crowd are that its database is spam-free and its ability to accurately measure sentiment.
I'm doing some work with Sysomos so if you want a demo of its Media Analytics Platform (MAP), which is launching soon, let me know.
cheers, Mark
I just got a copy of ePostMailer from http://Spryka.com and I would recommend to anyone who needs to send out an opt-in email mailshot. Its the best free desktop based email marketing software I have used so far.
Hi Marshall
As a technology company who have been 'media monitoring' in digital media since 2001, this thread brings a smile to my face. It used to be about websites. Now it's gone 'social'. Same, same, but different. Fact is - as many of the references in this thread proves - there are some very clever people and companies swarming around this space right now. But many of these efforts are fundamentally limited by the quality of underlying technologies being used. There are 2 stark realities that will drive the continued evolution of services in this market: 1) COMPREHENSION, and 2) SCALE.
Comprehension is all about understanding the meaning of content. It's simply not good enough to be using boolean based, keyword technolgies to find and then so-called 'understand the conversation'. You need to understant who's doing what to whom, when, where and how. Natural Language Processing and text analysis are the only way to radically improve automated intelligence in this way. And automation is a key driver, as SCALE is everyhting right now. If you want to garner insights from out there, you better do it fast, and big.
Social Media Monitoring will move quickly from isolated, stand alone, boxed-product-driven services to being an integrated part of the marketing analytics. And robust, lrage scale platform technologies like OpenAmplify and Calais are finally giving developers the tools they need to meet the challenge.
Remember that there are horses for courses. Not all clients are ready for the deluge of information that comes from the tools that are already out there.
The biggest barrier is not the toolsets that exist. They all provide lots of rich information (of course, more information will prove to be better over time).
The biggest barrier is *making meaning* out of all of the random data points. The SM monitoring tools that find *relevant* ways to do that will see market share growth.
Jamie - I 100% totally agree. That's my point in the post above. I would argue strongly that keyword 'hits' from the web/twitter/wherever are essentially valueless. The information itself is a comodity. Clients want to pay for relevant, insightful data. They want QUALITY, not quantity. Boolean search delivers noise. Keyword based analysis technologies are largely ineffective. And, to be fair, anyone can plot a nice chart on numbers of hits using Flash ;-) The quality of insight needed to move this market forward can only come from more sophisticated foundation technologies.
Mark Redgrave, Founder & CEO
http://www.openamplify.com
Hi Marshall,
I totally agree with the two Marks, who have both similar first names and similar thoughts. Social media monitoring will quickly become yet another metric for web marketers and users. Monitoring sentiment helps companies understand where they are going with their brands and consumers. Current filtering/listening devices aren't enough, and I agree that perhaps there is no "killer app." Why? Because the social and information web is changing so quickly that it is a very steep hill to create evolving listening platforms that meet the needs of the evolving uses.
That leads me to my final thought - where is web monitoring going? Surely the semantic web is already here, but to put it bluntly we are at the "semantic web 1.0." The next step may very well be using the semantic web as a listening and monitoring device, rather than an informational tool. Additionally, I suspect that monitoring activites on mobile phones (through the use of the apps) is soon to follow.
Marshall,
Great post and honored to join in the conversation!! There is a clear distinction between Social Media Intelligence and Social Listening Platforms -- the future is ever evolving. We know that integrating Social Data within Lead Generation, CRM and Direct Response, as well as Ad Serving is happening now.
The lack of making this data actionable is a major challenge. For PR purposes, if you do not know the reach of a person or the audience compensation, where do you focus? Some brands have 100’s, if not 1000’s, of daily mentions making it very difficult to determine a credible relevance. Keywords and conversation monitoring is not enough to determine Social Influence or the Relevant Reach of a persons / community has.
There is a great deal of talk in monitoring conversation vs. how consumers and their social personas connected around specific topics. Where can you start a positive conversation and who are your potential brand ambassadors for a brand these days? This is as important, if not the most important aspect of Social Intelligence. In reality, the majority of Brand Customers are not always bloggers and twittering on every topic about that certain brand.
Social Intelligence provides a clear way of seeing people and personal connection to a topic or group. We are seeing some enlightening innovation at UNBOUND Technologies www.unboundtech.com and welcome your feedback.
Chase
Marshall, Great post and alot of provocative comments here. I am the CMO at Biz360, a company that has been in this space since 2000 -- www.biz360.com.
At Biz360, we have measurement solutions called Opinion Insights and Community Insights which already offer some of the unique and innovative features you've discussed such as real-time alerts based on threshold monitoring as well as sentiment scored aggregated product opinion down to the model, attribute and feature level. I invite you to check them out.
It's important to remember that, as monitoring tools become more sophisticated they are only as good as the actionable insights they deliver. Automation, supported with experienced human analysis, I believe, remains the best way to get there.
Also, while social media has rapdily become a major source of influence in the purchase decision making process of millions of products and services, companies are wise to incorporate this data stream with other sources (e.g. direct customer feedback, primary research, news media monitoring, etc.) as they determine their engagement strategies.
Tony
http://twitter.com/tpriore
As several other commenters here have more or less pointed out, measurement and monitoring solutions are just tools. Insight comes from human analysts that understand your business and know how to use these tools. Some tools are better than other, and they are of great help to our analysts in helping solve client problems and automating some tedious tasks. We are eager users of a wide range of tools. But those that put their faith in technology to be the "silver bullet" will end up disappointed. We've been around natural language processing and text analytics for a while now, and understand the potential benefits, but also the challenges. Automated sentiment analysis is difficult enough to get within tolerable margins of error (and sometimes even one error is enough to undo credibility in the eyes of an executive). But I think we are a very long way away from a "magic algorithm" that can decide whether a website, or a blog post, matters or not. For this you need basic human judgment.
Nils
Context Analytics
10 years since the Cluetrain Manifesto?
Its time someone remixed this Martin Scorsese/Robbie Robertson vid.
To include the line 'Catch the Cluetrain'
http://short.ie/thecrazyriver
it's right, but a long way to go for a real success business case.
Marshall - great discussion and I agree with Anna B, Darren, Dan, Jamie and Mark.
I use tools such as Radian6 and Techrigy (which I regard to be the 2 best monitoring tools out there) as part of our measurement packaged service. These online tools are great but there is a real need for a 'simple to understand' report which shows actionable insightful conclusions to inform strategy. These automated tools cannot account for the nuances, sarcasm and synonyms of the English language however clever their technology or boolean searches are.
I work for Media Measurement (www.mediameasurement.com) and we use human analysis to provide this extra level of insight for agencies and comms professionals to create an informed social media strategy and measure its effectiveness.
We break down social media measurement into 3 categories:
1) Monitoring
2) Quantitative Automated Analytics
3) Qualitative Contextual Reporting on most impactful conversations
Interested to continue the conversation further...
Cheers
Tim
tim.williams@mediameasurement.com
www.twitter/williamstim
By employing some of the most talented individuals in the field of web design ,most of the companies has become a leading design professionals not only in Chicago, but also across the globe
This is a great discussion. I work for a big digital agency. My company and our clients have used a variety of the typical social monitoring tool ie: Radian6, Techrigy and a few others. At first everyone was impressed until we really understood the actual data behind them. Lot's of bells and whistles presenting incomplete and speculative data sources. As we and our clients got savvy on the whole SMM game we found Wool Labs and their WebDig suite of tools. None of the sex appeal (interface wise) of the others but tons and tons, almost too much data. We tested them heavily then recommended them to one of our global clients who had been previously using Radian6 and SM2. They instantly dropped the other two. We are only using WebDig now. Substance over surface.
Great post and honored to join in the conversation!! There is a clear distinction between Social Media Intelligence and Social Listening Platforms -- the future is ever evolving
At the end of the day as Tim from Media Measurement has suggested, the human mind could be the path of least resistance, especially when it comes to monitoring a conversation. Rather than trying to find an automated solution, you could hire a bunch of psychologists, or perhaps develop a crowdsourced solution, to analyze the threads.
I agree that monitoring a brand on the ever shifting and seemingly growing social media landscape is quite difficult. It takes a lot of effort and the tools really aren't quite there yet. They will be soon for sure. My concern is that social media will be overcome by people using it for business, branding and commercial purposes and the original use - socially interacting with people - will be lost.
Andrew: "My concern is that social media will be overcome by people using it for business"
I think that social media will develope further and become more decentralized in the future and so literally out of control when it comes to commercial influence. Look at forums today, some of them got chat rooms (shout boxes) and hopefully soon all the real-time communication will be linked and so transformed to a hypercomplex net in which twitter etc are ancient hotspots that will become smaller in time due to decentralization.
Social Media Monitoring tells you no actionable information to help make important business decisions. What companies need is Social Media Research not Buzz Monitoring but actual Market Research performed by researchers. Check out. www.conversition.com