Four Ad-Free Ways that Mined Data Can Make Money
Machines can do wonderful things. Side by side with the rise of a new world of publishers, the computer scientists of the world are cranking it up as well - building new ways to create value from the sea of data being published by people. And then they take their work and they sell it to advertisers!
Barf-o-rama!
We have some appreciation for advertising technology and we certainly appreciate our advertisers here at RWW - but why do so many innovative technologies end up slinking away into the ad tech world and watching their grand visions for user empowerment fade?
The most obvious answer to that question might be that advertising is where the money is made. Data mining, machine processing large quantities of information in order to unearth patterns or other valuable insights, seems just made for demographic and behavioral targeting by advertisers.
We argue here, however, that money can be and is being made from data mining in ways other than by by sale of information to ad networks. Cooler, more exciting ways. We briefly discuss four markets for mined data that we believe exist now or could hold strong demand for analysis of aggregate data from online activity.
Let's be honest - we're thinking about Twitter here. When people ask how Twitter is going to make money, we think data mining has huge potential. More than Twitter, though, all kinds of apps will soon trade in user data as a primary currency.
Here's what we think that could look like.
Traffic
The most obvious example that's already real is Internet Service Providers selling customer web traffic data to traffic analyst firms. When you see a company measures web traffic of sites around the web, you can be pretty sure they are buying data about what sites you are visiting from your ISP.
This isn't the most interesting example because traffic analysis does find some of its meaning in advertising. It's also used for competitive intelligence, identifying vertical leaders and generally adding some semi-verifiable sophistication to our understanding of the landscape of the web. Unfortunately, as any publisher online will tell you - the resulting traffic estimates from these services are often wildly inaccurate.
Sentiment Analysis
More interesting than simple comings and goings is sentiment analysis of language used online about a given topic. There are PR uses for this data, but there's also a market for it in analyst firms who use it to make recommendations to their subscribers and clients.
This was the real technology being built by Summize, the search engine recently bought by Twitter. You might have noticed that though Summize is now called search.twitter - there's still no link to it from the Twitter site. Perhaps search wasn't the most important part of Summize after all - perhaps it's the sentiment analysis that's got the most potential.
Vertical Trend Watching
Ok, so maybe sentiment analysis of online activity could be solid enough to be interesting and worth a lot of money some day. And if wishes and buts were candy and nuts, we'd all have a merrier Christmas.
You know who's not messing around when it comes to stuff like this, though? People who trade in money. Hedge fund buyers in particular are particularly willing to try out hard core technology in order to get more and better information faster than anyone else. They are nuts for crazy tech; they pay thousands of dollars for research tools that could wrap Google Reader up like a pretzel and swallow it in one bite.
Check out our review of power news dashboard FirstRain, and RootMarkets a company that aims to trade in futures of web browsing data, ultimately for lead generation.
We want to see this kind of data crunching research tech outside of financial markets, though. We'd love to see some trends crunched out of the Twitter streams from Real estate pros, people in the Navy or biotech researchers. Users are segmented into these categories already by the directory Twellow, for example. We think rapid analysis of emerging trends in those verticals is something people would pay for.
Benchmarking
Google Analytics will now let you identify what kind of industry your website serves and once you do, they'll tell you how your website traffic trends compare to what's being seen by others in your industry. FreshBooks, a startup that provides online invoicing for independent professionals, offers benchmark data by industry to its users as well. Compared to other graphic designers, for example, you're charging less and getting your invoices filled slower than most.
Benchmark data helps people and businesses make better decisions, hopefully saving or making more money than they would have otherwise. Isn't that a lot more interesting than advertising?
Pointing out patterns of information gets people talking, too. Recommendation engine Strands offers a mobile banking service that prompts users to fill out their profile information by sharing interesting trivia with them about patterns in the data of users as a whole. "Did you know: married people spend 110% on groceries what single people do? Are you married or single?" Knowing whether customers are married or single lets a bank offer them targeted services, to understand the risks faced by their customers etc.
The Future of Data as Currency
These are just a few ways that large quantities of data can be used to derive value other than targeted advertising. All of them are more interesting than advertising, too.
Just like grocery stores give customers discounts in exchange for capturing their purchase histories, so too will users of online applications receive compensation for the data they co-produce with service providers that's subsequently monetized.
Beyond money, user co-producers of data will likely call for the ability to take their data from one service over to another, where they can contribute it to another aggregate of data and thus participate in another instance of value creation through the processing of data. That's data portability, or one way to articulate it.
We hope to see more examples of creative thinking about data mining and more startups that avoid taking the path of serving advertisers as their ultimate customers. The use of a tool impacts its orientation over time and these great technologies we are beginning to use online should be formed with greater goals in mind. There's too much utility at stake and the world's problems are too great for all this potential to be stunted by the seductive call of ad money. We hope an economy will grow to support alternative uses of user data and we hope it happens soon.
Top photo: Data processing center, CC from Flickr user Marcin Wichary
1 TrackBacks
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/4734
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
I like to think that one day, possibly powered by something like Mozilla Weave, all of the experiences we have with data, whether it be sitting in front of a laptop, browsing on our mobile phone, or tapping an object that is NFC enabled, is stored in one central location that we have access to and that we let third parties have access to in order for cheap or even free services.
The information we choose to give about ourselves is the only information that they'll get and "data mining" as you put it happens because we let it happen, not because some spider crawls the internet to find out everything we've ever done.
I see user privacy as something that will come back, not because users care about identity theft, or their boss finding out that on weekends they love to get hammered, but because soon people will realize that letting people have access to their garbage (their browsing history) will yield them better and better services and content.
Posted by: Stefan Constantinescu | August 24, 2008 11:44 AM
Sentiment is weakest of linguistic markers, and there are no standardized thesauri for comparing the scoring of different engines used in the likes of brand monitoring.
However, there are ontological scoring methods that rely to a greater extent on "redress" linguistics. We shall see these NLP toolkits out soon.
Generally, taking the overall polarity score for a post or page, using a non-standard thesaurus is a flawed method.
Posted by: Alan Wilensky | August 24, 2008 12:11 PM
Marshall,
Is RootMarkets still around? I know Seth Goldstein has moved on to start Social Media - http://readwritetalk.com/2007/10/09/seth-goldstein-ceo-socialmedia/
- Sean
Posted by: Sean Ammirati | August 24, 2008 6:37 PM
Sean, fair question. A LinkedIn search shows 15 people currently listed as Root Markets employees. Their website looks active as well.
We've also spent some time focusing on how mined data can serve those other than advertisers. What about empowering online publishers with:
In the latter two cases, it's indirectly serving advertisers needs but still -- the publisher is the primary beneficiary for once.
As for benefiting users, what about using mined data to connect people with each other (for discovery or personal promotion), refining search results (affinity-driven) and providing users with control/transparency.
(Disclosure: our company Others Online is doing this today.)
Posted by: Jordan Mitchell | August 24, 2008 11:28 PM
Well written post, Marshall.
Posted by: me.krishworld.com
|
August 24, 2008 11:52 PM
Alan Wilensky said...
We shall see these NLP toolkits out soon
NLP is not data-mining at all. It is geared towards search. The definition of data-mining is to reveal obvious but hidden knowledge that is buried in the data (ie, knowledge-discovery) and NLP falls way outside this definition.
Posted by: Falafulu Fisi | August 25, 2008 9:08 AM
In the best-case-scenario advertising network model, sites should be able to contribute either inventory or data to the system. Assuming that privacy policies are obeyed and enforced (no small feat, of course), this creates a very clear growth path away from a straight ad model to a more robust knowledge model.
Posted by: Gary Stein | August 25, 2008 9:16 AM
Marshall,
Cool! I actually was really interested in the concept, just assumed it had gone away.
By the way, great post! I've actually been thinking about it quite a bit since reading last night.
- Sean
Posted by: Sean Ammirati | August 25, 2008 10:49 AM
Very interesting post but I see there is a little misconception here.
Data mining does not reveal obvious information but the opposite (think of the classic beer-nappies example) To this respect it is wrong to say that DM has anything to do with analytics, or demographics or trend watching. A simple arithmetic comparison of the search queries "iphone" and "read write web" IS trend analysis but not DM. A DM system would rather output that around 45% of RRW readers have an iPhone.
I have the feeling that if efficient large scale Data Mining is ever possible then ads would be the last idea to think of. AI would be possible.
Posted by: panos | August 25, 2008 3:08 PM
Panos said...
To this respect it is wrong to say that DM has anything to do with analytics, or demographics or trend watching.
Panos, I specialize in data-mining & machine learning algorithm development. I write the algorithms from ground up, some are based on peer review publications (if those algorithms are new in the literatures) and some from books. I also use some open source such as the popular Java WEKA project from University of Waikato New Zealand, so I do really (I mean really) know my subject domain of what I am talking about.
It is obvious that you have no clue to what data-mining is. Analytics is where data-mining is mostly and heavily used.
Panos said...
Data mining does not reveal obvious information but the opposite (think of the classic beer-nappies example).
Yes, I know that example. The example uses the familiar Association Rule data-mining algorithm. That piece or nugget of knowledge is already sitting in the data-base (ie, it is obvious but hidden). An analyst applies the Association Rule data-mining algorithm to the whole database and this piece of hidden knowledge pops up, which were not known to the management.
Panos said...
A simple arithmetic comparison of the search queries "iphone" and "read write web" IS trend analysis but not DM
What is your definition of trend? You're confusing string matching search (which is not data-mining) to concept searching which is data-mining, besides searching queries is not trending since it doesn't show a relationship to time, because that's what trend is about, a variation of a measurable quantity over time.
Posted by: Falafulu Fisi | August 25, 2008 5:12 PM
@Falafulu Fisi
I guess we are not here to argue in person but nevertheless you rang the bell and I am in.
Panos, I specialize in data-mining & machine learning algorithm development...I also use the popular Weka...It is obvious that you have no clue to what data-mining is
That's cool. So you must have noticed the Related Projects that the WEKA authors refer to in their site. Can you see Agent Academy? It couples DM with Multi-Agent technology. Well, surprise, I wrote it...So, like Socrates, I know enough to not know anything.
Analytics is where data-mining is mostly and heavily used.
I never claimed the opposite. You may also use a lighter to open up a bottle, but this does not mean the lighter was meant for that.
Yes, I know that example...(ie, it is obvious but hidden).
Ok you are great in theory, but you repeat the same mistake. An obvious thing cannot be hidden at the same time. Pick one.
What is your definition of trend? You're confusing string matching search to concept searching which is data-mining,
I never talked about string matching. My point was that if people search more for "iPhone" than "RRW", then "iPhone" is more trendy than "RRW". This is the most elementary inference you can make. As I said this trend analysis does not need DM to work.
...searching queries is not trending since it doesn't show a relationship to time
So Google is out of business and this trend comparison of "iphone" and "n95" is just an illusion. Is that time on the x-axis??
Posted by: panos | August 26, 2008 1:12 AM
Panos said...
It couples DM with Multi-Agent technology. Well, surprise, I wrote it.
Yes, but multi-agent technology, is not that difficult to learn compared to learning concepts & algorithms of machine learning/data-mining, since the algorithms are more complex (mathematically) where multi-agent is symbolic where you don't need a prior knowledge of calculus. That is learning multi-agent , you don't need to understand difficult concepts of differential calculus, linear algebra, multi-various statistics, signal processing, feedback control theory and so forth where algorithms from these complex domains have found applications in data-ming .I myself have used JADE (Java Agent Development Environment) open source in the past.
One can use WEKA (ie, its thorough API documentation) without understanding anything about the algorithms, even seasoned developers (perhaps with 20 years or more) who are experts in agent technology do that.
Panos said...
As I said this trend analysis does not need DM to work.
Man, it is quite obvious to me that you have no clue to DM. There is a branch of data-mining (a sub-field) that concentrate on time-series/temporal mining, where trending is one of the many metrics to be mined. Take a look at references (freely downloadable PDF papers) with papers related to trend/temporal/time-series data-mining. The algorithms described in those publications are not available in WEKA, and I am not surprised at all that you haven't heard of them, since you only experience the names that are already available in open source project that you use. I have implemented some of those algorithms for financial time-series mining.
Panos said...
So Google is out of business and this trend comparison of "iphone" and "n95" is just an illusion.
Well, again, this shows that you don't understand data-mining. What makes you think that the graph is something data-mining? The graph was a straight plot of the term counts. There was no data-mining involved. It is the same as a plot for total monthly sales fro a product of a company. There was no automated discovery process being applied to the data. What you quoted was not data-mining.
Here is what I meant. Read the following paper (PDF is freely available), may be you can understand of where data-mining apply. The ICA signal processing algorithm is use to mine patterns in chat room topic discussions, ie, trend mining.
SIGNAL DETECTION USING ICA (independent component analysis) - APPLICATION TO CHAT ROOM TOPIC SPOTTING
I can give you more references (journal papers) on time-series mining, but I think, I've already given enough free information to you on this thread.
Posted by: Falafulu Fisi | August 26, 2008 7:55 PM
Yes, but multi-agent technology, is not that difficult to learn compared to learning concepts...
Quoting Pirandello, "it is so if you think so". The Windows OS is close to MAS architectures. I guess a seasoned programmer can build an OS quite as easily.
There is a branch of data-mining (a sub-field) that concentrate on time-series/temporal mining, where trending is one of the many metrics to be mined.
What I said was that THIS trend analysis (the one in the article or Google trends) does not need DM to work. I didn't claim that DM is not used in trend analysis. Let's me check once again. Ok, I didn't!
What makes you think that the graph is something data-mining?
From the previous, I didn't say this graph is DM. For the 3rd time, the point is current trend analysis(article,Google
Posted by: panos | August 27, 2008 6:55 AM
Yes, but multi-agent technology, is not that difficult to learn compared to learning concepts...
Quoting Pirandello, "it is so if you think so". The Windows OS is close to MAS architectures. I guess a seasoned programmer can build an OS quite as easily.
There is a branch of data-mining (a sub-field) that concentrate on time-series/temporal mining, where trending is one of the many metrics to be mined.
What I said was that THIS trend analysis (the one in the article or Google trends) does not need DM to work. I didn't claim that DM is not used in trend analysis. Let's me check once again. Ok, I didn't!
What makes you think that the graph is something data-mining?
From the previous, I didn't say this graph is DM. For the 3rd time, the point is that current popular trend analysis techniques(article,Google Trends etc) is not real DM.
That is learning multi-agent , you don't need to understand difficult concepts of differential calculus...Man, it is quite obvious to me that you have no clue to DM...Well, again, this shows that you don't understand data-mining.
Before I push the 'Ignore' button,a friendly advice: Unless you are the lost Jim Gray dude, you should take it easy for a while. If you spend your time telling people(Weka-cited also) how little they know about DM , I guess you must have had little time developing your skills.
Posted by: panos | August 27, 2008 7:07 AM
Panos said...
The Windows OS is close to MAS architectures. I guess a seasoned programmer can build an OS quite as easily.
Windows OS is built by hundreds of developers, if you can get 200 seasoned developers, I am sure they can. Compared to how many of those seasoned programmers would understand SVD (Singular Value Decomposition), a popular linear algebra/data-mining algorithm that is widely used in concept search engine and many data-analytics application? I bet you the answer is not many. You can sail thru a non-numeric programming book if you're new to it, and this is fact, but you can't assume that you do the same if you're reading a numeric-based programming book. I had done contracts with software houses where they have seasoned developers and some of them had been there since the early 70s, from Cobol, Pascal, Delphi, to .NET/Java in modern day. I always encouraged some of those season developers to learn the algorithm that I have developed for them, because I might not be there (since I am there temporary) if they need me at some stage in the future, when there is a problem with the numeric module I wrote. I showed them publications/sources of the algorithms, to refer to in case they need to modify/fix a problem of what I had developed for them. I've never had anyone (seasoned developers) said, yes, they now understand the algorithm, that perhaps they can deal with it themselves in case there is problem. I frequently asked why they don't try to learn those algorithms, they say, that there are things which are learnable, and other stuff which are unlearnable, regardless of how many hours you put into it. They say, that numerical computing is one of those things that are beyond being learnable to them, and this is a fact of life. I often heard people making remarks that programmers are good at math, simply because they can program. This is a misconception. Numerical computing is a completely different beast to just being programming. One can move easily from numerical computing to just being programming but it is a very difficult task of moving from being just programming into numerical computing and this is fact.
Panos said...
popular trend analysis techniques
Let me ask you. What analysis (algorithmic-wise) that has been done to the data? Repeat after me, a million times. There was no analysis at all. The so-called trend that you're babbling about, was a straight retrieval. That is, the data (term counts) was straight query and a retrieval from the data-base. This number didn't run thru a filtering processing or anything like that at all, it was just a straight pull out. To mine data (as the document on ICA I have pointed out to you in my earlier post shows), there is the normal process of data-mining (pre-processing/filtering and pattern identification). None of those applied to the so-called trend-analysis. You're mistaken the term analysis to mean machine analysis (which is the foundation of data-mining). Your definition of analysis , means the human (the user) who looks at the graph, and make inferences on what the trends look like. Yes, and this doesn't fall into the definition of data-mining. Trend-analysis (I mean analysis), is done without any visualization at all, since it is the machine that discovers the knowledge and not the human. Your definition of trend-analysis relies primary on the ability of the human to interpret the graph, ie, the human is the one that decides what knowledge that shows up in the graph. Do you get the difference? So, don't call it trend analysis because there was no machine analysis, it is you (the user) who does the analysis.
Panos said...
I guess you must have had little time developing your skills.
Yeah right. Did you read the trend analysis document that I quoted? I thought not. It shows that you don't have skills in the domain, and may be its too difficult for you yeh? You need real skills to understand the document let alone of how to write the algorithm, and that capability only belongs to those who have quite a bit of of time developing their skills.
Posted by: Falafulu Fisi | August 27, 2008 3:45 PM
Manufacturer and Exporter of fiat-445 tractor spare parts, Engine part, automotive, Transmission parts, Hydraulic Parts, auto parts, Body Parts, Electrical Spares
Posted by: windsor export | September 11, 2008 2:08 AM