Is This Why Twitter Changed Its Replies Policy?
Yesterday afternoon Twitter made a fundamental change to the options available to users by eliminating the option to receive messages from our friends sent publicly to people we are not following. We called it a disaster that would seriously disrupt serendipitous discovery of interesting friends of our friends.
Twitter has offered two explanations for the change. First, that very few users were choosing to receive these kinds of messages anyway and that it was confusing. Then, this morning, the company put up a blog post saying simply that "there were serious technical reasons why that setting had to go or be entirely rebuilt--it wouldn't have lasted long even if we thought it was the best thing ever." So what's the story? Here's our best guess.
Mainstream Users
First, it is clear that most users have not chosen to receive public replies sent by their friends to people they themselves aren't following. Non-early-adopters in particular are quite likely to consider these kinds of messages noise. Many of us early adopters believed such messages were a part of the magic of Twitter; it's a great way to discover people your friends find interesting. Information overload can be dealt with by forming groups in a third party twitter client like Tweetdeck or Seesmic Desktop. Getting over our antiquated sense of guilt and obligation concerning reading every message we receive would help too. (We've argued that online noise is good for you.)
The fact is though that most Twitter users now probably only receive replies sent to people they know. That's been the default setting for several months and we assume the system has thus been architected for this use case.
Scalability
What does that mean? Probably this: when you publish a Tweet that begins with @username, the computer that all your user files lives on in Twitter HQ probably already knows not only who your friends are, but also who their friends are. Everyone who is your friend but is not friends with the person you are replying to won't even have the message sent to them. Making that determination locally and limiting the tweets broadcast to your friends is probably much more efficient for Twitter than sending that message to all your friends.
Even updating the "friends of friends" files for all your friends every time you make a new friend - is more efficient than sending out replies to all your friends who aren't following the recipient.
Sending Tweets from one user to another is traffic-expensive and if most people don't want replies directed at people they don't know, then none of us are going to get that kind of message.
Most people don't want a noisy public conversation? Most people don't want serendipitous introduction to new people? That sounds like most people don't want a key part of what makes Twitter most magical. That magic is expensive and if millions of Oprah followers don't even want it, then a few thousand @marshallk followers don't get to have it either.
Who can blame them? Everyone wants Twitter to scale. Maybe it has to be neutered in order to do so. Architecting the social graph can't be easy. We can be sad about the decisions that the company makes in order to try to do it, but that probably isn't going to change things.
Instead of strolling through the social graph to discover new users, Twitter users will likely be given a new set of recommendation features pointing them to new people to follow. It may be a "people you follow also follow these people" type of feature, hopefully it won't just be more recommendations based on who Twitter's established leaders like. Or perhaps those of us who want all messages published by our friends can pay for a premium account. I'm ready to do that right now.
Thanks to the super-smart Alex Iskold, who we trust a lot and who helped talk us through this technical speculation.
Update: Hours after we put up this post, Twitter has reversed this policy. The story just keeps getting more interesting!
Just as soon as Twitter is back up, let's be friends. You can find ReadWriteWeb on Twitter, as well as the entire RWW Team: Marshall Kirkpatrick, Bernard Lunn, Alex Iskold, Sarah Perez, Frederic Lardinois, Rick Turoczy, Sean Ammirati, Lidija Davis, Jolie Odell and Phil Glockner.
Share
Facebook
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
Regardless of the reason, as I indicated on my blog in the wee hours of the morning, the new Twitter policy is essentially killing transparency that government is trying to enact.
It's one thing to follow Oprah or Ashton and not want to see @ replies sent by either of them to their fans who tweet them questions. It's something else entirely to now be prevented to seeing @ replies sent by @WhiteHouse, @UKParliament, or @JohnCulberson to anyone.
I was going to tweet a link to this story, but of course we're in the "maintenence window" now, so I did something old-fashioned and unfamiliar. I linked it into my piece on this topic over on scripting.com. Like riding a bicycle in the age of super-sonic airplanes. It still works though. :-)
A great explanation of what's going on and the challenges of tracking hyperconversation. I'm sad. But new discovery methods (hopefully that AREN'T popularity driven) could be interesting.
I think you're right on with this - the friends graph is relatively easy to store and probably resides in memory.
Having to go through all the friends and see if they have this particular setting enabled is probably what was pounding the servers. It seems like this is also the fate of their re-building parts of the architecture in chunks.
Damon (@dacort)
FTR, I want the old setting brought back. But in response to Ari, if it's only @whitehouse et al that you are worried about, why not just go to the twitter page for those accounts to see all of their tweets? The government transparency is still there -- it's just not being delivered directly to your Twitter feed.
Let's also keep in mind that Twitter not only needs to scale it needs to make money. Does that function allow them to monetize? Probably not as much as other functions and I think what we are seeing is the shift of we have a cool product and we will make it cooler to we have a cool product now how do we pay for it?
If they're doing this to try to give themselves scaling headroom, it's really just a very temporary bandaid. The problems will return soon with even modest user growth. They need to fix the fundamental underlying problems with their architecture to provide long term scalability, not put their fingers in the dike by removing features to marginally reduce the number of tweets being handled (if they're telling the truth about how few users had that feature enabled, the amount of headroom this will give them is truly minuscule).
From some perspectives, it's actually more difficult to filter @replies than to not. If no filtering at all was done, it would be brain dead simple -- every tweet goes to every follower. This is how it used to work. There's nothing magical about @replies. They're just tweets that happen to start with '@'. When you get into filtering, you have to check each and every follower's follow list. This has to be expensive in the cases of users with a lot of followers. Think about it. When @aplusk tweets, they have to check the follower lists of over a million users, some of whom themselves might be following thousands of people. This is a lot of extra work to hide those tweets -- it's the hiding, not the showing, that is expensive.
What a shame. The beauty of it all has been the opportunity to meet and know people you wouldn't typically come across. Every evening, without fail, I see a group of tweeters in England chat about their morning coffee and what tech/media stuff they have to do that day. They're not @'ing me, but I get to be part of the daily ritual... and I find it a lot of fun.
I have to say, am glad I started using Twitter early enough to get a chance to connect with a really amazing global community. No offense Oprah, Ahston & others who are now constantly in the "recommended follows" lists.. I like meeting and knowing the everyday folk that share little bits of their life. --sukhjit
Here is my read from the above: Twitter is forced to choose Scalability over Serendipity. The magic of Twitter somehow does not outweigh the burden that is the Masses.
The folks at Twitter seem to be WAY behind their users in understanding WHY people use Twitter. They don’t seem to understand it has evolved far beyond “What are you doing?”
One of the fastest growing (and most useful) ways to use Twitter is for customer service. In fact, I counsel agency types who think they can run Twitter accounts for clients that this would be unwise because followers will constantly be asking customer service questions. This is in the same vein as what Ari Herzog referenced with his great article on Huffington Post.
Obviously, if we were still in the mode of thinking Twitter was just about “What are you doing?” then getting rid of the @replies feature wouldn’t make much difference. There’s no point in seeing @replies to non-friends if all we want is random statements about people’s thoughts or activities. But those of use who’ve been using Twitter for quite some time know that "eavesdropping" on conversations is EXACTLY what makes Twitter viral and valuable.
Perhaps @Ev and @Biz need to do a little research to find out HOW people are using their product since it’s no longer used in the way it was originally intended – and for good reason!
@CarriBugbee
I've written on my blog how Twitter's list of 208 Suggested Users has basically determined the Top 100 Twitter Users on Twitterholic, Twitter Counter & WeFollow since it began in February. One user account which was added saw their follower count go up over 5000% in one week. Their system of recommendation is very powerful and, if you look over their list of who to follow, it reflects the subjective opinions of staffers. It's mostly celebrities but there is not logic to why one person is added & others of equal or more importance are left off.
I can see the usefulness of personalized recommendations but I want the choice of who to follow to be in my hands. I appreciate your explaining the technical details of what is involved with sending out messages to different lists of users (although it still seems like it would take more work to filter out Tweets from just those you follow to just those you follow) but time after time I've read Twitter's founders talk about adding new features in (trending topics on right-hand menu, @mentions instead of @replies) not take standard ones away.
I also don't understand how a feature that they say is used by a small minority of users (me included) can cause such havoc to their system. If it was 75% or 50%, I can see the bother but I'm hearing they claim only 2% choose all @replies. 2% of users' choice to see all Tweets from those they follow can send Twitter into a downward spiral? The system is more fragile than I realized.
Saying that users told them that this feature was confusing (what users, when was this poll taken?), and then removing the feature...and then only to throw your engineers under the bus with a semi-technical reasoning is absolute horrific brand/company/community management. Zero excuse for this. This isn't a beta service with 10 people on it, we're talking 15+ million. Oprah love, Kutcher love. With that comes responsibility.
P.S. Twitter should also realize that every change that streamlines & makes the system as functional and efficient as a piece of plumbing also makes it about as appealing. Does Twitter really want to become the Walmart of social networks, opps, I mean, information networks?
Seems like as big a technical challenge to filter so I don't see certain posts. While I'm sure everything is a technical challenge, I'd discount any developer who used this as a reason to change the feature... that's just lame. But, in any event, it's all speculation at this point. I say we just wait and let twitter figure out how to fix it up. I do think it's fair to say it was a misguided change. Just change the default but leave the option. Taking away features always disrupts people.
Oh, I think the following statement misrepresents how twitter works--that is you say "Sending Tweets from one user to another"... is expensive. Well, nothing it getting "sent" (except emails). Rather, I ping twitter and get a response. Sure, if they filter the response I suppose it's less data--but not really considering that it gets paged. Say I get most recent 20 replies. There's no bandwidth saving. I call bs on the claim it's a technical issue. (Sure, everything is technically challenging but it's not any more difficult than anything else.)
If you try to fix a system (scalability) by changing its function (eliminating visibility of certain forms of @ replies), you are effectively breaking the system.
Perhaps there's already another Twitter-like app out there that people will migrate to if this feature isn't fixed?
I was an acting Director of Technology for one of my clients for 7 years and understand all about scalability, complex coding issues, etc. Here is the bottom line. There is no question that the code can be rewritten and enhanced to make everyone on Twitter happy. That means those that only want @replies from those they follow would only get that. And those that want @replies additionally from those they don't follow would get that. This can all be set up in options.
Marshall is right that this is about the hyper-connectivity in tracking the @replies from people we follow to those we don't follow. My guess is that this involves both major coding changes plus a major expansion in hardware capacity.
Twitter may have exploded on the scene but they are still in start up mode both operationally and financially. The real question is what does Twitter want to be? If they want to continue down this new path that is attracting so many users so quickly, they are going to need to make the investments to do so. If they don't make those investments, they are going to stall as people move to the next New New thing like friendfeed.
As a management and marketing consultant for the last 24 years, my professional advice to Twitter is that they must go for it, scale up fast, make the big investments or they put their existing backers investments at risk.
To me, Twitter is (or was) different than other social media sites because we could shape what Twitter was, use it the way we wanted and drive its evolution. Notice the ever expanding creation and use of hashmarks. Where else can we create our own category tags and use them to see like minded Tweets?
Twitter was and should be about openness, freedom, and user driven development. Twitter (the company) needs to keep this philosophy alive and find ways to expand it even further. That is how they will remain relevant in the future and draw more users.
Don Shapiro
President
First Concepts Consultants, Inc.
www.firstconcepts.com
Donshapiro1 (on Twitter)
So this is kind of like a "Not-quite as private direct message" in that to see the @reply from you to someone else, someone will actually have to go to twitter.com/yourusername.
If there are sound technology reasons behind it, fine, but I have noticed a drop in my feed numbers, however, if this makes it easier to follow popular and active power users that @reply a lot, then I'm for it. There will be ways to discover new people to follow.
It could be helpful on Twitter.com, if you are logged in, to have color coded userids in responses with different colors for friends, followers, and mutual friends.
Marshall says that the system seems to have been architected this way, I agree, and I wonder what the stats were on this feature "choice" before they made it the default setting. I wonder if they are dramatically different.
I, too, use the chatter of whom I follow to find new and interesting people and have often wished that Twitter allowed you to shut off the @ replies for individuals and not have it be all or nothing. I like to follow a persons every tweet for awhile, but if they don't have great conversations with others or ever offer me any new people to listen to, it would be preferable to turn off their @ replies, while keeping the others I do benefit from. So leave it on for @rww but not have to see @JimmyFallon comment back to every loyal fan.
Perhaps this would be a burden for those with thousands of followers, to create unique settings for each person, but for them the all or nothing option can still be available.
I don't build these things so perhaps that level of specification would cause some tear in time and reality and thus destroy the earth, but it would be marvelous to have options and not edicts.
Isn't it wildly irresponsible to speculate on the technical issue behind @replies without, you know, actually knowing Twitter's code? The problem may not be what's speculated above at all. Instead, it may be the "in reply to ..." calculation that's costly. Or it may be Scala's fault. We just don't know. And it's silly to pretend that we do.
The beauty of Twitter, which seems obvious to some and completely hidden to others, is that it is a communication tool.
And herein lies its truest and best quality: the combination of broadcasting and narrowcasting that is personal, unedited and transparent—to everyone. Top that off with the ability to select who you follow and you've recreated the most successful method of communication there is: face-to-face conversation.
Unfortunately, there is a potential trap to this freedom and it comes about through attempts to control the users. (It's a common and perilous start-up mistake.)
The paternalistic, rational actor idea that everyone is quantifiable and you can cross-reference who-might-like-whom through a series of algorithms (a la Facebook) is tremendously problematic for a variety of reasons, and I wish social media would stop attempting to implement it. Here's why it's problematic on Twitter: some of the best interactions arise from its cocktail party effect—the serendipitous, chance, slightly off-kilter, overheard remark and the response to it.
Take that away and it might as well be a tool we already have in our toolbelt: IM, email, text messaging, social network sites, blogs, etc.
The sociologist Mark Granovetter wrote about the power of "the strength of weak ties" and Twitter capitalized on it, mightily. But they didn't stop there, they allowed anyone to listen in on anyone else's conversation—weak ties that are so weak that they are almost non-existent. Brilliant, but sadly it now seems that this feature was created completely by accident and not design. Some luck you fall into, some luck you make...
So, if we're searching for ways to bring people together, consider this: sometimes all you have to do is put a bunch of people in the same room—they'll figure it out from there. That's what Twitter was. I hope to see it like that again.
@carlacasilli
I just wrote a post talking about this and explaining exactly why I believe the "technical issue" explanation Biz threw up today. You can read it here:
http://briantroy.com/blog/2009/05/13/twitter-replies-and-multicasting/
@Brian Can you explain why they offered a faux excuse before telling us what the real problem was? ; )
Posted by: Liz
|
May 13, 2009 2:24 PM
Nice explanation. Thanks Marshall.
had to skip the comments above for times sake so apologies if this has already been said but the whole viral/stumble "hey, cool!" way of randomly striking-up conversations with folks in a rubics cube / "beautiful mind" environment... these things are twitters USP. there are myriad ways to communicate privately. I use esp most of the time. (unsuccessfully) but, ignoring the trash and the noise, for building an organic community, or even a collection of strange bedfellows who never, ever would have met in real life, twitter really is unique. and, often times a stepping stone for graduating to platforms more suited to in-depth communication/relationship building, etc...most of us start with one user. it's this shoots and ladders way of connecting with many more that makes this the most friendly and effective way of expanding your network, and your mind. how many times have you read a tweet that was either funny or scary, (or both!) and followed the thought through to the original tweet to gain perspective and context? that's how I've met all the best people.
Isn't this a quick and dirty fix to put a cap on spam? We all tend to watch our @replies pretty regularly and the spammers are starting to figure this out that throwing in the @username into a spam-tweet is a great way to get somebody's attention.
#FollowFriday will look more like this now RT @lizwebpage RT @markwalters RT @lovepeaceunity still works but longer now.
RT @CarbinCopy JC Enjoy the Ride. can also work for replies for people still wanting to share with their Peeps on Twitter. ByPass this. thier comments with other peeps.
Twitter really isn't that good. What's the big fuss?
I'm intrigued by the temporary 'fix' that Twitter has put into place:
"So here's what we're planning to do. First, we're making a change such that any updates beginning with @username (that are not explicitly created by clicking on the reply icon) will be seen by everyone following that account. This will bring back some serendipity and discovery and we can do this very soon."
Makes me ask a number of questions:
Does this 'fix' suffer from the same scalability issues as the original problem?
If this fix does suffer from scalability issues is it just that they know they aren't going to use it for long, or that usage is going to be so, so, small (presumably a small percentage of the 2% of users who had decided to see public replies anyway)?
If this fix doesn't suffer from scalability issues, is the scalability somehow linked to tracking conversations as opposed to just who sees what?
Doesn't this make the whole thing just much more confusing? You will now see replies where the person replying has not used a 'reply' function explicitly - this is not under your control, but the person tweeting, and behaviour will look erratic as a result.
it seems Twitter brings the twitter economy to all us.
I'm sorry Liz... missing something, What faux excuse? I'll grant you this is not one of my more lucid and insightful posts...
Posted by: Brian Roy
|
May 14, 2009 2:56 PM
I just love the magic of twitter, I've been on twitter for less than a month and I have found a host of friends who twitter daily and even text msg, and make calls regularly.
Thank you twitter for this wonderful social media and the minds behind updating the system.
DrUw
If I am understanding this correctly, I am sad to know that I no longer will be able to find new and interesting people through this venue. I have found people to follow and they in turn me from reading their tweets to followers.
had to skip the comments above for times sake so apologies if this has already been said but the whole viral/stumble "hey, cool!" way of randomly striking-up conversations with folks in a rubics cube / "beautiful mind" environment... these things are twitters USP. there are myriad ways to communicate privately. I use esp most of the time. (unsuccessfully) but, mantolama ignoring the trash and the noise, for building an organic community, or even a collection of strange söve bedfellows who never, ever would have met in real life, twitter really is unique. and, often times a stepping stone for graduating to platforms more suited to in-depth communication/relationship building, etc...most of us start with one user. it's this shoots and ladders way of boyacı connecting with many more that makes this the most friendly and effective way of expanding your network, and dış cephe kaplama your mind. how many times have you read a tweet that was either funny or scary, (or both!) and followed the thought through to the original tweet to gain perspective and context? that's how I've met all the best people.