Interestingly, Google also notes that image spam, which is generally filtered out quite well by modern spam detection software, has seen a major resurgence. Amanda Kleha, a member of Google's message security and archiving team, theorizes that this might be due to new spammers getting into the market after the shutdown of McColo and 3FN, and these new players are starting out with well established methods, even if they are not very effective. Kleha also notes that spammers might just be testing how well the current generation of spam filters handles these messages in order to perform statistical analysis based on which subject lines and content make it into users' inboxes.

Google also notes that one of the largest spam attacks in the last quarter was based on an old school "newsletter" template (with malevolent links and images thrown in there for good measure). This attack unleashed about 50% an average day's spam volume in only 2 hours. So while it might not have been highly sophisticated, there was surely a massive network behind it that was able to send out this huge amount of spam in such a short time.

According to MessageLabs, Google Sites spam only accounts for 1% of all spam at the moment, but they expect this technique to become as popular as similar techniques being used to distribute spam using other free Google online services, including Google Docs, Google Pages, and Google Calendar.

The benefit to using Sites for spamming is that it's harder to block the resulting URLs generated by the service. Unlike Google Pages, whose URLs are in the format of accountname.googlepages.com, a Sites URL begins http://sites.google.com/site/. The format of these URLs, which contain "google.com," are more difficult for traditional signature-based anti-spam tools to block. At the tail end of the URL, the spammers' sites will contain site names that are composed of random letters and numbers.

Sites is certainly not the only Google product that has been adopted by spammers. For example, I noticed an increase in Google Groups-related spam messages arriving in my Gmail inbox recently. Google Sites looks to be more of the same. Spammers are certainly clever, so it's up to the makers of anti-spam technology to combat this latest threat of "Google Spam." Clearly, just because something is hosted at google.com, it should not automatically be considered safe or trustworthy.

The bigger question here is how the rise of Google spam is being addressed by Google themselves? Surely, they are concerned about their name becoming associated with sites hosting malware and spam?

Google would not confirm how they were addressing this specific problem or how they address spam in general, saying that they needed to be careful not to provided spammers with any clues as to what they do. However, they did say that they expect spammers to use every means possible to try to send spam and that they have a very robust spam-fighting effort at Google. They also claim that they disable these accounts immediately and will continue to do so.

Using crowdsourcing to combat spam submissions on an already trained populous that already votes on everything seems like a smart way to outsource an otherwise difficult task.

In addition to these problems, the site also found that the generally American make-up of the moderators left those of you in Australia, New Zealand and other parts of the future stuck with valid submissions sitting in spam boxes, as the moderators were fast asleep.

Thus, Reddit has "deputized" its users, enlisting them all in the battle against spam submissions by including a box that will appear "at the top of the front page every once in a while".

Average Reddit users will have the ability to pitch in and say whether or not a specific submission is indeed spam or was inadvertantly flagged, much the way you can train your email program to detect spam.

We asked Jared Goralnick, founder of AwayFind, how this might differ from standard spam filters on email, to which he replied that the method itself was not novel, but "taking just a few of the messages (the quarantine) and making them very prominent (the the front page of the site) seems novel".

The sort of filtering Reddit is employing, he said, elaborates on the old binary sort of spam filtering, where something is either spam or not spam, adding the quarantine as the third category. While this is not new, the method of dealing with that third category is interesting.

"In short, the technology behind their decision has a deep history..." he said, "...they've always been very community-oriented and this seems like a good next step."

]]> Discuss]]>

"Two years from now, spam will be solved," said Bill Gates in 2004 addressing World Economic Forum in Switzerland. But 2007 will go down as the worst year yet for spam, a trend that has held for the past four years, according to Rebecca Steinberg Herson, vice president of marketing at Commtouch, an email security firm.

In an interview last month with USA Today, Gates reminded that though the volume of spam has increased, more of it is being deleted by spam filters. "Sure, there's a lot [of spam] out there, but software is deleting 99.9% of that anyway," he said, though Microsoft has revised that claim to 85% - 95%.

Gates does have a point -- spam filters are getting better. For example, last January, 50% of all spam was image based, but due to better filtering technology, Symantec now says that the percentage of image spam has dropped to just 8%. These quick advances in spam-fighting technologies force spammers to constantly change and update their methods. Eventually, researchers believe that better filters will begin to reduce spam from a colossal pain to a minor annoyance.

"As more people have inboxes protected by better and better spam filters, their experience of spam gets closer to Gates' vision," Richi Jennings, lead analyst at email security at Ferris Research, told USA Today. "He was a bit overaggressive with the prediction, of course. But spam isn't an easy problem to solve."

Last month we reported on a study from research firm IDC that predicted that 2007 would be the first time that spam out numbered legit email. Our readers didn't think that sounded right: surely spam outnumbered legit email years ago. "Spam sure as hell surpassed legit emails in my inbox -- years ago. Mine. My mom, dad, sisters, brothers, aunts, uncles, every single friend I've talked to about it, my cat and dog, Boobo my hamster, everyone..." wrote one commenter.

Barracuda's report corroborates those feelings and calls into question the IDC report. Certainly, from my own personal experience, it is a lot easier to believe Barracuda. I use three email accounts on a regular basis, and across them, I get about 2500-3000 pieces of spam each week. I get a lot of legit email, as well, but not enough to outnumber the unsolicited stuff. Luckily (for most users), I am in the minority. According to the report, 65% of email users get less than 10 pieces of junk mail per day (half get less than 5). Just 13% find themselves in the unhappy position of receiving more than 50 spam emails per day.

Barracuda's report also found that spam is not only annoying, but it is the most annoying form of junk advertising. 57% of respondents to a survey question asking what the worst form unsolicited advertising was said spam, compared to just 31% for postal junk mail and 12% for telemarketers.

Unfortunately, spammers continue to evolve their tactics to beat the filters. In 2006 there was a rise of image spam and botnets. This past year, spammers were seen using attachments (like PDF files) as well as using more advanced identity obfuscation techniques.

The good news is that spam filtering technology is evolving right along with the spammers, and it works well. Thanks to filters, I only see about 3-4% of the spam I get (which is still a lot given the immense volume). Here's to a spam free 2008 -- hey, a guy can dream, right?

Both the DM email service and the keyword-based spam-blocking feature look like excellent ideas. In addition to blocking spammers to your account, SocialToo will also alert Twitter's @spam account whenever you block somebody and flag them as a spammer (TweetDeck, we should note, offers a similar feature).

Fighting Twitter Spam

SocialToo, of course, was responsible for quite a few of these spam messages in the early days of Twitter; though this February, SocialToo's CEO Jesse Stay announced that the service would end support for auto DMs, and SocialToo had been already offering some basic tools to fight auto-following spammers. Others have now jumped into this market, and the problem is getting more pronounced by the day. Of course, the problem of Twitter spam in general is getting worse, too, and we are still waiting for a good third-party solution to filter spam messages in our regular Twitter stream or for Twitter to put its foot down and implement better spam filters itself.

According Twitter's chief scientist Abdur Chowdhury, the company's Trust and Safety team is "constantly battling against spam to improve the Twitter experience and we're happy to report that it's working." Between May and October 2009, the percentage of daily of spam messages on Twitter regularly exceeded 9% and spiked to over 10 on a few days. As Twitter's primary monetization strategy currently involves selling access to its firehose stream to search providers like Google and Microsoft, it is definitely in the company's best interest to keep the number of spam messages as low as possible.

Report a Spammer

If you want to help Twitter in its fight against spam, remember that you report a spammer directly to Twitter's anti-spam team from every user's profile page. Just look for the "report for spam" link in the right sidebar or in the drop-down actions menu.

Detecting Human-Posted Spam is Hard

We have definitely seen this increase in human-posted spam here at ReadWriteWeb over the last two years or so. While early comment spam was easily detectable because it had nothing to do with the actual post, we now have to take a closer look at all the links our commenters use in their personal profiles in order to weed out the spammers. Often, comments that look perfectly legit will include a link to a Viagra or SEO site in the profile link.

What About Regular Spam?

Besides the rise of human-powered spam, traditional spam is still going strong as well. Akismet notes that "old-fashioned" pill, porn and malware spam still tends to originate from Eastern Europe and the Russian Federation. Spammers there still operate huge networks of malware-infected machines that run spambots.

According to Akismet, the number of fake blog networks on services like Blogspot, Weebly, Tumblr, Ning and WordPress is also becoming more frequent and more highly organized. Instead of just abusing other people's blogs, these spammers just create their own blog networks.

Other forms of blog-related spam that are on the rise are auto-blog pingbacks from people using auto-blogging plugins (mostly for WordPress sites), as well as hijacked blogs and wikis.

From Porn and Pills to Pet Food and Roofing

Akismet also notes that while early blog spammers used to focus on the traditional (and highly lucrative) niches around pornography, pills and malware, today's spammers are often more interested in search engine optimization than hawking fake Viagra. Because of this, modern blog spam often includes links to "dentists, roofing and pet food."

Looking Forward: More Viruses

For 2009, Google assumes that this upward trend will continue and possibly accelerate again, as malware and link-based attacks become more effective. During the second half of 2008, Google noticed that virus volume in email increases six-fold from the first half of the year. While the overall volume of these messages is still small, Google assumes that spammers will increasingly rely on these viruses to rebuild the infrastructure that became unavailable after the McColo shutdown.

Symantec's MessageLabs has been seeing a similar rise (PDF) in spam during the last two months of 2008, and also attributes this to the the rise of new and better botnets.

Google's video sharing site still hasn't instituted as much as a captcha requirement in order to send a message through its service, something that even MySpace did only last week. If the proliferation of spam blogs on Google's Blogspot is any indication it may be a long time before YouTube does anything about spam emails driving users back to their site.

Regardless of all that, unless increasingly high-profile social media outlets like YouTube take effective steps to stop both messaging and content spam we can expect not only more unsolicited email but also a pile of media coverage on the topic. I just thought I'd get in on the news cycle early and put it in context.

Check out a summary of the week's Web Tech action on Read/WriteWeb with our Weekly Wrapup!

They've been working on a solution to cleanse the stream of Twitter spam; their new product tags and blocks computer-generated tweets with a minimal margin of false positives. BotKiller is a product of Rarefied Technologies, an open-source company that implements advanced algorithmic classification for enterprise applications.

The company claims that its "specialized lexical parsing" can find and block computer-generated content by analyzing the metadata and the conversations and relationships between post authors and the larger network.

Currently, the service is focusing on spam filtering for real-time UGC, and Twitter provides a case-study playground for the the product's accuracy and effectiveness. Overall, the company cites a 95 percent accuracy rate for spam filtering with a set of false positives equal to less than one percent of all blocked tweets.

Here, we can see a sampling of blocked tweets from a sample of about 3 million tweets:

Clearly, there are still false positives, some of which do appear to be auto-tweets about new blog posts. Rarefied CEO, Gabriel Ortiz, wrote to us in an email this afternoon, "We haven't really made a decision regarding auto-tweeted blog posts, right now we're trying to tag the ones that are from obvious spam blogs, such as those selling prescription drugs or promoting multilevel marketing scams, but we're not yet blocking ALL such blog post tweets, as some of them might be more legitimate."

According to Ortiz, the BotKiller product is currently just a proof-of-concept for their real-time spam classifier. As such, whether the service will be free for end users or how pricing would be tiered is yet to be determined. "We're hoping we can partner with someone who has a desktop or mobile Twitter client to deliver the filtering service to users," he wrote. "It would also make a lot of sense for Twitter themselves to license our solution so they could just mark as private tweets which are likely to be spam, thus keeping them out of the public search and trending topics without stopping people who wish to read messages of that nature from doing so."

If you click on the links, you'll end up on websites that asks for your email address in exchange for live streaming football games.

We reached out to Facebook about this latest spam attack. Says Facebook's Manager, Public Policy Communications Andrew Noyes:

Protecting the people who use Facebook from spam and malicious content is a top priority for us, and we are always working to improve our systems to isolate and remove material that violates our terms. Recently, our Help Center Community Forum experienced an increase in spam claiming to offer streaming video of American sporting events. We are taking steps to address the issue and encourage people to protect themselves by never clicking on strange or suspicious links.

Just last week a spam attack hit Facebook, flooding the social network with sexually comprising photos of Justin Bieber, abused dogs and a naked grandma across Facebook. Facebook said that attack was the result of a browser bug.

Earlier this year, spammers used Facebook Events to try and trick users into submitting their email addresses.

What can you do to protect yourself? Don't click on links that seem suspicious, and never give away your email address away.

Says Darya Gudkova, Head of Content Analysis & Research at Kaspersky Lab, "naturally, this type of advertising is more interesting and gets more hits." That's bad news for YouTube because when something works, spammers keep at it... with a vengeance. Once word gets around that video spam is more successful than traditional methods, there's no doubt that it will only increase.

How Would YouTube Handle Video Spam?

So what will YouTube do if video spam becomes a real problem on its network? We would like to think that it would take the offending content down, but that could be easier said than done. After all, this isn't like the copyrighted content that their Content Identification tool can easily identify and remove. That tool works by comparing unique signatures somewhat like a digital "fingerprint" from a content owner's copyrighted file to user uploads across the site. Then, if a match occurs, the copyright holder has the option to have the video taken down.

Identifying a spammer's video would be much harder. Just because someone is using YouTube to sell something, that doesn't necessarily mean it's video "spam." That moniker should only be reserved for videos which are truly undesirable messages where fraudulent activities are underway. The question is, how would YouTube know?

Assuming that video spam takes off, the best thing the site could do to police online content is to include a "report spam" button for videos themselves, as it now has for video comments only. 

Of course, for potential victims of video spam, the best thing is not to get duped into visiting YouTube in the first place. Spam filters will simply have to adapt to this new technique. Unfortunately, that will be yet another challenge for Google, which, in addition to owning YouTube, also offers a feature in its webmail product Gmail that automatically embeds any YouTube videos referenced in the email directly in the message itself. That makes it even more convenient for video spammers, who wouldn't have to convince their victims to leave their inbox and launch a new browser window: just click a button on the video embedded below.

Title: Please ignore any Quechup.com Invites - It's a spam engine

Message: I am writing this with profound embarrassment.

I was recently tricked by a spam engine posing as a new social network. I got an invite from someone I knew and trusted, signed up to see what was about. Two days later, Quechup sent out SPAM to my entire address book.

Please ignore or delete all emails claiming to be from me that mention a site called Quechup.com.

Please also accept my apologies for any inconvenience this may have caused you.

I won't mention the name of the person who sent this message, because it isn't their fault. Clearly Quechup has broken the golden rule of social networks and abused the trust of their users. Quechup's tagline is "the social networking platform sweeping the globe" - however it seems to be doing that via an email spam campaign.

Our recommendation: stay away from Quechup. You may also adopt my own current rule and not sign up to new social networks unless there's a clear - and new, unique - benefit to you. After all, how many social networks do you need?

A new report of security company Symantec says that global spam is at its lowest levels since 2008. The geographic center of spammed accounts has also shifted from Russia to Saudi Arabia. Worldwide spam is now down to one in every 1.37 emails. In the United States, spam accounts for 73.7% of all emails.

Spam levels are now the lowest they have been since McColo, a California-based ISP spam control center, was taken down in 2008. That is, in part, due to the shutdown of the spam-sending botnet Rustock in March 2011. Spam, phishing, viruses and other types of malware are all still major problems in the Internet ecosystem but it looks like progress is being made against the botnets and those that control them.

One of the most interesting trends to emerge from the June 2011 report is that pharmaceutical spam is declining yet the prefix "wiki" is increasing in spam messages. In some cases, the two have merged, such as the WikiPharmacy that spam messages are directing users to. Other major spam targets have been tax returns in India and fake aid to Japan after its catastrophic earthquake and tsunami in March. After pharmaceutical spam (which accounts for 40% of all spam messages), adult/sex/dating was the next highest category, with 19% of all messages.

The United States is also no longer a major generator of spam. Spam messages originating from the U.S. declined from 10.7% of all spam in 2010 to 2.8% in June 2011.

Spam may be at its lowest levels in three-plus years, but that does not mean it is dying out or is not a major problem. In June there were still 39.2 billion spam messages sent.

Phishing Evolves, Grows More Targeted

Email phishing is becoming more targeted. Spammers are now using tactics known as "spear phishing" and "whale phishing" designed specifically for a small set of users.

Our enterprise editor, David Strom, reports from Symantec's headquarters in Mountain View, Calif.

"The report shows that virus authors are getting better at micro-targeting: 75% of the malware has infected less than 50 or fewer individual PCs. One virus assembly kit called Harakit is distributed to an average of 1.6 users, meaning that it is used to deliver custom-built attacks that is targeted for a specific individual."

Examples such as Harakit might fit in with "whale phishing" where specific, high-ranking executives are targeted with phishing emails that have been dutifully researched by the phishers and are targeted to get into the executive's computer, which often has access to far more data than a mid-level employee.

South Africa is the most targeted location for phishing attacks with one in every 111.7 emails. The U.S. sees a phishing attempt in every 1,270 emails while Japan sees hardly any (in comparison) at all at with one in 11,179 emails.

Web-based malware is on the rise. MessageLabs identified an average of 5,415 sites each day harboring malware, adware and spyware, an increase of 70.8% from May 2011. That increases the chances of "drive-by" downloads where a user visits a site and becomes infected with malware.