Archive for the 'WordPress.com' Category
I’m finally making progress on getting Bad Behavior 2 debugged and in some sort of releasable shape. I was hoping to have it ready by now, but I had computer problems earlier in the week and spent most of a day and night working that out.
So I’ll be spending this weekend and probably Christmas Day working on Bad Behavior 2. Such is my life.
This is the third in a series of updates on the roadmap to Bad Behavior 2, the next major version of the Web’s premier link spam killer for PHP-based sites of all types.
The Bad Behavior API and callback layers are complete; the core code is now completely independent of the host application, which should make it much easier to port to other PHP-based systems. As proof of concept I’m developing an ExpressionEngine extension, in addition to the traditional WordPress plugin and MediaWiki extension. Other platforms should be able to get on board pretty quickly.
The first pre-release code should be out sometime this weekend, sleep and cash flow permitting. Those of you who are porting to other platforms will be able to work from this codebase with minimal or no changes through the final 2.0 release.
Remember, Bad Behavior is a user-driven project. If you feel that Bad Behavior has been useful to you and want to support its continued development, feel free to send along your holiday wishes. Yes, I know ’tis the season to max out the credit cards. Still, providing you with software that worries about spam so you don’t have to is what I do. And without your support, I’ll have to go do something else. (Thanks again to those of you who already contributed!)
I’ve received several reports that the crawlers used by Syndic8 and PubSub are being blocked by the latest version of Bad Behavior.
The denial message reads, Header ”Pragma” without ”Cache-Control” prohibited for HTTP/1.1 requests.
(This is part of a series of new tests for spambots which claim to use the HTTP/1.1 protocol but actually do not do so properly.)
The trouble has been traced to a problem with those particular bots, and Syndic8 and PubSub have been notified.
Until they are able to fix their bots, you can add them to Bad Behavior’s whitelist.
Update: If you’re being blocked by Bad Behavior, don’t comment here. Read this.
About a month ago I posted a roadmap for the next major version of Bad Behavior, the PHP-based automated link spam killer. Now it’s time for an update.
First off, I mentioned in a comment on a prior post that I would be waiting to see the next version of ExpressionEngine before I went very far with the next version. Though I was told a beta would be available in November, I have yet to see it. If I don’t see it in the next few days, Bad Behavior will move forward, without support for ExpressionEngine.
Second, I have the basic structure of Bad Behavior laid out. It consists of two components: a core consisting of the test suite itself, and a glue component for each host platform. I’m also planning an administrative interface that will hook into each host platform, though I am not sure if this will be ready for all platforms at the time of release. Finally you’ll be able to configure Bad Behavior and view its activity within WordPress, MediaWiki, or whatever platform.
Third, the architecture is in place for Bad Behavior to show more informative error messages, each one including a unique key which either the user or the blog admin can look up to determine what went wrong and how to fix it. While all of the keys have been set, the documentation for each remains to be written. Bad Behavior will now serve errors such as 400 and 403, depending on the request, rather than 412.
And I’m experimenting with automated methods of detecting spam attack runs which may originate from dozens of different IP addresses and have somewhat different signatures. I may call for some assistance with this in the near future, and this isn’t likely to make it into 2.0, but it is in the works.
Finally, this post wouldn’t be complete without a mention of something strange that happened when I posted last month:
Without any further contributions to Bad Behavior development, I’ll work on it in my limited free time, and it’ll take somewhere around six months. If I were to receive, for instance, $500 in contributions, I could devote a significant amount of time to it, and complete it within the next month. Hey, don’t laugh, that’s only a few cents per user.
I didn’t expect to receive much of anything, and I had just picked the number out of thin air. The surprise was that I actually received $490! Clearly I didn’t complete it within a month, but that’s mostly due to my decision to wait for ExpressionEngine. I’m not waiting on them any longer, so you should expect an early Christmas present sometime within the next couple of weeks.
Be sure and review the roadmap and comment on it now, before I go too far and any necessary design changes become difficult or impossible.
And I wouldn’t mind if you want to contribute that last $10 either. It is my birthday, after all. 
Last week I told you all about Automattic Spam Stopper, the new anti-spam solution for WordPress from Matt Mullenweg. There’s been some new news, and you’re going to hear it here first.
First off, the plugin has been renamed to Automattic Kismet, or Akismet for short.
Second, it now requires a WordPress.com API key, which you can find on your WordPress.com Profile page. (Click My Dashboard, then Profile.) If you don’t have a WordPress.com account, you won’t be able to use Akismet at this time, until you somehow finagle yourself an account. The fastest way is probably to use Flock. You don’t actually have to blog at WordPress.com to use Akismet, you just need the account to get the API key. You can use the API key at more than one blog, too.
Matt plans to have Akismet free for personal use, and charge “pro” bloggers $5 per month for the service. He’s defined pro bloggers as anyone making over $500 per month from their blogs. He also has a program set up for large enterprise installations, though I only know of one customer for that right now. However, anyone who participated in testing Akismet prior to today will be grandfathered in and have a free enterprise account forever.
Akismet is surprisingly effective at stopping spam. After having built a sufficiently large corpus of spam to draw from, it’s killing about 99.9% of incoming spam, and has a false positive rate less than 0.1%. However, when the central service goes down, all comments go into the moderation queue. The service has had some downtime, and on the sites where I’ve been testing Akismet, I’ve had to watch the moderation queue fairly closely. Matt says he’s working on new more reliable hosting for the service.
So where does Akismet fit into the overall spam prevention picture?
Akismet has a great advantage over most anti-spam solutions: by seeing incoming spam from all over the Internet, it can identify new spam very quickly, perhaps as soon as seconds after a spam run begins, once it’s in wider usage. It also is better in spam management, having to sort through hundreds of spams to find a legitimate one that might have been blocked by mistake. It presents spam in a compact format that makes it pretty easy to scan through and spot legitimate comments.
However, Akismet has a couple of drawbacks which are common to most anti-spam solutions for WordPress, and a couple of unique drawbacks of its own. The obvious ones are that it’s a for-pay solution for many people who might want to use it. It uses a central server which is subject to downtime. Though Matt hasn’t said much about the secret sauce, it definitely analyzes the content of incoming posts. And finally, it does nothing to keep the spammers from using up your bandwidth and database space.
For most people running a personal WordPress blog, Akismet is the ideal second line of defense. It will entirely replace plugins such as wp-hashcash, Spam Karma 2, AuthImage, etc. In fact, it makes most other anti-spam plugins entirely redundant.
The one anti-spam plugin which Akismet will not make redundant is Bad Behavior. There are several reasons for this. Bad Behavior is a first line of defense, stopping spammers before they can read your site at all, waste your bandwidth, or drop junk in your database. This is especially important for self-hosted sites, or sites hosted on dedicated or virtual dedicated servers, where CPU time and bandwidth are precious. Like most other anti-spam plugins, Akismet does not and cannot conserve its users’ bandwidth, CPU and disk usage from a spam attack. Bad Behavior does, meaning it will continue to be an integral part of most people’s anti-spam arsenals.
You may not think this is important, especially if you have never received a large amount of spam at once. But the day is coming when you will, and having that first line of defense can mean the difference between your site staying up, and your Web host shutting off your site. Spammers can easily hit you so hard as to create denial-of-service conditions, and Bad Behavior has been proven to mitigate this effect. In fact, it’s even stood up to the Slashdot effect without blinking.
I should disclaim at this point. I am involved in the development of Akismet, having rewritten a significant amount of the code from the time it was known as ASS, and integrating CJD’s Spam Nuker into the plugin. I continue to remain involved with Akismet as long as there’s work to do on it (and there are a couple of bugs I need to fix).
As I said yesterday, however, I remain committed to the development of Bad Behavior. It is still sorely needed as a first line of defense for WordPress, not to mention all of the other platforms on which it now runs.
What the future holds? Nobody can say for sure, but I predict that for WordPress users wanting to remain spam-free, the combination of Akismet with Bad Behavior will prove to be a double whammy to blog spammers. For everyone else, Bad Behavior remains the first line of defense, and Matt has said that Akismet could be ported to other platforms as well. Someone else, I think, will have to take up that challenge. My hands are full already.
P.S. Matt’s started a web site for Akismet, where you can find more information.
We see them every day, and usually we make fun of them.
They’re the stupid. The daft. The incompetent. The people who can’t seen to find their way out of a wet paper bag.
Some examples:
People looking for a job application for Target or Walmart (I misspelled that on purpose; if you’re really curious, ask me why, but not in comments) and can’t comprehend the simple fact that the applications simply are not online.
The people so high on methamphetamine they apparently didn’t realize they were smoking right in front of a police station.
The stupidity of government officials and the stupidity of phone companies.
I could go on and on and on and on and on and on and on and on and on and on and on and on and on and on and on and on and on and on and on and on. But I won’t.
The question of the day is: How do we make people less stupid? Is it even possible?
What’s wrong with our world, that so many people live their daily lives in a fog of stupidity?
Recently, Matt Mullenweg, creator of WordPress, had a bright idea on how to stop blog spam. He wrote up some code, distributed his new WordPress plugin to a small group of testers, and so was born the so-called Automattic Spam Stopper, or ASS.
I was able to obtain a copy of Automattic Spam Stopper for review and made a quite disturbing discovery, namely, how it works.
Whenever a user makes a comment to your WordPress blog, ASS forwards a copy of the entire comment, the metadata such as username, email address and URI, as well as your blog address and Web server environment variables, to a central server for analysis. The server then returns the response “true” if the comment is judged to be spam.
Mullenweg isn’t saying what the “secret sauce” is for the server, so as to frustrate the spammers. “By the time we’re done spammers around the world will quiver in their boots,” said Mullenweg.
So how does the server determine what’s spam? Users of the plugin submit copies of any spam they receive by marking them as spam in the WordPress administration panel. ASS then forwards copies of these to the server for analysis.
The submitted spam, however, remains in your database, but hidden from view. This could cause resource constraint (disk space) problems, and backup/restore problems, for many users, especially after time. WordPress does not automatically remove spam from its database, and does not provide any method for removing it from the database. A third-party plugin, however, does provide this function.
Right now Mullenweg inspects all comments submitted this way manually, before the server considers them to be spam. If he judges them to actually be spam, then they are added to the server’s corpus, or database of submitted spam.
He has not said, however, whether legitimate comments are kept on the server, or whether anyone else looks at the submissions. Thus, ASS may not be a good anti-spam choice for private blogs, or for blogs which frequently use password protection to limit access to their contents. In a very real sense it comes down to whether you trust Matt Mullenweg with your readers’ comments. Some people will, and others won’t.
Mullenweg envisions ASS as a service which is free for personal use, and paid for business use. “I would be more comfortable with something where it was free for regular people, and only businesses or enterprises paid (enough to support everybody),” he said.
“There may be ‘keys’ or accounts at some point to prevent abuse,” he said. “However the plugin and API are designed to be pretty easy to recreate, so if someone wanted to run their own spam [prevention] service they could easily.”
That much is true. I could create a server to do this in rather short time. And I almost did. It’s been an idea that’s been discussed before among WordPress anti-spam gurus, and ultimately rejected.
To date no one has been able to provide a centralized server solution which ensures the integrity of the database, for instance. Mullenweg ensures the integrity of his database by inspecting all comments manually, but this “solution” doesn’t scale very well, and is untenable once ASS is released to a wider audience. He has proposed that users be registered and receive keys in order to use the service, but even this doesn’t prevent spammers themselves from registering and submitting garbage to the database.
In addition, no one has been able to provide a centralized server solution which ensures the privacy of users whose comments are subject to this sort of analysis, especially with respect to private blogs and password-protected posts, where users expect their comments to be private. I’ve come up with an idea or two on how this might be done, but I’m not sharing until I’m certain it really can be done; if it were really that easy, it seems that someone would have done it already.
Now if Mullenweg can solve the problems of privacy, integrity, scalability, and those gigabytes of spam clogging up his users’ databases, he may be on to something. But everyone else who’s had this idea ultimately scaled it back or dropped it entirely. I fail to see how Matt’s ASS is any different.
In the meantime, if you’re looking to stop spam without compromising your users’ privacy, consider Bad Behavior, which is shockingly effective despite not looking at the content of comments at all, and Spam Karma, which does, but doesn’t send the whole comment, and much of your server information, off to who knows where.
Update: Some other reviews of Automattic Spam Stopper:
