Archive for the 'Bad Behavior' Category
Recently it was suggested to me that Bad Behavior could incorporate support for Stop Forum Spam.
Stop Forum Spam is meant to be a list of IP addresses, emails and usernames which spammers use when registering or posting spam to forums. It seems to work well, but it has some shortcomings.
First among them is it has no native support for DNSBL. Instead, it exports its data to a third party DNSBL where the data is commingled with other data from unknown sources, making it difficult to use effectively.
Second is that it has no clearly defined removal policy. It does provide a form where people can request manual removal, but it also implies that a “network administrator” has to request removal.
After much experimentation with blackhole lists over the years, Bad Behavior currently uses only the Project Honey Pot http:BL list (and it is disabled by default). This list works very well at catching actual spammers, and it provides instant automatic removal for the very few legitimate users who happen to get caught by it.
Bad Behavior is meant to provide as little inconvenience to legitimate users as possible. When it happens, the user must be given clear directions on how to resolve the problem and ideally must be able to restore their access as soon as possible, e.g., by removing the viruses from their computer, etc.
Because it lacks a removal policy and clear process, it will not be appropriate to incorporate Stop Forum Spam at this time. I will continue to monitor the service and if it changes to allow for easier removal by legitimate users, then it may be incorporated in the future.
Bad Behavior 2.1.2 has been released. This release fixes bugs and is recommended for affected users as described below.
Please note: The 2.0 series of Bad Behavior is receiving limited updates, including unblocks, bug fixes and security fixes only. Future development is taking place in the 2.1 development tree.
Who should upgrade?
Users who use the new URL whitelisting feature should upgrade to ensure that whitelisting works correctly in all circumstances.
What’s new?
New in this release (since 2.1.1):
- A logic error in the URL whitelisting feature caused URLs to fail to match the whitelist if the if the web browser requested a URL containing a ? character. This issue has been fixed.
Download
The 2.1 development releases will not be offered through the WordPress automatic upgrade facility.
Download the 2.1.2 development release of Bad Behavior now!
Support
This release would not have been possible without the support of people like you who find Bad Behavior valuable enough to make a financial contribution to ensure its further development.
Your contributions ensure that I can continue to devote time to bringing you the features you want, as well as continuing work on making spammers’ lives hell.
If you haven’t already done so, consider setting up a recurring contribution for as little as $5 per year, or make your most generous one-time contribution for any amount.
Thank you again for supporting Bad Behavior development!
Bad Behavior 2.1.1 and 2.0.36 have been released. These are a security release and affected sites should upgrade as soon as is practical. This security issue was fixed in both the 2.1 development series and the 2.0 stable series, resulting in today’s simultaneous release.
Please note: The 2.0 series of Bad Behavior is receiving limited updates, including unblocks, bug fixes and security fixes only. Future development is taking place in the 2.1 development tree.
Who should upgrade?
WordPress users should upgrade to prevent internal data from leaking to the web browser when the database encounters an error. Users of other platforms are not affected.
What’s new?
New in this release (since 2.1.0 and 2.0.35):
- Due to recent changes in the WordPress database code, any database errors that may occur because of WordPress, other plugins, or server trouble may be inappropriately displayed in the web browser. This could result in the leakage of information useful to attackers. This issue has been fixed. Thanks to Andrew Zhang for reporting this issue.
Download
The 2.1 development releases will not be offered through the WordPress automatic upgrade facility.
Download the 2.0.36 stable or 2.1.1 development release of Bad Behavior now!
Support
This release would not have been possible without the support of people like you who find Bad Behavior valuable enough to make a financial contribution to ensure its further development.
Your contributions ensure that I can continue to devote time to bringing you the features you want, as well as continuing work on making spammers’ lives hell.
If you haven’t already done so, consider setting up a recurring contribution for as little as $5 per year, or make your most generous one-time contribution for any amount.
Thank you again for supporting Bad Behavior development!
The first 2.1 development release of Bad Behavior is now available. It contains a number of new and frequently requested features, and may be appropriate for you. Please review the information given, and if you do not find it appropriate for you, then continue to use the latest 2.0 stable releases.
Who should upgrade?
Users who use Bad Behavior’s whitelisting features, or who customize Bad Behavior’s settings on a platform other than WordPress or LifeType, should upgrade to take advantage of new features offered in this release.
What’s new?
Development of Bad Behavior 2.1 generally follows the roadmap outlined earlier. In this initial release, the following features have been implemented:
- Bad Behavior now reads whitelists from a separate file which is preserved through updates. See below for preliminary instructions on using this feature.
- On platforms where Bad Behavior cannot store settings in the host platform’s database, Bad Behavior now reads settings from a separate file which is preserved through updates. See below for preliminary instructions on using this feature.
- Bad Behavior’s core has been reworked to facilitate testing its core logic. While the actual logic tests have not yet been written, a test mode is available for developers to experiment with. See below for preliminary instructions on using this feature.
Whitelists
Bad Behavior now reads its whitelists from a separate file named whitelist.ini. This file is not distributed with Bad Behavior, so that future upgrades do not disturb the whitelist. This means that anyone who wants to use the whitelist must download the whitelist.ini, customize it, then upload it to their server. Place the whitelist.ini in Bad Behavior’s top level directory (the same directory that contains bad-behavior-wordpress.php, README.txt, etc.).
Note for IPv6 users: At this time, single IPv6 addresses can be whitelisted, but IPv6 networks cannot be. This will be fixed in a future release.
Settings
On some platforms, such as WordPress and LifeType, Bad Behavior stores its settings in the host platform’s database and provides an interface through the host platform for changing the settings. On other platforms, Bad Behavior is not capable of storing its settings in the host platform’s database, either because there is no database, or because the database cannot be used in that way.
On these platforms, Bad Behavior can now read settings customizations from a settings.ini file. This file is not distributed with Bad Behavior, so that future upgrades do not disturb your settings. This means that on those platforms, anyone who wants to customize their settings must download the settings.ini, customize it, then upload it to their server. Place the settings.ini in Bad Behavior’s top level directory (the same directory that contains bad-behavior-wordpress.php, README.txt, etc.). This feature has been implemented for the MediaWiki and generic ports; other platforms will need to implement the feature in their platform connectors before it is available to you.
Testing
Bad Behavior’s core logic now supports “black box” testing. This won’t be of much interest to most people, except that testing will help improve the quality of the product. A test suite is still planned and will be released later.
In addition, Bad Behavior now supports a live “test mode” in which it will not actually block any requests, but will report on whether they would have been blocked. This is fully implemented in the WordPress port; to use it on other ports, the platform connector must provide a method for the platform to report the results. To enable test mode, define a PHP constant BB2_TEST.
Download
The 2.1 development releases will not be offered through the WordPress automatic upgrade facility.
Download this development release of Bad Behavior now! You can install Bad Behavior using the usual installation instructions; there are no special requirements for this release.
Remember to subscribe to the Bad Behavior RSS feed to receive notice when Bad Behavior development updates are available.
Support
This release would not have been possible without the support of people like you who find Bad Behavior valuable enough to make a financial contribution to ensure its further development.
Your contributions ensure that I can continue to devote time to bringing you the features you want, as well as continuing work on making spammers’ lives hell.
If you haven’t already done so, consider setting up a recurring contribution for as little as $5 per year, or make your most generous one-time contribution for any amount.
Thank you again for supporting Bad Behavior development!
Bad Behavior 2.0.35 has been released. It is a maintenance release and is strongly recommended for users of shared web hosting services.
Please note: The 2.0 series of Bad Behavior is receiving limited updates, including unblocks, bug fixes and security fixes only. Future development will be to the 2.1 development tree.
MediaWiki and WordPress users who have not updated in the last year or so should take note of special upgrade instructions below.
Who should upgrade?
Users whose sites are on shared web hosting services should upgrade to prevent “MySQL server has gone away” database errors. Users whose sites are on dedicated servers or virtual dedicated servers should review the information below before upgrading.
What’s new?
New in this release (since 2.0.34):
- In very rare circumstances, Bad Behavior users may see MySQL database errors stating “MySQL server has gone away.” This issue does not occur in Bad Behavior’s default configuration. It primarily affects shared web hosting providers using cPanel/WHM, though it may affect other shared web hosting providers or, even more rarely, people who run their own virtual dedicated servers or dedicated servers. A workaround has been placed in Bad Behavior to mitigate this issue.
In order for this issue to occur, three things must happen:
- Bad Behavior must be configured to use the http:BL DNS-based list (which is disabled by default);
- A DNS query sent to http:BL must take an unusually long time (several seconds or more); and
- The time the DNS query takes must exceed the MySQL server’s configured wait_timeout value.
In this scenario, because MySQL has not seen a database query for a long time, the server drops the database connection. Shared web hosting providers set the wait_timeout value relatively low in order to preserve resources on their typically very active databases; each open connection uses scarce memory that could be serving another user. There is nothing wrong with this, in theory, though a few hosting providers set wait_timeout so low as to be frequently unusable. For instance, cPanel/WHM based web hosts may have wait_timeout set to 10 seconds, which I feel is completely unreasonable.
The “MySQL server has gone away” error appears when the host platform under which Bad Behavior runs (e.g. WordPress, Drupal, etc.) fails to trap the error and reopen the database connection. There is nothing Bad Behavior can presently do to cause the host platform to reconnect to the database. If your host platform does this, its database code needs to be reworked so that it reconnects and resends the query when it gets the “MySQL server has gone away” error.
Another issue here is that DNS lookups typically take less than a second. When http:BL is active, Bad Behavior screens the IP address for all POST requests against it. The Internet not being 100% reliable, it’s expected that occasionally DNS queries will be slow. If DNS lookups consistently take significantly longer, (e.g. they’re slow for more than a day) this may indicate trouble with the DNS servers being used by the web server. There isn’t much that Bad Behavior can do about this.
In the meantime, the workaround released today will ask the MySQL server to temporarily raise the timeout to 90 seconds, for that connection only, whenever it does a DNS lookup. (The timeout for most requests is left alone.) This should be more than sufficient; with my own server set at 60 seconds I have never seen this error.
If you continue to see “MySQL server has gone away” messages and/or slow loading times when you log in or navigate your administrative pages, disable http:BL. If that still doesn’t help, you may have other trouble; contact me and we’ll figure it out.
Finally I want to thank Jen Lepp at DrakNet Web Hosting for providing the information which finally allowed me to track down this rare but annoying issue.
Support
Thank you to everyone who has chosen to make a financial contribution toward further development of Bad Behavior. Your contributions ensure that I can prioritize Bad Behavior development and make more frequent and timely releases, like this one.
Download
Download Bad Behavior now!
Special Upgrade Instructions
Users of MediaWiki and WordPress upgrading from version 2.0.20 or earlier should follow these special directions (from 2.0.21 or later, upgrade normally):
For MediaWiki: Before installing this version of Bad Behavior, manually remove (e.g. using FTP or ssh) any old versions you may have, including the lines added to LocalSettings.php. Then install the new version fresh, following the installation instructions for MediaWiki.
For WordPress: If updating to this version through the automatic updater fails, manually remove (e.g. using FTP or ssh) any old versions you may have installed. Then upload and install the new version fresh, following the installation instructions for WordPress. After doing so, future automatic updates should proceed normally.
For other platforms: No changes to your upgrade procedures should be necessary.
Don Jones from Concentrated Technology wrote in to tell me that the Cahoots web collaboration software will ship with Bad Behavior as of version 2 RC2, which was released moments ago.
Cahoots is described on its SourceForge wiki page as “a PHP-based knowledge-exchange community application.” From viewing their demonstration site and feature list it looks like an interesting and useful way to do project collaboration.
If you’re a Cahoots user, welcome to the Bad Behavior community!
Users of Bad Behavior on Microsoft Internet Information Server 7.0 may find that Bad Behavior’s error pages are not always displayed to spammers and malicious crawlers.
This behavior occurs because of a misfeature in IIS 7.0 where it serves custom error pages by default, overriding the error page generated by your web application. This is, of course, utterly broken behavior, and just one of many reasons you shouldn’t use IIS on Windows to serve your web site.
But some people still want to do it, so here is a workaround to allow Bad Behavior (not to mention every other web app you have) to display its error pages properly.
You will need to know your way around IIS already; I cannot give you any support for this.
You will need to modify your IIS configuration to set the web server httpErrors element to “passthrough”. An MSDN blog post from the IIS team explains how to set this property for an individual web site or for your entire server.
Not only does this provide a fix for Bad Behavior, it also fixes WordPress 404 pages, as well as numerous problems where a web application expects to generate its own error page. Of course, it’s utterly ridiculous that this requires a workaround and is not the default behavior, as it is with every other web server. But that’s one more reason not to use IIS.
Bad Behavior 2.0.34 has been released. It is a maintenance release and is recommended for specific users of WordPress identified below.
Please note: The 2.0 series of Bad Behavior is receiving limited updates, including unblocks, bug fixes and security fixes only. Future development will be to the 2.1 development tree.
MediaWiki and WordPress users who have not updated in the last year or so should take note of special upgrade instructions below.
Who should upgrade?
WordPress users who use the W3 Total Cache plugin should upgrade to ensure that users are not blocked inappropriately due to flaws in W3 Total Cache.
What’s new?
New in this release (since 2.0.33):
- On some WordPress installations which use the W3 Total Cache plugin, W3 Total Cache could inappropriately store the error page which Bad Behavior serves to illegitimate requests. When this happens, the cached error page would be served to subsequent legitimate requests. Bad Behavior 2.0.34 contains a workaround which forces W3 Total Cache to not cache these error pages. (To be clear, W3 Total Cache is still broken and needs an update, but this resolves the immediate problem.)
Authors of caching plugins should consider following the “standard” set by WP Super Cache: check for a constant DONOTCACHEPAGE which can be set by other plugins; and checking to ensure that non-cacheable error responses are never cached, regardless of which plugin generates them.
Support
Thank you to everyone who has chosen to make a financial contribution toward further development of Bad Behavior. Your contributions ensure that I can prioritize Bad Behavior development and make more frequent and timely releases, like this one.
Download
Download Bad Behavior now!
Special Upgrade Instructions
Users of MediaWiki and WordPress upgrading from version 2.0.20 or earlier should follow these special directions (from 2.0.21 or later, upgrade normally):
For MediaWiki: Before installing this version of Bad Behavior, manually remove (e.g. using FTP or ssh) any old versions you may have, including the lines added to LocalSettings.php. Then install the new version fresh, following the installation instructions for MediaWiki.
For WordPress: If updating to this version through the automatic updater fails, manually remove (e.g. using FTP or ssh) any old versions you may have installed. Then upload and install the new version fresh, following the installation instructions for WordPress. After doing so, future automatic updates should proceed normally.
For other platforms: No changes to your upgrade procedures should be necessary.
Advisory: WordPress users who use caching plugins should check to ensure that the caching plugin does not cache error pages. This behavior violates Internet standards and may cause users to be blocked from your site. This issue may also affect caches external to WordPress, such as squid and ISA, and content distribution networks such as Akamai. See below for details.
In the last 24 hours I’ve received complaints from Bad Behavior users that legitimate requests are being blocked. These users are using WordPress caching systems. In each case, the caching system was inappropriately caching the blocked page which was served to an illegitimate request. The caching system would then serve the blocked page to subsequent legitimate requests.
To be perfectly clear, this is a problem with the cache, not with Bad Behavior. The HTTP standard, RFC 2616, explicitly prohibits caches from “negative caching,” or storing the types of 4xx error pages which Bad Behavior serves to illegitimate requests. (The only cacheable error is 410, and Bad Behavior does not use this error.)
Currently I know of two WordPress caches which have this problem: Hyper Cache and W3 Total Cache. There is currently no workaround; to resolve the problem, either Bad Behavior or the caching plugin must be disabled.
In the case of Hyper Cache, you can replace it with WP Super Cache, which does not have this problem. There is no comparable replacement for W3 Total Cache; it’s otherwise an excellent product which combines many different techniques to speed up your site.
Other WordPress caches may be affected as well. If you don’t see your favorite caching plugin listed below, leave a comment and I’ll test it for this issue.
Current test results with Bad Behavior 2.0.33 and WordPress 2.8.6:
Batcache 1.0 = OK
Hyper Cache 2.6.3 = Broken
W3 Total Cache 0.8.5 = Broken
WP Super Cache 0.9.8 = OK
With respect to external caches and content distribution networks, normally these do not engage in negative caching. However it is possible to configure them to cache error responses. When this functionality is used, it should be limited to caching 404 and 410 errors.
Finally, the development roadmap includes features which will let Bad Behavior communicate to other plugins whether it has approved or blocked a request. If you want to support this feature along with future Bad Behavior development, consider becoming a sustaining contributor.
Spam isn’t the only threat to your web site.
Another very real threat is criminals who use automated attacks against thousands or even millions of web sites, hoping that a few will let them in so they can take over your site, forcing malicious software on your unsuspecting visitors and posting as many links to their garbage as they want.
On Monday the SANS Internet Storm Center noted one such attack seen in the wild which uses a distributed network of virtual machines that all talk to each other and share data on which passwords they’ve tried against which WordPress blogs.
After obtaining a copy of the attack script and testing it in a virtual lab, I’ve determined that Bad Behavior already blocks this script as it is currently written.
Even so, the script has given me some good ideas on how to improve Bad Behavior further to protect against malicious attacks of this type. I will be rolling out some of these changes in the following days in the 2.0 branch.
The first release in the 2.1 development branch will be coming later this month, as well. If you want to see it sooner, consider becoming a sustaining contributor to Bad Behavior development. Your contributions ensure that I can devote development time to Bad Behavior on an ongoing basis.
