Al-Muhajabah's Movable Type Tips

more on crapflooding

Last fall, I posted about the CommentSwap plugin. This plugin was originally developed in summer 2002 and I implemented it at that time. The rationale of the plugin was to prevent Google and other search engines from indexing empty comments forms (I subsequently created the TrackbackSwap plugin based on CommentSwap to do the same thing regarding empty trackback listings). When I posted about CommentSwap, I noted a side benefit to it. The way that CommentSwap works is to link to different-named versions of MT's comment script depending on whether or not there are comments on the entry. You can then use robots.txt to prevent Google from indexing the version that's linked where there aren't any comments. That is, you're preventing Google from indexing the empty comments forms while not preventing it from indexing the ones with comments. The side benefit is that spambots usually just look for links to mt-comments.cgi or forms that post to mt-comments.cgi. They want to hit as many sites as possible, so they don't want to spend time on each site trying to find out the name of the comments script. If they don't find mt-comments.cgi on your site, they'll probably move on to an easier target. So, when you're using CommentSwap, your entries without comments are protected in a small way from spambots. For most bloggers, this is a pretty substantial number of their entries. This is a side benefit. Obviously, you would want to protect all your entries from spambots, whether they have comments or not. But if you had previously implemented CommentSwap, you were already part protected when the comment spamming began, as I was. Real anti-spam measures involve renaming your comments script entirely and preventing Google from indexing any of your comment forms. These pretty much obviate the purpose of CommentSwap. Still, when I renamed the other version of the comments script earlier this month, I kept CommentSwap around, mostly because I was too lazy to modify my templates that have the tag in them. Besides, having two variant names is better than one: if a spammer figures out one, that will still only work for some of your entries, not all of them. Crapflooders are different from spammers. Their aim is to bring down your site, while spammers want to get their link displayed. A lot of the tricks used against spammers will also help protect you against crapflooders. However, because the crapflooder is targeting your site specifically, they're more willing to take the time needed to find your renamed comments script, which is probably not the case with most spammers. This isn't always the case; it wasn't in the crapflooding attacks I experienced. The crapflooding attack against me appears to be a "bias crime"; I was targeted because of my religion, by a person whose primary motivation was probably that bias. As it turns out, there's a backstory to the larger phenomenon of crapflooding. Somebody designed scripts for the express purpose of targeting MT blogs and made them freely available on the web. My crapflooder(s) apparently grabbed it from there, but he wasn't very clever in using it. I can't count on being that lucky again, so I'm continuing to look into measures to protect my site against future crapflooding attacks. I've detailed some of these previously (and here). One of the things I did was to force comment posters to go through preview mode. This is just a matter of removing the "post" button from the comments posting form and leaving only a "spell-check" (my name for preview) button. Yoz Grahame suggests (see tip 5) a further step. Have two different versions of the comments script. One is linked to on your site and the comment posting form uses it. The other is used by the preview form. The trick is that the first version is hacked so that it can only preview. The fully-functional version is hidden in the preview form so that it takes some digging to find its name. Implementing this on my site was complicated both by CommentSwap and by the use of EZ Subscribe. However, once I figured it out, it was relatively easy to implement and involved hacking and uploading the new "external" versions of the script (the ones linked to CommentSwap), uploading a third and fully-functional version of the script and editing mt.cfg to change the name of CommentScript (you have to take this step whenever you change the name of your comments script). It certainly doesn't make the site impregnable; nothing does short of not having comments at all and I'm really depending on the third line of defense, throttling, to prevent any attack from getting too bad. But it's another way to make your site that much more protected. Update: Comments have been closed on AMIB for a long time, but I had implemented this hack on my MT 3 installation (which houses this blog and a number of others). It worked great as an anti-spam measure. However, with the upgrade to 3.2, the mt-comments.cgi script has changed so that I don't know how to make a copy that can only preview. So I had to give that up. Sure enough, I'm starting to see more spam :(

dealing with crapflooding

"Crapflooding" is a type of denial of service (DOS) attack on your blog in which a script is used to flood the blog with hundreds of comments a minute (trackback crapflooding and search request crapflooding are also possibilities). As with regular DOS attacks, this may bring down your MT installation, your site, or even your webserver by overloading it. Crapflooding also leaves you with hundreds of comments to delete. The unsophisticated crapflooder does his work from a single IP address. For this, MT 2.661 and MT-Blacklist (get 1.63 beta or later for compatability with MT 2.661) are sufficient. MT 2.661 incorporates IP throttling. If more than a certain number of comments (or trackbacks or search requests) are submitted from the same IP address within a certain amount of time, the "throttle" kicks in and prevents additional comments (etc) from being submitted. MT 2.661 also automatically bans any IP address that triggers the throttle a certain number of times. And MT-Blacklist allows mass deletion of comments (or trackbacks) from the same IP address as a blacklisted comment or trackback. Thus, MT 2.661 and MT-Blacklist can stop an unsophisticated crapflooder before too many comments are posted and make the clean-up easy. Unfortunately, many crapflooders are more sophisticated and engage in Distributed DOS attacks. In a DDOS attack, open proxies (usually in developing countries with poorly regulated ISPs) are used to create the appearance that each server hit is coming from a different IP address. MT 2.661 and MT-Blacklist are unable to stop a DDOS crapflooder because throttling never kicks in as long as different IP addresses are used, and MT-Blacklist can't bulk delete either (MT-Blacklist can also filter by text string, but only if the comment contains a text string that you can blacklist without killing off legitimate comments, and some DDOS crapflooders also submit random characters for the comment name, email, URL and text). Fortunately, the MT community includes some top-notch programmers who have developed solutions, at least to the first problem. Shelley Powers explains the steps to a safer blog with links to files that permit true comment and trackback throttling: a certain number of comments or trackbacks in a certain amount of time activates the throttle, regardless of IP address. I didn't pay too much attention to the latest security developments until I got hit by a crapflooder (two crapflooders, actually). The first crapflooder was sophisticated and hit me with a DDOS attack. The second one was unsophisticated and used a single IP address. In order to stop the first attack in its tracks I had to delete the MT comments script from my server and that also prevented the second attack from accomplishing anything. I then had time to upgrade to MT 2.661 and get the latest version of MT-Blacklist. Then I did some hunting around until I came across Shelley's blog entry. One other thing can be learned from this experience. These losers are using an automated script. The first crapflooder didn't notice for awhile when he started getting 404s after I removed the comment script. The second crapflooder tried for four hours but apparently didn't realize or understand that he was getting 404s. If the crapflooder is using someone else's script, he probably just has to enter the URL of your site and an entry ID and the script does the rest. For this reason, tricks such as changing the name of the comment script, forcing commentors to go through preview mode before posting, or using a CAPTCHA (an image file of a word or number unique to the entry that must be entered in a form field) can be effective. The crapflooding script only looks for mt-comments.cgi and not some unusual name you've given the script, and it doesn't know to enter a CAPTCHA or that posting to the comment form only sends the comment to preview mode.

more ways to soup up MT search

Using a hack by Stepan Riha, you can give your visitors the option of choosing which fields in an entry they wish to search. The default is to search all of them (title, entry body, extended entry, and keywords) but the hack allows visitors to select only some of the fields for their search. There's also the PartialWords hack. With this, your visitors can search for "mov" and get back results for all of "movable", "moved", and "moving", among other things. Once you've applied the hacks, you'll have to add the appropriate code to your templates. The instructions for each hack give suggested code, or you can look at my search template (scroll to bottom of page).

get better control over your rebuilds

Adding new tricks to MT is fun, but after awhile, it can start slowing down your rebuilds, especially when comments and trackbacks are posted, which affects your visitors, not just you. Unfortunately, MT doesn't offer a lot of rebuild options out of the box. Along comes Sean Willson with his rebuild type hack. This is for advanced users only, since it requires substantial tinkering under MT's hood and in the database. Follow the instructions at Sean's site. If you've already hacked some of your MT files, I recommend using the patch files instead of uploading his new versions. That way, you don't overwrite your hacks. Once you've patched all the files and made the database modification, you will have a bunch of new rebuild options for each index template. You can rebuild it only when new entries are added, only when new comments are added, only when new trackbacks are added, when new entries or new comments are added, when new entries or new trackbacks are added, when new comments or new trackbacks are added, when anything is added, or never. If you have templates that don't display comments or trackbacks, nor list the number of these, you can set them to rebuild only when a new entry is added. Now, when new comments are posted or trackback pings sent, the rebuild process will go just a bit faster. This is also useful for your RSS feed, since it prevents the feed from showing as updated (because it was rebuilt) when a comment was added or ping sent but no new entries have been added.