Botbusting

Okay, this week has heated up a great deal for me, so it doesn't look like I'll be able to continue my coding series until Monday. In the meantime, I want to talk a little about combating bots of different sorts. OnionRings knows more about the technical and security stuff than I do, so he may have some stuff to add or correct. Maybe he'll do some stuff about tightening up the backend, but my expertise is on the frontend/touchy feely stuff, so I'll just focus on that.

A CAPTCHA is an image that is (ideally) impossible for a computer to read but easy for a human. It's used to verify that a user is not a bot, rendering brute-force attacks unfeasible. However, even if it is easy for a human, it's also very annoying. For that reason, you need to use them with moderation.

The biggest technical threat is brute-force attacks on account logins. If someone gets the password of the admin account... well, I don't have to tell you the kind of hurt that can result. Combating this can be as simple as implementing brief (15-minute) bans after a certain number of failed login attempts. I don't think it's feasible to use CAPTCHA in this case because it's something that everyone will be presented with, especially if they don't have their browsers set to keep them logged in. I dislike entering more than a couple of them a day, and any unnecessary ones annoy me. However, if you implement temporary IP bans and change your password regularly, it should afford you a reasonable amount of protection. Other options here may be to require CAPTCHA on all attempts after the first one (decided by IP, not cookie), or to create an account that is used only for administration and give lower permissions to your regular account.

You can use CAPTCHA on registration, but I don't like to spend too long filling out registration forms, either. Besides, a human can register, then let a bot loose on the site. It's still not a great solution, although if you have fairly generic registration forms (or use an unmodified script), that may be a concern.

Instead, I suggest requiring CAPTCHA on a user's first few posts/comments/messages. After the first few, no more annoyances. One idea I just came up with and have yet to implement is simply to require CAPTCHA for all users with a 0/0 ratio. If you're using your account, you're likely not a bot.

No matter how careful you are, some will inevitably slip through. As I've mentioned before, proactively promoting moderators means that what does get through can be dealt with reasonably quickly. This is especially nice in the case of porn spam with images attached, which is never nice to be presented with unexpectedly.

If you have any other suggestions, I'd love to hear them.

A Moderator's View

Unfortunately, I fell asleep before I could write what should have been yesterday's entry. I hope you didn't miss us too much. I'd like to focus a bit on some experiences I've had as a moderator of TorrentFries. I wasn't always an admin or even moderator at TorrentFries. One day I logged in to find a full host of "shiny new buttons," as we like to call them at TorrentFries, and what a true surprise that was. It's a gratifying feeling to be an uploader and to give to the community, but as a moderator and admin, that responsibility takes on a deeper meaning of giving to the community. I'm glad to have been able to give in that unique way.

As a new moderator, some of my initial duties included moderating the forum, responding to torrent reports and weeding out torrents that didn't follow the rules. As a moderator team, it's obviously pretty important that everyone gets along so the work gets done without too much hassle. If things get slack and neglected, it's apparent right away.

For instance, at one time we were experiencing problems that required users to use the staff contact form to fix, but nobody was addressing the problems. When 25 or 30 messages piled up, it became difficult to manage. I eventually got enough time to clear out the problems, but it took a full 2 hours to take care of. The point is this: our team is good, but without clear direction or responsibility, things tend to not get done if there's a lot of work to get done. The admin has to either pick up this slack or divvy it up, but sometimes that choice has to be made.

Another problem we saw was spam in the forums. Thankfully, there were mods checking the forums, but our registration system didn't include CAPTCHA or any other deterrent to bots, so we began seeing a heavy influx of pornographic material posts. Without fixing the root of the problem, it was difficult to keep the issue under control, and with CurlyFries busy he couldn't rewrite a part of the login. No one else could do it, so we just kept getting bombarded with spam.

Save small issues like these two mentioned, the site as a whole could easily run itself without CurlyFries logging in at all. Nevertheless, there needs to be some direction and fiddling done by the sysop no matter how well the mods work.

Additionally, there always seems to be something that mods should be doing instead of bantering on the forums. It's difficult to moderate torrent comments with any efficiency, and even ensuring that the uploaded content follows the rules seems take a backseat at times. So, in short, as an admin, aim to get a staff that can run the site, but expect that they won't always do everything on their own accord.

How Not to Admin a Tracker

I'm assuming most of the audience that is actually moving in the tracker direction is running or planning to run a small operation, for the time being at least. That means spamming all over the place for your site, and that means your previous identity (with all its slip-ups) is irrevocably connected with your site.

Chances are, when you first got involved in the P2P community, you were scared shitless. If you weren't, you should've been. If you were paranoid, you should have made a new account unrelated to any other you might have online. If you didn't, go do that now. Otherwise you're opening yourself up to a whole world of hurt. After a while, you start to relax, to become blasé about the whole copyright issue, and you start to get sloppy again.

Now take this unconcerned you and put it in charge of a BitTorrent tracker. You're probably a little concerned, but that goes away fast. It's just a small operation, after all. It can't hurt.

The problem is that small operations have a way of becoming big operations, and your behavior has a way of sticking around. Even if you've been careful to delete posts as you grow, the curse/blessing that is archive.org will remember your transgressions. All of a sudden, you're at the head of a booming site, and oh shit, what have you done? You go to cover your tracks, but it's too late, of course.

Chances are that if you're headed down this road, you're not far along. You can still change now. Don't make these silly mistakes. If you're going to be involved in your own community, which I recommend, create a new username for it. Don't let hubris enter into the equation. It's tempting to join other torrent sites and go "look at my awesomeness!". Don't. Your admin username should be used only in the context of your own site, and in your dealings with other sites. It's tainted in a different way, and it's dangerous to mix things together.

As well, it's helpful to have an alter ego. Create somebody that is plausibly of the same character as you, but comes from a dramatically different background. Trust me, it's a good idea. You're role-playing, and you should never drop that identity, even with your closest online friends.

Of course, this is the product of much painful experience. Do as I say, not as I do. Accessible as I want the prospect to be, it can be dangerous work, and you have to take it seriously, no matter the size of your site.

The Rules: Writing a Constitution

Yesterday I walked into a shop and was greeted by a sign thanking me for not urinating in the produce. Wait, no I wasn't. You know why? Because there are certain codes of behavior that are implicitly expected of all members of a society. Yet somehow sites feel the need to inform me that I could be banned if I start stirring up hell in the forum. Great, thanks for that.

Quite simply, there are some things that just don't need saying. Sure, you're going to get trolls in your forum, but making a rule telling them to go away is going to do nothing for you. Rules are for the people that genuinely care about their standing with the site. The people that are just around to make trouble aren't going to be reading your rules, and certainly won't make a point of abiding by them. You can't legislate common sense.

So what are your rules for? They're for people like me. I'm a member of a bunch of different trackers, and whenever I sign up, I always take a peek at the rules before I get down to business. I know what a BitTorrent tracker is, I know the usual etiquette, but I just want to find out if there's anything special about how your site does business. Do I have to post in rhyme? Have a cute, fuzzy avatar? Download only certain torrents? I don't want to dig through a bunch of bullshit about obeying "the moderators [sic] expressed wishes! [sic]" in order to find out what really matters.

Conversely, a lot of sites that have automatic ratio bans (which I've ranted about in previous posts) won't detail these bans in their rules. The hardest and fastest (and most variable) rule of all is somehow overlooked. The staff don't have to touch it as the banbot does all the work, but it's the rule with the most impact on new users, good and bad alike. Whether or not I agree with the concept is a moot point, at least tell me.

You're writing a constitution here, not a criminal code. If you've never read your country's constitution, go do it. It's probably available online. In general, they're pretty succinct and accessible for all their importance, and you'd be surprised what rights are being violated. Your criminal code, by contrast, is probably a few thousand pages long. The constitution is basically a vague statement of purpose, while a criminal code gets into the nitty gritties. Since your moderators are not police officers bound by the letter of the law, you don't need a criminal code. You simply need to give guidelines by which your site should operate.

A lot of sites have 15-20 rules, mostly the stuff that came preset with TBdev or whatever they're using. In my mind, you should be able to get everything you need in less than 10, preferably 5 general rules and 5 upload rules, in addition to the behavior that one normally expects of a generic good member elsewhere on the internet. The fewer rules you have, the better.

That's not to say you should have people responding to your moderation actions complaining that they didn't know something was against the rules. If it's the sort of rule a person can't be normally expected to know, then by all means throw it in there. Otherwise, forget about it. You're entrusting your moderators with plenty of discretionary powers, so you don't need to spell everything out.

I'd finish up with some examples from TorrentFries, but quite simply, it's stuff that applies only to that site and to none other. That's kinda the point.

Perils of Remote Access

I thought since I've shared quite a bit of technical information in my past few posts that it was time for an admin's parable. I wrote a little about the perils of remote access servers earlier, but I didn't really go into detail about how 'normal' activities should be rethought in the environment of sole access being remote. Remote administration can be a wisening experience, unless you have some out-of-band administration tool (such as a DRAC card). With may hosts, you may only have the option to do a hard reboot or a reimaging (often at a fee).

In one case, CurlyFries was looking to add a static IP to the server to move critical services off the primary IP and domain. Most services, such as SSH, can listen on any address and can be restricted to certain addresses. It's handily secure to only have ports 80, 443, and 5222 open on a heavy-trafficked IP address.

It's fairly straightforward to add a static IP to a system. I would never dream of being timid about making such a change. At least not nearly as timid as when making a critical Apache or MySQL configuration change. Nonetheless, I managed to screw up with a typo in the /etc/network/interfaces file and got a serious error when restarting the networking. Thankfully, I hadn't lost terminal access, so I went back to look for the problem.

When I thought that I found it and fixed the problem, I restarted networking again. No joy. I was well aware of the implications of having a bad network interfaces file: no network interface access. So, I restored the backup configuration and restarted the service. At that point, my terminal died and I was then unable to establish any SSH connections to the server. The web server was down. Confounded, I cursed Debian for being so ridiculous and hoped that the configuration had taken effect. Within the hour, we had our TorrentFries rebooted, and to my relief the original configuration was working.

Had the networking stopped working when I tried to load the invalid configuration instead of when I loaded the original working configuration, we would have been toast. Not only would we have been forced to do a complete reinstall, but we would have lost quite a bit of data in the process due to not moving the rather infrequent non-automated backups off site regularly. *cough*

We barely escaped a disaster that would have spelled the second time in my capacity as an admin to need an OS reinstall on the remote server. The only time we've had to reinstall was when a certain curly type of administrator dropped the MySQL privileged users and my consequent mucking with things resulted in an inoperable system. The lesson to take away is threefold:

1. Backup offsite before making any changes that could effect the operation of the site.
2. Be mindful of what senarios could leave you with no server access. These generally include but are not limited to network configuration, SSH configuration, firewall configuration, and boot loader configurations.
3. Don't close an open terminal session unless you're sure you can get back in. After I make some changes, I open a second SSH session to ensure access before exiting out of my working session.

I hope you found that at least mildly entertaining enough for you to remember when it's your server you'll be losing access to. There's nothing quite as disappointing as the realization of failures easily avoided.

Back to the Basics

I realize that our focus has of late moved from tracker-specific discussion to more-general stuff that applies to any site or server and can be found in a bazillion places online. Since this blog is unique because it does chronicle the operation of a BitTorrent tracker, I'm going to get back to the meat of the thing for a bit. OnionRings plans to continue with his Linux series, but is sadly indisposed this evening.

In some senses I will cover some of the same material as our second-ever blog entry, but with a different focus. While that entry gave a quick overview of why and how to set up a tracker, I'm going to look a little more closely at the technical and logistical aspects of bringing a tracker online.

First off, what do you personally need to bring to the table? (You're probably the only one at the metaphorical table at this point, but the bringing should commence now.) Beyond anything else, you need to understand the BitTorrent protocol, and understand your tracker of choice. We'll talk a bit about choosing trackers shortly. If you plan on running a PHP/MySQL tracker, you'd better have at least a working knowledge of PHP and MySQL. From the get-go, you'll need to be able to peer into the inner workings of your site and see what makes it tick. You don't just need an understanding of the protocol, but of your own tracker. Trust me, admin panels are nice, but some things just require you to get down and dirty. However complete your chosen script may look at first glance, there will quickly come a time when you or your users bemoan the lack or poor design of some feature or another, and you will be forced to go in and remedy the situation. As the tracker grows and moves to its first dedicated server, it will also become important to know your way around Linux to some extent.

Secondly, you need a hook to pull people in. The most obvious aspect is to provide something that others can't. A unique idea is neat, but it's entirely possible to carve out a niche in a "market" that is well-developed but not saturated. For example, if you have terabytes of obscure movies that you have spent years collecting, starting a movie tracker to share these can be an excellent starting point. Of course, you will need to keep all the uploaded torrents active, even though you will be getting very few peers at first. For this purpose, you may want to consider renting a seedbox. The added cost may be hard for a brand new tracker admin to swallow, but your ability to saturate the connections of your first members will do a lot to encourage them to stick around and maybe even contribute some stuff of their own. As well, a unique design may sound frivolous, but it will lend your site credibility and a sense of longevity. I've covered some methods of early promotion in the post I mentioned above, so I won't rehash them here.

I hate to be defeatist, but if you can't meet these requirements, you should think twice about starting a tracker. Of course, the lovely thing about being a human is the capacity to learn, but you should get going on that learning well before you even consider getting into tracker territory. While I'm trying to demystify the role of the tracker administrator, it's certainly a job that not everyone has the skill and disposition to fill with any great success.

Moving on to more technical requirements, you will also need to choose a tracker. That's a given. The three most popular trackers under active development are TBdev, TorrentTrader, and the new Project Gazelle, although there are a ton of other options out there, so you shouldn't feel constrained to choosing one of these three. Shop around a bit. Detailed comparison of these and other trackers is well outside the scope of this entry, and perhaps even of this blog, but this should point you in the right direction. At the end of the day, it's up to you to select the tracker that best suits your particular tastes and needs. As a side note: for the love of God, don't use TorrentTrader Classic.

So, why not just boot Azureus or µTorrent or one of these handy ubiquitous little torrent clients that happen to include the ability to operate as standalone trackers? Hopefully this isn't a question you were asking yourself, but I'll answer it anyway. First, this isn't the purpose they were designed for. Just because they can run a tracker doesn't mean they should. They're not optimized for the purpose, they don't include features that are necessary for the smooth operation of a tracker (they're inherently open to anyone that wants to use them), and they include a ton of features and bloat that will just bog down your server. What's more, part of the sort of tracker we're discussing here is the index, or frontend. This is the bit where torrents are uploaded, downloaded, where users interact with one another, all that sort of stuff. The tracker itself is ridiculously simple in operation, as we will see in my eventual piece-by-piece breakdown of what it takes to build one from scratch. The important business is the complex frontend, and BitTorrent clients' wannabe trackers just can't provide this.

So, once you've selected the script that best fits your needs, it's time to choose a host. I've had some requests to list a number of torrent-friendly hosts, but that's not something I'm going to do for a number of reasons. The most important reason here is that I don't want to endorse a host that turns out to be unsafe, or to lull readers into a false sense of security. You may have noticed that even (especially) the biggest trackers tend to move around a lot. The world of web hosts is intrinsically volatile, so to proclaim that one host is perfectly safe, even if presently true, is to ignore the fact that this may not continue to be the case in the future. LeaseWeb was once a safe haven for trackers, but after losing a lengthy court battle on behalf of Demonoid and other trackers, all the will to fight has gone out of them. Nowadays, if a copyright holder says "boo", they'll turn around and shut you down.

So simply accept that no place is perfectly safe, and operate under the assumption that you could be taken down tomorrow – you could be. A good strategy to find places that are safer is to run whois lookups on established trackers' IP addresses. Presumably, if they are able to operate on a particular host, that host must be somewhat resistant to legal threats. The only problem is that many larger trackers own their own servers which they operate via colocation in the host's datacenter, while you are likely unable to afford more than a rented dedicated server (an important distinction to make). Not all hosts supply both options, and not all do so for reasonable rates. In addition to colocated and dedicated servers, I've talked about shared and virtual hosting in that previous article, so I'm not going to rehash that here.

This should be all you need to get you off the ground. Again, I'm not going into exacting detail on some of the technical points because I'm assuming that you will be able to handle the fiddly bits of your own accord. If you can't, like I mentioned, you probably shouldn't be running a tracker.

If there are any points on the subject (or any other, really) that you would like us to cover, leave a comment on this post and we'll do our best to touch on them in future posts.

Site Administration Tools

Monday I wrote a technical piece about installing LAMP so today I'll talk a little about the type of administrative tools we use on the site. I've always been a big fan of using the tools UNIX gave us. That means strict command line pleasure. It means things are clean on the system and no extra resources are used or security holes opened. But not everybody agrees. Even I find it nice to use a GUI once and awhile so as not to get bogged down with the mechanics of the command line.

MySQL seems to be the toughest to administer with the tools UNIX gurus gave us. Writing SQL statements isn't bad if you're familiar with SQL, but you'll likely feel better with PHPMyAdmin or MySQL Administrator (MySQL Query Browser). The difference is that PHPMyAdmin is web-based and MySQL Administrator is a remote client. To me, the difference is that there's no garbage on the server when you use a remote client to connect to to your server like there is with installing PMA.

Consider the security implications as well. When installing PMA, you open up an avenue for attack via the web. When using MySQL Admin allowing remote connections, you're doing even worse. Therefore, the safest and most efficient option is to tunnel your MySQL Admin connection over SSH to get a localhost connection using a remote tool. I think this method applies for other remote tools as well. Ketchup likes using MySQL-Front. Using a remote tool allows room for preference.

In other areas, it can be handy to have a GUI as well. If you have many servers running on your box (such as mail, Jabber, and Apache/MySQL) it can be handy to increase your proficiency and ability to edit obscure configuration files with greater ease by installing a management tool. I've used Webmin before, and I have mixed feelings. It gives very good control over the multitude of configurations on a Linux box, but I always see it as another security vulnerability.

That's all for now, unfortunately. Next time I post, we'll discuss more about LAMP configurations.