Site Administration Tools

Monday I wrote a technical piece about installing LAMP so today I'll talk a little about the type of administrative tools we use on the site. I've always been a big fan of using the tools UNIX gave us. That means strict command line pleasure. It means things are clean on the system and no extra resources are used or security holes opened. But not everybody agrees. Even I find it nice to use a GUI once and awhile so as not to get bogged down with the mechanics of the command line.

MySQL seems to be the toughest to administer with the tools UNIX gurus gave us. Writing SQL statements isn't bad if you're familiar with SQL, but you'll likely feel better with PHPMyAdmin or MySQL Administrator (MySQL Query Browser). The difference is that PHPMyAdmin is web-based and MySQL Administrator is a remote client. To me, the difference is that there's no garbage on the server when you use a remote client to connect to to your server like there is with installing PMA.

Consider the security implications as well. When installing PMA, you open up an avenue for attack via the web. When using MySQL Admin allowing remote connections, you're doing even worse. Therefore, the safest and most efficient option is to tunnel your MySQL Admin connection over SSH to get a localhost connection using a remote tool. I think this method applies for other remote tools as well. Ketchup likes using MySQL-Front. Using a remote tool allows room for preference.

In other areas, it can be handy to have a GUI as well. If you have many servers running on your box (such as mail, Jabber, and Apache/MySQL) it can be handy to increase your proficiency and ability to edit obscure configuration files with greater ease by installing a management tool. I've used Webmin before, and I have mixed feelings. It gives very good control over the multitude of configurations on a Linux box, but I always see it as another security vulnerability.

That's all for now, unfortunately. Next time I post, we'll discuss more about LAMP configurations.

Linux Part 2: Installing LAMP

I'm going to resume from Linux Part 1: Installing Configuring Sudoers and OpenSSH with part 2: Installing LAMP. In the world of Ubuntu, virtually no one installs LAMP from a command line on Ubuntu Server. Ubuntu provides the option to automatically install LAMP during OS setup, so most of the time, it's done that way. Purchasing a dedicated server often doesn't give you that option, however. Yes, many tutorials cover installing LAMP, and they are just as good, but note that they're installing on Ubuntu Desktop, sometimes assuming you have a GUI. Thankfully, installing LAMP on Ubuntu from the command-line isn't hard at all.

Let's start by running the best command ever, apt-get. This will get all the packages we need.

sudo apt-get install apache2 php5 php5-mysql libapache2-mod-php5 mysql-server libapache2-mod-auth-mysql

Alternatively, you can try

sudo tasksel install lamp-server

Apt will download the packages and install. Then, you'll see this as MySQL is being configured:



You'll have to confirm your password, and then you're done. Restart Apache.

sudo /etc/init.d/apache2 restart

Let's make sure everything went okay. Browse to your IP address. You should see an "It Works!" page. This means Apache is running. Let's check PHP. Run

sudo nano /var/www/test.php

Write out this single line to the file:

<?php phpinfo(); ?>

Browse to your addresss/test.php and you should see lots of information about the current state of PHP.

Securing MySQL

That installation was so easy, we'll do a little security cleanup too.

Log into MySQL:

mysql -uroot -p

Your prompt will change to mysql>

Create a user you'll be using to administrate the databases. By default, MySQL is secure in the fact that it will only allow root to login locally. That's excellent until you realize that MySQL can be a pain to administer locally. I don't mind too much, but some people like to install PHPMyAdmin. This can open you up to potential remote access attacks on the root user's password. Therefore, if you're going to install PMA, we'll get rid of the root user to make it more difficult to enumerate users. Let's start by creating a MySQL account, Zeus.

mysql> GRANT ALL PRIVILEGES ON *.* TO 'zeus'@'localhost'
-> IDENTIFIED BY 'yourpassword' WITH GRANT OPTION;

You should consider also adding the privileges to this user @ 127.0.0.1
mysql> GRANT ALL PRIVILEGES ON *.* TO 'zeus'@'127.0.0.1'
-> IDENTIFIED BY 'yourpassword' WITH GRANT OPTION;

At this point, you should create a user that will be the one accessing the database. Using your "God" user to access MySQL from your web application is bad karma and invites security risks. Google 'create mysql user' for the syntax on creating a user with less privileges.

Feel free to be creative with your names...
Flush the privileges and exit.

mysql> FLUSH PRIVILEGES;
mysql> quit

DO NOT ATTEMPT TO FORGET THIS PASSWORD! It's a pain to restore a MySQL root password, and even more of a pain when the root user doesn't exist, (some say impossible) so take it from me-- remember your privileged user's password for cripes' sake.

Login and delete the root and anonymous user once zeus was created.

mysql -uzeus -p

mysql> DELETE FROM mysql.user WHERE User = ' ';
mysql> DELETE FROM mysql.user WHERE User = 'root';
mysql> FLUSH PRIVILEGES;

Check out your work:

mysql> SELECT User, Host FROM mysql.user;

Don't drop the debian-sys-maint account.

Apache and MySQL Control

Here's how you can restart Apache and MySQL without rebooting your server if you have changed a configuration file.

sudo /etc/init.d/apache2 restart
sudo /etc/init.d/mysql restart

Replace 'restart' with 'stop' or 'start' if needed.

Thanks for reading again, next we'll show you a bit about the different administrative tools available for Linux.

Terminal Case

I was going to continue OnionRings' series on Linux configuration, but I don't have access to a clean Ubuntu install at the moment (or any other kind, actually), so I'll leave that topic for now. Instead, I'll rant a little about the future of the MAFIAA, and the RIAA in particular.

The popular opinion among pirates is that downloading music hurts no one. I've heard a variety of justifications for this, but the most popular opinion is that the free publicity (which presumably leads to album sales) counteracts lost sales. I'm no expert, but you're sure as fucking hell less of an expert than I am. What I will say is that, if this is true, it's true only for small bands and independent labels. The big boys can afford to buy as much publicity as they want. So they're right, it is hurting their sales. It's not hurting their sales by the number of copies downloaded times the full sticker price of the album, but it's hurting them nonetheless.

Don't mistake this for a bad thing. The big labels would have you believe that without them, there would be no music. Listen, bud. There was music before you came along, and there will be music long after you are laid to rest. What we will see the death of is McMusicians, singers and bands that everyone has heard of and that the most gullible of us consider to be the epitome of good music. You know who I'm talking about.

What will be left without the big labels? Musicians that aren't in the business of making millions of dollars. Record labels that are employed by the musicians, not the other way around. A world where the music industry is easier to break into because people are more open-minded about music they haven't heard on the radio at least 15 times. A world without the fat, corrupt husk that pretends to serve the needs of musicians and music-lovers alike.

The best bit of all: you can live in that world today. Listen to internet radio and find new bands. Go anywhere with live music, be it a concert, bar, club, coffee shop, or street corner. Buy independent records from independent record stores. Learn to play an instrument yourself! The labels aren't afraid of downloaders. They're afraid we'll realize that we don't need them. Downloading music won't hurt them as much as the realization that you never need to buy anything from them, ever again.

Naturally, all of the sentiments above can be directly applied to the film industry as well. Notice how movies are all shit these days? Try independent cinema. Trust me, there are still movies being made that don't suck.

Okay, that's all for now. I'll see you on Monday, or better yet, OnionRings will. I'll see if I can make him do an article next time.

A brief update: apologies to anyone that was expecting this to be a pun and for me to cover command-line stuff. I think OnionRings is indeed planning to get his series going again for Monday, and I'll try to get a little further ahead on my posting as well.

Linux Part 1: Installing Configuring Sudoers and OpenSSH

Okay, I'm going to start you off with a little advice for beginning Linux users looking to get rolling with a torrent site such as ours. Note that startup sites will most likely look for shared hosting whereas the following advice is aimed at admin's moving from a shared host to a virtual private server (VPS) or a dedicated server (dedi).

In these posts, I'll be customizing the advice for a PHP/MySQL tracker such as ours at TorrentFries. I'll also be running under the assumption that you're running Ubuntu Server Linux, specifically Hardy Heron 8.04. Additionally, these are pretty common instructions you can find at a variety of blogs across the net.

Let's just get some things straight before we get technical. If you're new-ish to Linux, I'd recommend Ubuntu. If you disagree, you don't need to be reading this anyway. Ubuntu isn't the most rock-solid distro available, it's just the easiest to administer (in my realm of experience) and it's stable enough that I've never crashed it on accident. It can also be found with most hosting companies. If not, Debian is the closest to Ubuntu and feels nearly the same.

If you're concerned, check out these survey-based distribution choosers:

• http://www.zegeniestudios.net/ldc/
  
• http://polishlinux.org/choose/quiz/
  

Logging In

You'll find that your hosting company will give you nothing more than an IP address (or a couple if that's the case) and a root login password. This means that you'll be accessing a server via Command Line Interface (CLI) over Secure SHell (SSH).

On Windows, I recommend using PuTTY for terminal access.
On Macs, you have Terminal!

Starting Configuration

Once you're logged in, create an admin group.

groupadd admin

If the admin group already exists, that's fine. Now create the first user with -g being the primary group (often just the username), -G being secondary group (admins), -s is the location to the shell, bash, -p is the password (you can omit this flag if you want), -d being the home directory, and -m creates the directory if it doesn't exist, and finally the username itself.

useradd -Gadmin -s/bin/bash -pPassword -d/home/UserName -m UserName

The password you just set can be tricky, so to be sure...

passwd UserName

You can add any other users in this fashion.
Let's give your new user sudo access first. Sudo lets you run commands as root while not logged in as root. Once, you'll have to use the UNIX text editor 'vim', but in the future, you can use a much easier text editor called 'nano'.

Edit: Ubuntu Intrepid Ibex (8.10) now uses nano for visudo.

visudo

Here's a quick tutorial on vi(m). Only use visudo when editing the sudoers file. Using any other method may cause corruption in the file, so make it a habit to use visudo.
Make sure your sudoers file looks like this in the group section:

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

Now that you have sudo access, try connecting with a new session. You should now see something like

UserName@host$

instead of

root@host#

Let's use our new user from now on.

Configuring OpenSSH

Now, lets lock down SSH. First, copy the config file so we can restore the original settings at any time.

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original

Now, let's edit the sshd_config file with nano text editor.

sudo nano /etc/ssh/sshd_config

Look for:
LogLevel INFO

Change to:
LogLevel VERBOSE

This will allow us to get more information about SSH activity. It's handy to see who's logged in and who's tried to log in. Run grep -ir ssh /var/log/* | less to get that good data you're collecting.

Look for:
PermitRootLogin yes

Change to:
PermitRootLogin no

This will prevent hackers from brute forcing the root password, since root login is disabled. They'll still try to enumerate your users, but this is a critical step.

Look for:
LoginGraceTime 120

Change to:
LoginGraceTime 20

Changing the login time will slow down any would-be automated attacks.

If you'd like, you can change the port you connect to on SSH if you feel better being more obscure. Just change the 22 to what you wish:

Port 22

Save and exit the file and restart OpenSSH:

sudo /etc/init.d/ssh restart

You won't notice any changes until you reconnect. When you do, remember that you won't be able to log in as root. If you need root, you will have to use sudo or su. Also, if you changed your port, remember to change it in PuTTY when you reconnect.

For more info on OpenSSH, check this out.

Up next: installing and configuring LAMP.

Guest Post: To Track Ratios, Or Not To Track Ratios

I'm a little short on time tonight, so I'm going to share a brief essay, courtesy of a friend of mine. He's quite knowledgeable about the BitTorrent protocol, and fairly critical of the status quo.

Talk about things that make you go hmmm...when TorrentFries was a private tracker, only allowing members to upload and download, they collectively reported uploading 100x more than they reported downloading.

The fact is it's impossible to track ratios accurately--the BitTorrent protocol wasn't designed with statistics in mind and doesn't support them reliably. Ratio updates are not infrequently lost and it's easy to fake ratio, undetectably, using any client.

Given its unreliability it is counter-productive to use ratio as a basis for rewards/punishments. You encourage cheating and end up rewarding the dishonest and punishing (by way of not rewarding) the virtuous.

You can try to identify cheats, but someone with a sky high ratio may have a seedbox and an inclination to support your community. At least until you mistakenly accuse them. And, of course, the smart cheats don't do so outrageously--so they never stand out in any way.

If you don't track ratios? The Pirate Bay continues to grow in spite of not doing so. Certainly their seed:peer ratio is lower than that of any ratio tracker, but their torrents survive. Download speed isn't everything--if people can wait weeks or months for a DVDRip to come out they can survive a day or two of downloading.

In spite of all that, I recommend you track ratios. They remind people that sharing is what torrenting is all about, and they are mostly accurate. Just don't make a big deal of them and, when people do complain, admit their failings and point out that they shouldn't be taken too seriously. Don't accede to request to "fix" people's ratios though--you open yourself up to a lot of work with no real benefit.

While I'm on the subject, I'd like to debunk a common myth--that it's impossible for everyone to have a ratio > 1. The theory goes that since everything which is uploaded has to be downloaded, the average ratio must equal 1. So, for everyone who has a ratio > 1 there must be someone who has a ratio < 1. So, people with low ratios are victims of bastards who seed too much, who should be pitied rather than punished.

Nonsense. The theory only works if you assume that nobody ever leaves--totally unrealistic, particularly if your policy is to ban people with ratios < 1. Even if nobody ever left, so long as momentum is maintained (ie. so long as people upload within a reasonable period of downloading, thus restoring their ratios) it would continue to fly.

Green Salad
Not an administrator. Not always wrong.

One thing I'd take exception to is his claim that the global ratio does not always have to be 1.0. It does when you factor in users that have left or been banned. If something is uploaded on a private tracker, another user must be downloading it. In my mind, it's also a little incongruous with his arguments against rewards or penalties based on ratio, since his argument relies on users being banned for having poor ratios. Since the global ratio must inevitably be 1.0, someone will inevitably end up banned. Allowing people to register with the expectation that some will receive summary bans seems pretty ridiculous.

That's all for now, folks.

Staff Privacy

Running a tracker can gather quite a bit of heat under your persona, especially if you live in a country that dislikes sharing the type of material on your site. It's good to insulate yourself as an admin from these type of things as much as possible, so be diligent and conscientious. Think ahead for the long term when exposing yourself. There was a good writeup on this sort of thing at TorrentFreak, so I'm going to expand from there.

Encryption is king, but nothing quite takes the crown like OTR messaging. For chatting between staff, running OTR offers encryption, authentication, deniability, and perfect forward secrecy. As a supplement and/or alternative, it's very convenient to run your own private XMPP chat server such as OpenFire or Jabberd. XMPP/Jabber allows greater control over configuration and supports SSL encryption.

Paranoia is a positive feeling as a admin, and nothing satiates that feeling quite like running your network connections over Tor, the encrypted onion routing software. This works well when you want to obscure your location or ISP. Practically, it's a difficult thing to put up with considering you'll likely see a significant drop in network speeds over Tor. Nevertheless, it's possible to run two browsers or their mobile package in order to not disturb your normal surfing. Using a VPN solution such as OpenVPN is another handy option to encrypt traffic or create a tunnel.

As a brief aside, never torrent over Tor. Despite the tempting alliteration in the names, they do not go well together. Tor is, by and large, extremely slow, and running BitTorrent through it is a gross abuse of the bandwidth donated by willing volunteers that believe in the free flow of information. Tor is slow for all of us, and if you're torrenting on it, that speed is your fault.

Anyway, if you're like me and don't trust a free email host with your information, set up an email server on your box with webmail. The most easily configured mail setup I've come across is Postfix with RoundCube Webmail, but be aware that the learning curve for the inexperienced is daunting. I still suffer headaches and driving pains to my geeky ego. :P At any rate, this allows you to send and receive site-related email without worrying about your origin or alias. Note that it wouldn't be wise to send email via your server's SMTP service as this could expose your IP in the mail headers.

If you've been good about obfuscating yourself, you won't need to delete server logs that might point to an IP you used, but it's reassuring to not log those things just in case. Scrutinize both Apache and SSH server logs as well as any other services you connect to.

I hope I've provided you with some options to protect your privacy. The best summary would be to quote the TorrentFreak article, "Identity is everything , If you never tell anyone, no-one will ever know."

Choosing a Server

BitTorrent trackers are strange animals. Servers are generally pretty well-rounded in terms of system resources (hard drive is proportionate to memory/processor/bandwidth), but trackers don't use each resource equally.

Trackers use very little bandwidth and disk space, but are comparatively heavy on processor and memory. The announce/response and scrape/response cycles contain very little data and are only infrequently initiated with any given peer. By bandwidth alone, it would be theoretically possible to track tens of thousands of simultaneous peers on a residential internet line. Obviously, a consumer ISP would be unable up to handle that volume of connections, but that's a pretty compelling illustration of the bandwidth situation. Similarly, since even a complete BitTorrent site (index + tracker) only needs to serve small metainfo (.torrent) files, the site and database combined for a reasonably-sized tracker should be no more than a few gigabytes. However, the sheer amount of traffic involved in running a tracker makes for a fair bit of work for the memory and processor.

To estimate the traffic on any tracker, remember that each peer will connect to the announce roughly once every 30 minutes, or however long your tracker's announce interval is. The protocol specifies that clients should connect more often when sending event=started/completed/stopped, so you'll have to round up a bit to compensate. Basically, just divide the number/estimated number of peers by the announce interval in seconds (30 minutes * 60 = 1800 seconds). This will give you a rough idea of the number of hits per second on your announce. On TorrentFries, the announce accounts for roughly 50% of the site traffic, 30% for scrape, and 20% for all other activity. Simply double the result to get a rough idea of the total number of hits on the web server.

hits per second = (peers / announce interval) * 2

This doesn't tell you a whole lot in terms of servers, though. However, since site growth is generally pretty gradual, there's no need to worry about what you'll eventually be running on. Just start basic and upgrade as the need strikes.

Our first server had an AMD Athlon 64 3500+ processor with 512 MiB of memory, a 1500 GiB bandwidth cap on a 100 Mbps line, and a 160 GB SATA drive. It was pretty rudimentary, but worked fine to about 8,000 peers (10 hits/sec). Right now we're tracking 25,000 peers on a dual Xeon machine with 4 GiB of memory and two hard drives in a RAID 1 array, and we're looking at about 10% utilization. It's got a bit of room to grow.

If you're curious to see what makes bigger sites tick, take a look at The Pirate Bay's setup.

Tough Calls and Conflicts of Interest

Okay, I figure I may as well finish the week off with a subject that's something of a continuation from yesterday's discussion of black-and-white dealings with members. As "nowhere" said on Monday, this is the sort of thing all staff should already know, although in my experience this is not always the case. Anyway, sometimes things aren't so clear, even for the more levelheaded staffers out there.

If an argument occurs between two established members, it can be a difficult call to make. The best choice with heated discussions is to lock or delete the thread in question. If the thread is locked, include a simple post saying something like "this discussion is getting out of hand". Don't take sides. Usually, this is all it will take. It's the topic's fault, not the members'. If necessary, send each involved party a cautionary message. Don't get into a "who started it" game, or allow the members to pull you into one. Depending on the circumstances, you could caution one member against baiting others and another against rising to the bait. Warnings are rarely necessary where otherwise-upstanding members are concerned, since they have a stake in the community and should understand that they should behave accordingly. However, repeat offenders should be officially sanctioned no matter their status.

That's still something of a no-brainer. Here's the bigger point, and the one where I slipped up as I'll tell you in a minute: don't get involved. Even if the subject of discussion is concerning the site, do your best to remain impartial. If it's a topic of some concern to you, step back and ask another staff member to deal with it. Similarly, don't use the locking method I described above to foreclose debate. Unless a subject is clearly inciting a polarized and heated debate (as in the example I gave on Monday of the "What do you think about homosexuals?" thread on Facebook), it should be allowed to run its course unless it gets out of hand. Just because you may not like the way the discussion is going doesn't give you any right to bring your extra powers to bear.

Sometimes, you just want to leap into the fray. Hey, a good argument can be fun, so long as it remains civil. However, if you're going to join up, leave your badge at the door, and make it clear to the members that you are doing so. If necessary, message a fellow staff member asking them to keep an eye on the thread and moderate any or all participants if necessary, even you. If you bring your powers into a discussion, your relationship with members will change forever. It can't be undone.

This is the point where I bring in my own painful example, where I became involved in a discussion and a user ended up leaving the site for good as a result of my behavior. During a political debate, I brought up my rather polarized views against Israel's foreign policy. The other user responded with equally-polarized view in support, and we fell into discussing the country's enemies. Eventually, I felt that the other had become quite offensive, both to the Islamic religion and Arabs themselves. I issued a summary ban. I realized almost immediately that this was an overreaction, so I scaled it back to a warning. Here's the worst part: I then asked one of my fellow moderators to send a message explaining the reasons behind the warning that I had issued. I didn't want it to look like I had become emotionally involved in the discussion, and didn't believe that I had. In retrospect, that was certainly the case. The other member reacted in an understandable fashion towards myself and my middleman: he sent us both indignant messages, deleted all his torrents, then his own account. I had, in the end, effectively banned him from the site.

The biggest lesson to take away from this example is that there's a right and a wrong way to involve other staff members. Asking another to do the work I didn't feel comfortable doing myself should have sent up red flags that what I was doing was not a good idea. Part of the problem is that I don't think he had quite internalized The Moderator's Bible that I posted on Monday, and didn't feel comfortable enough to say no to me. That's another reason that mods need to be relaxed and comfortable in their roles, since they may need to call you out when you behave improperly yourself. Hey, we're all human. Well, we are under a layer of internet, anyway.

Troll Control

Anyone who has spent any length of time in an online community knows about trolls: insecure teenagers looking for recognition by eliciting a reaction from community members, be it positive ("LMFAO") or negative ("fuck off"). The easiest way to draw a reaction is to incite the contempt of others, so that's exactly what they aim to do. Having this sort of person around is naturally detrimental to the tone and integrity of the community. But what can you do to get rid of them?

The obvious response is to ban them outright. Well, this one has a few problems. First, as negative reactions go, summary bans are pretty close to the top of the list and will incite the troll to offend again, both on your site and on others. Proxies do exist, after all, and trolls know this on account of having been banned from other sites before. As well, this sort of reaction is entirely arbitrary and may make other community members nervous to say things to which you may take offense. In short, they become afraid of you. Some administrators thrive on this, but really, how does that make you any different from the trolls? Just because you hold the banstick doesn't (shouldn't) mean that your own behavior in the community is beyond reproach.

Of course, the best response to a troll is to cut off their food supply. To post responses, as well as conduct overt moderation, will feed their insecurity. Instead, caution your community not to respond. If necessary, delete a few posts - other posts - so it appears that no response has been made. Again, it is a bad idea to moderate posts that are annoying but not rule-breaking. However, you may be able to find more creative alternatives. I once edited the shoutbox code on TorrentFries so that the shouts of a chronic troll would appear only to himself and me. He left of his own accord soon after. (I'm pretty sure the troll was male. To the best of my knowledge, I've never encountered a female troll.)

On larger sites, this sort of time-intensive personal response is not practical. Instead, liberally delete affected posts, or entire threads where they have become infested with the troll/response cycle. However, do not hand out warnings or even directly acknowledge the deletion and underlying reasons until rules are directly broken. "No trolling" isn't a good rule, since it is very arbitrary in definition and to acknowledge this status makes the troll feel successful as this is their goal.

However, true rule-breaking will likely happen within a few days, and you should respond swiftly at this point. By waiting until rules are broken before responding, you make it clear to the troll that they will not be dealt with in any special manner. Since they want to feel special, this will leave them unfulfilled and allow you to respond without making the situation worse.

It is possible that the troll will attempt to contact you or another staff member directly. Respond to their messages politely, formally, and cooly. Answer any questions directly and as succinctly as possible, or provide a concrete reason why responding to a particular question is not an option (privileged information and so forth). Do not make any other comments, particularly regarding the character or behavior of the troll.

Never ban a troll without sufficient cause, although you can certainly make the rules a little less tolerant where the undesirables are concerned. You cannot allow them to feel like a part of the community or they will get the idea that the staff and the staff alone hate them, leading them to return via proxy "for the sake of the community". To the contrary, if you do your job right, they will leave of their own accord after a few days and a ban will not be necessary. I have never had a troll that lasted more than 4 days.

If you stray from this path, particularly by arbitrarily banning the troll or rising to their bait, your site may end up with a poltergeist to call its own, and they will become extremely difficult to shift. One way or another, best of luck in dealing with these little shits.

Write Your Own Disclaimer (in your spare time)

By popular demand, I give you: do-it-yourself disclaimers.

Okay, first I'd like to offer a disclaimer about this post. I'm not a lawyer, and I haven't consulted with one in writing mine. I'm planning to have a lawyer member look over/rewrite our own disclaimer. Well, a member that is ostensibly a lawyer—see my previous comments on judging online character.

Anyway, there are a number of important points to cover. I know Demonoid has 34 pages of text in their disclaimer, which apparently contains some amusing points for those with the attention span to read through it. I guess they figure the length is a deterrent to people actually reading it, but lawyers are renown for their long attention spans. I think it's more important to make your point in a concise manner that your friends can understand as well as your enemies.

Basically, you need to address the following points:

No "uploaded"/copyrighted material is stored on this server. It's not. Only metainfo files (.torrent) are stored on the server, and they only contain SHA1 hashes which describe the material. All material is located on individual users' computers, so you're perfectly safe in making this argument.

Given that part of the disclaimer, for the love of god don't use your server as a seedbox. That'll just increase your culpability in this sort of thing and will make the bit about copyrighted material a total lie. What's more, running it as a seedbox can have all manner of detrimental effects on your server's performance, not just in terms of bandwidth but also drive speed. I know you probably have a 10 or 100 Mbps line with very low utilization, but resist the temptation.

The site cannot be held accountable for the behavior of its users. It is merely offers an indexing service. It's true. You can make the analogy to Google potentially indexing child pornography sites, but that's a little extreme. The point is that you merely provide a service.

It is the responsibility of the users to ensure that they hold the copyright or attain proper permission prior to uploading material. You may want to hide this statement a little, since some members will inevitably take it to heart. I've definitely had some concerned posts to that effect, to which I've given some rather tongue-in-cheek responses. Obviously, the idea is to turn one big fish into a whole lot of little fishies, reducing your own culpability without greatly affecting the masses.

Finally, you will comply with any and all formal copyright notices sent from the proper jurisdiction (where your host is located, since they shouldn't know where you are personally located). Obviously, you're not hosted in the US... are you? The idea is to make a hell of a lot of work for anyone that would attack you, encouraging them to go after other sites. You also need to provide a clear email address for copyright complaints. Better that they go whining to you than to your web host, which will always have contact information prominently displayed and will (with some exceptions) likely take you offline if a copyright owner says "boo".

It's doubtful that this will do a whole lot of good, but you never know. Once I get some better legal advice, I'll make another post correcting any glaring mistakes...

Site Contingency and Redundancy

Today's post will cover contingency and redundancy, of which we've learned the hard way over time running TorrentFries. Backups are crucial to a holistic approach on contingency and redundancy, but there’s more. By the way, I’m OnionRings, the “fellow admin and general Linux and security geek.”

If you’re just beginning a site, hardware redundancy isn’t possible since you will likely have a shared or dedicated server host, but there are a few other areas of redundancy to think about. Attack issues of redundancy and weaknesses before they occur and in the order of likelihood. In the case of TorrentFries, the most likely thing to happen is for a host to get a copyright infringement notice.

Selecting a host that’s least likely to take fright in a copyright notice is the best choice to reduce risk of downtime, but that can be difficult to know right away. We’ve often asked our hosts what their policy on copyright material was before moving to them. Usually they are happy to take your money up until they receive a notice.

In the past, a host has given us 12 hours notice to remove material before the server would be shut off. That might sound like a lot of time, but when they issue them at 5 am in your time zone, it could be a few hours before you even notice the warning. At that point, you need a new host quickly. Therefore, select a host or two that would be sufficient while you have the time to research it and before you get a takedown notice. It can take days for a hosting company to get you login credentials much less the time it takes to research your best option. Plan ahead.

TorrentFries was once put out of commission after receiving an infringement notice while our host and the domain registrar were the same company. In that case, we were locked from our account and couldn’t even redirect our domain to the new server we purchased. So if uptime means anything to you, register your domain with a different company than you host with.

Short of having an expensive hot spare server wasting CPU cycles somewhere, there are other ways to mitigate risks of takedown. Parceling out the server’s work to subdomains that reside on different hosts is one way to achieve better contingency. In this case, put the tracker announce and scrapes on a separate server than the main site so the tracker doesn’t go down if the main site is taken down. Leveraging this technique can be a handy method to keep staff communications up during downtime where forums, email, or Jabber are critical. If set up with some thought, your database latency won’t take a huge hit and purchasing a couple cheaper servers might be only slightly more expensive than buying one powerful server.

Finally, there’s quite a bit of rework involved in installing and configuring software each time you move to a new server if you’re on a VPS or dedicated server. Even though I’m a Linux geek, I have to look up some things that I don’t use everyday. It’s nice to have a list of the tutorials you’ll be using or a checklist of the things that need to be done starting from disabling root SSH login and ending with the final tweaks in Apache or MySQL configurations. If you’re savvy enough, write up a bash script to handle all those dull tasks.

Hopefully your site will learn from some of our mistakes and you’ll be able to keep your uptime positive.

The Moderator's Bible

I'm going to rewind a bit today and expound a little on a subject I touched on last Thursday, namely, the task of trying to get my moderators to loosen up and maybe understand their role a little better. This entry is perhaps something of a cop-out, but it's going to consist entirely of a message that is permanently stickied in the staff lounge. Hopefully it'll be of use to somebody.

Disclaimer: Take what I say at face value here - I'm being serious and saying everything straight up and no sarcasm involved. This is where I need that serious smiley.

IMO, this is a general thread that should be stickied across all community sites, like those "How to Upload a Torrent" threads often are. There's nothing specifically directed at any of you or all of you as a group, but everything that should be taken to heart for all mods everywhere - as the subject line indicates, a "Moderator's Bible".

The following is an example of what happens from time to time in a forum. You've all read the exchange and know who is involved, but I've genericized the names anyway since it really doesn't matter who is involved - we're all culpable at some point, staff or no. This is more to point out that in a community the term "moderator" doesn't mean "a person who wields shiny buttons". It means "a person who moderates". Think of a moderator in a debate. They have the ability to cut off somebody's microphone, but the job description is to keep the debate moderate and on track (well, in this site's case abandon all hope on that last point).

Here, things are admittedly a little different. In a debate setting, everyone in the debate is seriously devoted to the debate, even if they get sidetracked or passionate or whatever and need to be brought to order. On most sites, there are retards and spammers and all sorts of undesirables, so the heavy hand of the law needs to be used more often. But never forget that everything you do needs to stem from the basic need to keep the entire site as a whole on track and civil, to moderate the site as a whole, not just its parts.

So, without further ado, here's the excerpt:

Bystander:

Ah, another one of my flock, I'm studying engineering ;D

Combatant 1:

Which Engineering discipline? If its Civil then YOU are one of my flock, I like my coffee with milk and 2 sugars please ::)

Combatant 2:

Hey, hey, take it easy on the nice lady... You don't want us pressing any buttons, do you? :P

If it wasn't for that sarcastic smiley at the end, you would have made my day. ;)

The stage is set... nothing would have come of it since Combatants 1 & 2 are both mature, upstanding members of the community.

Concerned Person:

and YOU, Combatant 2, would appear to be correct that Combatant 1 was sheparding Bystander.

When a third party enters the game and takes sides, the sarcastic, joking air evaporates. Next, Combatant 2 attempts to explain his position, setting the stage for potential further conflicts:

Combatant 2:

Thus my comment about being nice to the lady. Sorry Combatant 1, but I won't have that, be it joking or not.

At this point, the mod steps in with a innocuous attempt to calm things down with a straightforward comment:

Moderator:

:-\ Is it just me or do I sense a sniff whiff of drama forming here?

Combatant 1 tries to explain his position. It's pretty effective, although this wouldn't always be the case.

Combatant 1:

I was implying that if she is a Civil Engineer, then I have a few years on her and if she came to work at my company she would be the general dogs body, degree or not she would still get stitched up with making the tea/coffee or doing the bacon butty run. :)

Moderator completely diffuses the situation with a straight-up remark, followed by a comment to hijack the thread in a whole new exciting direction. :D

Moderator:

Yeah, I figured as much. Usually just a matter of __ took __ the wrong way, so __ says __, and things just go downhill from there.

Now that we've got all that sexual tension out in the open ;) we can move on.

Sorry again for my choice of examples here, but this sort of thing is so rare around here that I kind of had to take what I could get. It's not a good example nor a neutral one, but it'll have to do. Around [popular site that I used to moderate], these things happen every day, partly because of the number of posts, but also because the community isn't as tight-knit and most people don't know each other that well. In fact, what happens over there is a thousand times worse, and with dozens of members taking four or five sides, some of them just saying that another side is idiotic for no reason at all. You can understand how quickly something like this could get out of hand if one user can't tell when another is being honest, sarcastic, ironic, mocking or even deceitful.

In this case it would've resolved itself without intervention, but I intervened (moderated) just to be sure. What made me think of writing this was a discussion thread I saw on Facebook asking for users' views on homosexuals. This is an obvious but somehow more insidious form of trolling, in my opinion. The thread will get hundreds of posts in an hour, but they will become increasingly heated, and in most communities the general opinion will result in the press of numbers attacking those of the minority opinion, and the minority posting nonstop making the same points over and over in an attempt to save face. Unless you are willing to watch every post, I would suggest you lock it immediately with a neutral comment like "this is an important discussion that should not be conducted here for the integrity of the community". This isn't the lazy way out; in fact, it's the safest way.

This isn't something that I'm telling you because I think you're performing badly as moderators. If anything, I want to make your more confident and aware of what your role is. You delete the porn spam, yes, and split the off-topic threads if you have a lot of time on your hands, but the rest of what you do has to stem from your own sense of keeping things level, in good faith and without any sort of bad feelings. The rules are guidelines both for members and for yourselves, but never feel you cannot overstep the limits of the rules or permit things not specifically allowed there as governed by your own judgement. There is no constitution here, and unlike in the real world, the intent of the law reigns supreme over the letter of the law. Above all, it is your role in keeping the site running smoothly and keeping its members in good harmony that you must adhere to.

So, if you've read this whole essay, kudos to you, and I hope it helps you out in some small manner. I, clearly, have a long attention span at the moment. And that's really all I have to say on the matter.

First person to reply to this thread with "I'm sorry" gets shot.

Backups

Okay, I can't stress this enough: an off-site backup is essential to the operation of any website, especially a tracker. Database backups are the most important, but backing up your files from time to time is necessary as well.

For example, if your thumb-fingered database maintenance *cough* results in a few thousand damaged or deleted rows (done that at least twice) or in the accidental deletion of your privileged user from the database (done that, too), you need a backup to fall back on. It's nice and convenient to have a local backup for that, but you also need to upload that backup to another server (or your home computer) in case the server runs into technical problems or you're suddenly taken down.

I was pretty lazy with backups while running on a shared web host, so I don't have a lot of wisdom to offer there. My host informed me that he ran nightly backups of all databases, so there was no need for me to maintain my own. (My PHP-based backup script had become pretty heavy for his server.) I think this is pretty normal behavior for shared hosts, although I don't know that all of them would give you access to those backups on demand. The point for them is protection from pissed-off customers in case they muck something up on their end or the server crashes or some such. So, when in doubt, back up.

The crux of the matter is that I'm lazy, and you probably are too. If it were left up to me, there'd be thorough backups for a few days, followed by a span of weeks or even months without backup. If you're lucky, that'll be all you need. How lucky do you feel, punk?

Therefore, you need to automate two things: the backup itself, and the process of uploading the backup to an external server. I'm not going to walk you through the setup process since that sort of info is pretty easy to find online, but I'll run over the general concept.

For the backup process, I set up cron jobs with two shell scripts: daily.sh and weekly.sh. When they're set to run is pretty self-evident. daily backs up the database (mysqldump -uroot -prootdatabasepassword --all-databases > backup.sql) and the uploaded metainfo (.torrent) files, while weekly backs up the entire web directory, all the PHP sources and so forth. weekly doesn't have to touch the database, since daily will also be running on that day. Each script then runs tar -czf backup.tar.gz files, with whatever files you intend to compress.

That's all it takes to ensure that you're safeguarded against data corruption. Naturally, there are other pitfalls that you can encounter. Hardware failure, hacking, inept administration, and of course the ever-present DMCA message—all can threaten your data and your backups. For that reason, it's a good idea to maintain off-site backups as well. You can download them manually, of course, but that's not going to be any easier to remember to do on a regular basis than to make the backups in the first place. Instead, I have my backup script upload to an external server via scp. I use that command instead of ftp because it's more secure and easier to automate.

However, scp does require that you have a UNIX box at the end. (Well, there are Windows ssh clients, so maybe there are scp servers as well. Stranger things have happened.) Since I run Mac OS X on my home computer, it's simple to automate the upload to there, or to any other server that I operate. If your shared host gives you rudimentary shell access, it's even okay to use that.

I'm sorry for being so lazy with the tutorial-writing in this one, but this blog entry is already getting rather lengthy and hopefully you can follow me anyway. I just have a few more points to make before I close.

Where one backup is good, archives of backups is better. If you set your script up to name its output files with the creation date, they won't overwrite one another and you'll have a nice collection in case you need to roll back a few days. I have plenty of hard drive space, so I still have backups from 2007. Who knows when you might need them?

Putting all this together, a simple shell script might look something like this (lines are double-spaced to avoid confusion in the case of line breaks):

#!/bin/sh

date=`date -I`

mysqldump --all-databases | gzip > /var/backup/backup-$date.sql.gz

scp /var/backup/backup-$date.sql.gz user@host:/dir/backup-$date.sql.gz

Okay, that's all for now, folks. I meant to throw in some horror stories about maintaining/not maintaining backups, but we're out of time. Maybe I'll get to that later on. Rest assured, such stories do exist, and don't forget: always maintain an off-site backup!

Recruiting Staff

Okay, I don't generally intend to follow the order of my subject list in yesterday's post, but since this is the first subject that came to mind, I'll hit it first and go from there. Maybe I'll end up following that order anyway, but not by any grand plan.

I was a moderator on a major site for a while, until I left due in no small part to a personality conflict with the administrator. What I took away from the experience (aside from a bunch of friends among fellow mods) was a shopping list of what not to do as an admin. Specifically, I decided to be open with my mods, democratizing the relationship as much as is practical. I'm not all-knowing, and I certainly don't feel comfortable with my word being taken as gospel. It took a while to convince them to relax, but I think it was a good move and it seems to have worked well.

As mentioned in the title, the point I want to make is specifically related to how to tell when members are ready to be promoted. Your mother probably told you something about strangers on the internet, and it's true, albeit with a lot less paranoia in the equation. You never do know who someone you're talking to online really is, but honestly, I don't care. If someone is reliable online, I don't care if they're a pedophile or serial killer in real life. I'm going to be dealing with their online persona exclusively, so the fact that I can rely on them online is all that's important from a staff perspective.

When I say recruiting your friends is a good idea, I want you to understand exactly what I'm talking about. These people need to be first and foremost the community's friends, but you're going to be working with them on a daily basis. The staff needs to be a strong team, and when there are conflicts, there's a problem. Besides, if you're doing your job right, you have nothing but friends in the community, and not the sort that come with being a figure of authority.

What I don't mean is that you should have your friends from real life or elsewhere on the internet join up and be promoted almost immediately. In this case, it doesn't matter how long you have known them or how much you trust them. The community in general has to trust them as much as you do, and they have to be committed to the community. That's not to say your personal trust isn't a great asset, but it can't be the primary factor. I did make that mistake once, allowing an online friend to join up and be promoted after some token site activity. As of this posting, he hasn't been seen on TorrentFries in over a year. He didn't do anything reprehensible, just disappeared one day. Just because someone is trustworthy doesn't mean they're committed to the site.

The other point I want to make is regarding timing. Moderating a site can be a big job, requires some serious team effort, and has its lulls and periods of frantic activity. I see no sense in hiring "enough" mods while leaving others on your waiting list in case you need them. People have lives, and it's nice for your mods to know they're free to take off whenever they need to without crippling the site in their absence. Hell, I could do that if necessary, since my mods had all the power needed to ensure that the site kept going on a day-to-day basis. These days, the only thing none of my fellow staff members have access to is the money, but that's a story for another time.

A side benefit of this duplication is that any of the little day-to-day issues that need dealing with such as trolling, spamming for porn sites, general rule violations and so forth can be dealt with very quickly. Even with "enough" staff members, not everyone is going to be online all the time. The more people that have the power to deal with this sort of thing, the more likely it'll be that someone will be able to deal with this sort of offense immediately.

Okay, one last thing. This shouldn't even need saying, but: if someone asks to be promoted, for fuck's sake don't do it. If it becomes apparent that they want or expect the position, a big black mark goes down beside their name. This sort of power-hungry person is going to be unreliable at best. The best staffers are the ones that will be baffled to sign on and find that they now have a ton of extra buttons on the site.

That's enough for today. Tune in tomorrow for... uh, something else. I'll work it out later.

Retrospective 7: Et Cetera

While we moved to a dedicated server well over a year ago, the time after the move was (until recently) characterized by a large number of gradual changes rather than the quick bursts of activity that I've discussed so far. The process of administration definitely became more complex, but in doing so, the issues we were dealing with became longer term in nature. The urgent stuff had been dealt with, but now the business of governing lay before us. It's very French Revolution, really, except without the guillotine and general in-fighting.

All of that is a very different ballgame. In fact, this gradual change is going to force me to abandon this "retrospective" structure altogether, favoring instead a more topical approach, dealing with each subject individually. I'm still going to be speaking historically for the most part, but story time is over. Now we're going to be talking about the real deal, what it takes to keep something like the site running on a day-to-day basis. Once that subject has been exhausted, we'll probably end up at something more closely resembling your traditional blog, but my fellow administrators will finally be introduced and will therefore be able to post, so you won't have to read my deranged ramblings exclusively from then on.

For now, a brief overview of what we're going to be talking about:

• Recruiting staff: My approach is pretty easygoing. Generally, if someone is clearly ready to be on the staff, they are promoted. There's no need to wait until everyone else is swamped before looking for help.
  
• Dealing with troublesome users: "Ignore them until they go away" does actually work. Sometimes.
• Conflict of interest: A case study in what not to do, hindsight lesson courtesy of yours truly.
  
• Backups: It's easy to neglect the importance of having up-to-date off-site backups. However, when disaster hits, you will either turn to your members and gloat at your awesomeness or shamefacedly admit that you just lost a month's worth of torrents and forum posts.
• The tracker announce: It's not as scary and impenetrable as it may look. I'll take you through the basic workings as I write one from scratch.

Hopefully that'll pique your interest enough to make you stick around for a bit longer. More to come, folks! I'll try to keep up this 5-day release schedule, at least until I run out of interesting things to talk about.

Retrospective 6: Early Linux Experiences

As I mentioned on Friday, I was completely hopeless with Linux when TorrentFries moved to its first dedicated server. Happily, I quickly discovered the most magical feature of Ubuntu: apt-get. With it, it's possible to download and install applications with a simple command, such as apt-get install htop. (By the way, htop is something well worth checking out. For those that don't know, top is a command that allows you to monitor the processes and load on a server. htop takes that a step further and allows more advanced process manipulation and general feedback.)

What I did know was what needed doing, specifically, setting up a LAMP (Linux/Apache/MySQL/PHP) server and somehow getting everything to work properly. For working this out, the Ubuntu documentation (both official and unofficial) was invaluable. I've found the unofficial docs to be more comprehensive since they do a better job of covering command-line and server material, while the official stuff seems more focused on the desktop GUI. Obviously, since the server version doesn't have a GUI, that's not especially useful.

Thanks to the magic of Ubuntu, a LAMP server needs only one command to install. Obviously, there's a bit of fiddly configuration that follows, but it's still a walk in the park compared to the downloading and compiling some OSes would have you doing. The masochists that prefer that sort of thing are welcome to it, and it's certainly true that the customization available during compile makes for a faster and more effective install... but I can still have my server online hours before you, and that's a matter of days if we're using beginner timeframes here.

The bottom line is that Ubuntu is a good choice for an introduction to Linux server administration. It's be messy at first, but Google is a great place to find tutorials that can walk you through just about anything. Once OnionRings came onto the scene, he was able to help me out with a lot of the more complicated stuff, although it took even him days to figure out how to set up a proper mail server. Trust me, Google Apps is the way to go for email.

Since Linux still isn't my strong point, I'm not going to go into any more detail here, but some of OnionRings' eventual posts should be a little more helpful in that regard.

Retrospective 5: Money

Continuing from where we left off on Friday's post: as a result of the move to a dedicated server, our monthly costs went from $5 (or $0, depending how you do the math) to roughly $80. That's a huge jump for a small site. I made a news post pleading for donations. EasySpeedy required 3 months' worth of hosting payments in advance, plus a €60 setup fee, when paying with PayPal. As a result, we had to raise €240 up front, or $360 US. The community came through. In two days we'd raised $250 and had enough with our previous donations to make the move.

However, going to the member base with hand outstretched is very poor form, and something I like to avoid at all costs. By the way, when you see a host do a donation drive, don't forget that there are a lot of generous people out there and the host may be making more money than you think. I heard secondhand of a site where things slowed down and the administrator announced they needed a new server and started a donation drive. A few weeks later, the site sped up again, "we're on our new server", and the admin walked away with a cool $10,000. Don't let that prevent you from being charitable, especially to a small site—please don't—but that's something to bear in mind.

Anyhow, we obviously needed to find a more sustainable way to supplement our income to ensure that we could make our next hosting payment. I hadn't gotten around to looking into advertising up to that point because it often requires credit cards, mailing addresses and other undesirables. The provider that we did end up with was Project Wonderful. They're wonderfully subversive to normal advertising models, which appealed to my rebelliousness. They certainly don't pay as well as other providers might, but it's all right. Instead of scams, spyware and pyramid schemes, we got ads for web comics and independent online clothing stores. Again, a little subversive.

I also implemented a few perks for donors. There's not a lot of incentive to offer without removing features for regular members, so I added the usual formula of donor star, your choice of username color (staff names in italic), and x days of status per dollar donated. It seems to have worked all right, and we did manage to meet our next quarterly payment with a bit to spare.

I have a feeling there was something else I was planning to mention here, but since I'm actually writing this days in advance as a result of my unnecessarily long post attempt on Thursday, I imagine I'll have time for it to come to me. If not, maybe you'll be lucky and I'll remember later on and tell you then.

Retrospective 4: Dedicated Hosting

In April, our host began complaining about the load of the burgeoning site, so we started shopping around for dedicated servers. Ketchup ended up finding a server for €60/month ($80 US at the time) with EasySpeedy, a Danish web host. I asked them in advance how they felt about running a BitTorrent tracker, and they responded with the attitude I mentioned in Tuesday's post: "if we don't hear about it, it's fine". And it was. We were hosted there for over a year before we outgrew that server and had to look elsewhere, and no copyright holder contacted them in that time.

There are a couple of notable things about this move. First, it was the first time I'd used Linux to any great extent, and certainly my first experience with server administration. I had enough experience with UNIX on Mac OS X to know how to navigate directory structures and edit files, but that's about it. I selected Ubuntu to install on the server because I'd heard it was the friendliest, and that certainly did help me a lot. I'm still amazed that I was able to keep the site running with such limited experience, basically working off what tutorials I could find with Google. In fact, I'm still learning, so later posts (particularly once OnionRings comes onto the scene) ought to have some useful info on that front.

Naturally, the second point is the financial. Sorry for the short posts lately, but I had to break what was originally a ridiculously long post down a bit. So, on Monday, we'll talk money.

Retrospective 3: Shared Hosting

Up to now, I've been talking mostly about setup. Now it's time to delve a little more into the day-to-day operation of the early months of the site's life.

As one might expect, things were pretty slow for several months after our January launch. With the low activity on the site, there wasn't a lot to do aside from tinker with features as suggestions filtered in. The more I customized, the more I realized how woefully unequal TorrentTrader was to the task. Unfortunately, I was committed and to change at that point would mean more work still. Instead, I quickly developed a policy of rewriting entire pages as a whole rather than tweaking what was already there.

One of the suggestions was that I add a favicon to the site. The existing logo didn't fit the small size very well, so I set out to make a new one. The product ended up looking pretty good, so I redid the entire site design based on the icon. What I wouldn't give to have that kind of time at my disposal nowadays...

Aside from the constant refinement of features, there's not a lot involved in running a small tracker, no more than what's involved in tweaking any other pet website. Since someone else may be taking care of your server for you, you may not even need to worry about that. The whole experience feels a little like running a lemonade stand on a deserted street, sitting in the sun while the ice in your mother's lemonade pitcher melts. It's just that exciting.

Despite the last few paragraphs of doom and gloom, we did grow slowly thanks to some of the methods I detailed yesterday. In March, a member asked how to donate to the site. Bemused, I opened a PayPal account. The $5 donation felt like a fortune. Since the site was sharing a hosting account with my personal stuff, we were essentially being hosted for free at the time.

Don't worry, all that changes soon enough. More on that tomorrow.

Retrospective 2: Promotion

Promoting any new site is a major issue. Word-of-mouth on the internet is not to be underestimated, but it won't do much for a site with no unique features—see yesterday's entry for a bit of discussion of that one. Even on an unusual site, word-of-mouth won't get things going very fast from the beginning, and you end up in the vicious cycle of having no material, making it difficult to pull in new members to add new material. Some people have ridiculously large collections of material gathered from various sources, such as audiobooks or movies. I know of people whose porn collections are measured in terabytes, but I prefer not to know them personally. Regardless, if you can upload your own collection piece by piece, you may be able to make your tracker appear much larger than it actually is, and use your unique material to pull people in.

Unfortunately, there's not a lot you can do by way of promotion in the early days aside from shameless spamming. A point related to my earlier mention of the usefulness of connections: I am active enough on some high-profile communities that I could get away with throwing links into my signature and posting a few spam threads. Don't try this as a new member, and certainly don't try to join communities exclusively for the purpose of posting spam messages. It will only generate ill will towards you and your site. If you don't have a place to do this, it may be a good idea to hold off on your plans until you do. It's important.

Dump sites are great promotional methods. Sites such as MiniNova and Demonoid allow you to upload torrents tracked elsewhere, so configure your new tracker to accept unregistered IP addresses (temporarily if you intend to go private) and upload your torrents to a bunch of dump sites like that. In the torrents' descriptions, include a comment such as "find more great torrents like this at www.example.com". You can even throw a text file inside the torrent to the same effect. You'd be amazed by how well it works.

I'll leave you today with a couple of graphs showing the rate of growth of the site. As you can see, it pretty quickly surpasses the size of an average private tracker, so the processes involved may start to diverge a bit as we progress and as I start to deal with a larger and larger tracker.

Retrospective 1: Conception

Since starting TorrentFries, I've had a number of people ask me how to start a successful tracker. The first question I ask is always "What sort of material do you want on the site?" Only one person gave me a response other than "everything". Incidentally, I'm now involved in some small way in the development of that other person's tracker. The two aren't directly related (I'm not a sword for hire), but they're not entirely unrelated either.

Listen, I'll let you in on a secret. There are tons of "everything" trackers out there. Come up with something unique to pull people in.

Personally, I believe in unconventional activism. In this case, deep-fried goods are too expensive, so I thought a BitTorrent tracker might get people talking about the problem. Since nobody else had one going, I set out to create one of my own.

The first step in the process was to find a host and a tracker. In this situation, personal connections are nice. I was lucky enough to know a guy that ran a shared web host, so I was able to get things running for $5/month, plus the cost of registering a domain name ($12/year). However, any shared host worth their salt should figure out that someone is running a tracker pretty quickly, so don't count on slipping that under the radar. Instead, you might consider a VPS, which allows you to rent part of a server, rather than having to pay for the whole thing. You should have plenty of control over your server, and get more leeway in terms of traffic. A basic VPS will cost you around $40/month.

However, be aware that there's no way you can buy security for any price you can afford at this point. The best you can hope for is a web host that will look the other way until they hear from a copyright holder. Most dedicated and VPS hosts I've encountered will do this; in fact, some hosts have told me this directly. What you can do is maintain daily off-site backups and have a backup host lined up in advance in case of the worst. We'll talk about contingency planning a bit later on.

Choosing the software involves a fair bit of research. I was pretty lazy on that point, and have been paying for it ever since. The only two trackers I checked out were TBdev and TorrentTrader Classic. TBdev is used by a lot of big-name private trackers such as Torrent-Damage, TorrentLeech, the late OiNK, LearnBits, BitMe, BlackCats, BrokenStones and so forth. However, the administration functions are almost non-existent, and the amount of customization available in TorrentTrader appealed to me. I didn't want to spend much time digging around in the code to get everything to work properly.

I've got a fair bit of experience with graphic and web design, so I took it as given that I'd build a new theme from the ground up. The product was in retrospect not that impressive, but it was certainly usable and unquestionably better than what I'd started with.

Finally, before going live, I asked a friend to take a look at the site. I expected him to spend perhaps 5 minutes and respond with a "looks good", but he ended up giving me a two-page essay including everything down to copy-editing in the FAQ. It was great help, and I was able to refine a lot of features that way.

Tomorrow: details on how to pull members in.

Background

Okay, this is my first real blog attempt, but I thought it might be a good idea to talk a bit about the life of a BitTorrent tracker admin (or SysOp, for those that subscribe to the TBdev lingo). The challenges of running a tracker are pretty unique to the subject, and I haven't seen any really in-depth discussions of what this entails. The closest thing I've seen is a TorrentFreak article which only really focuses on the personal implications. Hopefully, this blog will give readers a good idea of what the process entails, to warn away the unprepared and prepare those that want to go ahead with starting their own sites.

I've been running a tracker for almost two years now. I won't say what tracker for security reasons (the same reason I'm using a different alias here), but the astute may figure it out anyway. If you do, please be quiet about it. I'll be talking frankly about security measures and stuff that is normally kept under wraps, so I don't want most people to make the direct connection to the site. Instead, I'll be referring to the site as TorrentFries.

TorrentFries is middlish in size, no Demonoid, but certainly enough to keep me occupied. Signups are permanently open, and accounts never expire. At the moment, the tracker is open as well, so you don't need an account to download stuff. The tracker occupies a pretty unique niche in the torrent world, and is the only one of its kind that I know of. I would love to see more of its kin appear, but so far there's nothing that I've heard of.

This brings me back to the original point of the blog. We need more trackers out there, and it's up to you to make them. So, learn from my mistakes and the things I've accidentally done right and you'll do just fine.

Over the next few weeks, I'll make a series of retrospective posts covering the process we've gone through up to this point. Once that's done with, we can get going on the here-and-now. So hop on and enjoy the ride. :)