Big-Tracker Finances

As I mentioned yesterday, I am seriously strapped for time this evening and won't be able to continue my announce-writing series as I had intended. I'll do some coding over the weekend and hopefully get things going again on Monday.

Early in the life of the blog, I discussed the financial logistics of running a small tracker. Well, TorrentFries grew out of that phase. As you grow, so does the complexity of your bookkeeping.

Obviously, the cost of hosting increases as you move to faster and faster servers. However, things also become a lot less predictable. I've previously advised that contingency is a good thing, and nowhere is this more true than in the case of finances. While you're small beans, you will probably be paying the same cost every month for a long time. Hosting is predictable, although we were impacted by the dropping US dollar, which caused our hosting costs (charged in euro) to increase without any benefit to us. Of course, we were growing and becoming richer as time went on, so that was more annoying than difficult.

Eventually, it is likely that donations will outstrip expenses. Don't let it go to your head and don't squander the money. A big site is always in danger of being taken offline or even coming under threat from the copyright police. Having a slush fund handy is invaluable, since you won't even be able to beg for money with the site down. At one point, we had as much as $800 in reserve, plenty to get us a new server if need be. In fact, we were saving up to build our own box to colocate somewhere friendly. That didn't end up happening, but it certainly could have with a few more months. At the time, we were making about $30/mo in ad revenues (Project Wonderful is not the most profitable provider around) and over $500/mo in donations, although donors are fickle and hard to rely on – another good argument for a slush fund.

Now, running into trouble with PayPal (which I assume you are using for your finances) is another concern. I suggest spreading your wealth across multiple accounts, a few hundred dollars in each. If one goes down, you still have access to enough money to tide you through until things get sorted out. Don't use Tor while accessing PayPal. If your geographical IP jumps around too much, PayPal will automatically flag your account as potentially compromised and will freeze your funds until you confirm the account with a credit card. Prepaid cards are handy there, but a massive pain in the ass nonetheless.

How you choose to use your donated money is entirely your own choice, but I have no ethical problem with paying for personal projects that have direct benefit to the site. For example, I bought Coda with donated money, and paid for a course in UNIX administration that significantly improved my ability to do my job. I went from absolutely thumb-fingered to only moderately thumb-fingered, and I think the cost was worth it.

Since there is a massive time investment involved in the operation of a torrent site, I wouldn't begrudge you making a profit, either. The excess donations I received would have amounted to something in the ballpark of $1/hr, and I don't think that's unreasonable. However, profiting from the site opens a whole new can of worms. Profiting from copyright violation puts you in different ethical and legal territory. The implication of sale is what landed the EliteTorrents admins in deep shit, and will get you kicked out of Spain, too. As always, it's a personal decision as to where you want your tracker to go, and I decided to take the high road.

Back to the Basics

I realize that our focus has of late moved from tracker-specific discussion to more-general stuff that applies to any site or server and can be found in a bazillion places online. Since this blog is unique because it does chronicle the operation of a BitTorrent tracker, I'm going to get back to the meat of the thing for a bit. OnionRings plans to continue with his Linux series, but is sadly indisposed this evening.

In some senses I will cover some of the same material as our second-ever blog entry, but with a different focus. While that entry gave a quick overview of why and how to set up a tracker, I'm going to look a little more closely at the technical and logistical aspects of bringing a tracker online.

First off, what do you personally need to bring to the table? (You're probably the only one at the metaphorical table at this point, but the bringing should commence now.) Beyond anything else, you need to understand the BitTorrent protocol, and understand your tracker of choice. We'll talk a bit about choosing trackers shortly. If you plan on running a PHP/MySQL tracker, you'd better have at least a working knowledge of PHP and MySQL. From the get-go, you'll need to be able to peer into the inner workings of your site and see what makes it tick. You don't just need an understanding of the protocol, but of your own tracker. Trust me, admin panels are nice, but some things just require you to get down and dirty. However complete your chosen script may look at first glance, there will quickly come a time when you or your users bemoan the lack or poor design of some feature or another, and you will be forced to go in and remedy the situation. As the tracker grows and moves to its first dedicated server, it will also become important to know your way around Linux to some extent.

Secondly, you need a hook to pull people in. The most obvious aspect is to provide something that others can't. A unique idea is neat, but it's entirely possible to carve out a niche in a "market" that is well-developed but not saturated. For example, if you have terabytes of obscure movies that you have spent years collecting, starting a movie tracker to share these can be an excellent starting point. Of course, you will need to keep all the uploaded torrents active, even though you will be getting very few peers at first. For this purpose, you may want to consider renting a seedbox. The added cost may be hard for a brand new tracker admin to swallow, but your ability to saturate the connections of your first members will do a lot to encourage them to stick around and maybe even contribute some stuff of their own. As well, a unique design may sound frivolous, but it will lend your site credibility and a sense of longevity. I've covered some methods of early promotion in the post I mentioned above, so I won't rehash them here.

I hate to be defeatist, but if you can't meet these requirements, you should think twice about starting a tracker. Of course, the lovely thing about being a human is the capacity to learn, but you should get going on that learning well before you even consider getting into tracker territory. While I'm trying to demystify the role of the tracker administrator, it's certainly a job that not everyone has the skill and disposition to fill with any great success.

Moving on to more technical requirements, you will also need to choose a tracker. That's a given. The three most popular trackers under active development are TBdev, TorrentTrader, and the new Project Gazelle, although there are a ton of other options out there, so you shouldn't feel constrained to choosing one of these three. Shop around a bit. Detailed comparison of these and other trackers is well outside the scope of this entry, and perhaps even of this blog, but this should point you in the right direction. At the end of the day, it's up to you to select the tracker that best suits your particular tastes and needs. As a side note: for the love of God, don't use TorrentTrader Classic.

So, why not just boot Azureus or µTorrent or one of these handy ubiquitous little torrent clients that happen to include the ability to operate as standalone trackers? Hopefully this isn't a question you were asking yourself, but I'll answer it anyway. First, this isn't the purpose they were designed for. Just because they can run a tracker doesn't mean they should. They're not optimized for the purpose, they don't include features that are necessary for the smooth operation of a tracker (they're inherently open to anyone that wants to use them), and they include a ton of features and bloat that will just bog down your server. What's more, part of the sort of tracker we're discussing here is the index, or frontend. This is the bit where torrents are uploaded, downloaded, where users interact with one another, all that sort of stuff. The tracker itself is ridiculously simple in operation, as we will see in my eventual piece-by-piece breakdown of what it takes to build one from scratch. The important business is the complex frontend, and BitTorrent clients' wannabe trackers just can't provide this.

So, once you've selected the script that best fits your needs, it's time to choose a host. I've had some requests to list a number of torrent-friendly hosts, but that's not something I'm going to do for a number of reasons. The most important reason here is that I don't want to endorse a host that turns out to be unsafe, or to lull readers into a false sense of security. You may have noticed that even (especially) the biggest trackers tend to move around a lot. The world of web hosts is intrinsically volatile, so to proclaim that one host is perfectly safe, even if presently true, is to ignore the fact that this may not continue to be the case in the future. LeaseWeb was once a safe haven for trackers, but after losing a lengthy court battle on behalf of Demonoid and other trackers, all the will to fight has gone out of them. Nowadays, if a copyright holder says "boo", they'll turn around and shut you down.

So simply accept that no place is perfectly safe, and operate under the assumption that you could be taken down tomorrow – you could be. A good strategy to find places that are safer is to run whois lookups on established trackers' IP addresses. Presumably, if they are able to operate on a particular host, that host must be somewhat resistant to legal threats. The only problem is that many larger trackers own their own servers which they operate via colocation in the host's datacenter, while you are likely unable to afford more than a rented dedicated server (an important distinction to make). Not all hosts supply both options, and not all do so for reasonable rates. In addition to colocated and dedicated servers, I've talked about shared and virtual hosting in that previous article, so I'm not going to rehash that here.

This should be all you need to get you off the ground. Again, I'm not going into exacting detail on some of the technical points because I'm assuming that you will be able to handle the fiddly bits of your own accord. If you can't, like I mentioned, you probably shouldn't be running a tracker.

If there are any points on the subject (or any other, really) that you would like us to cover, leave a comment on this post and we'll do our best to touch on them in future posts.

Write Your Own Disclaimer (in your spare time)

By popular demand, I give you: do-it-yourself disclaimers.

Okay, first I'd like to offer a disclaimer about this post. I'm not a lawyer, and I haven't consulted with one in writing mine. I'm planning to have a lawyer member look over/rewrite our own disclaimer. Well, a member that is ostensibly a lawyer—see my previous comments on judging online character.

Anyway, there are a number of important points to cover. I know Demonoid has 34 pages of text in their disclaimer, which apparently contains some amusing points for those with the attention span to read through it. I guess they figure the length is a deterrent to people actually reading it, but lawyers are renown for their long attention spans. I think it's more important to make your point in a concise manner that your friends can understand as well as your enemies.

Basically, you need to address the following points:

No "uploaded"/copyrighted material is stored on this server. It's not. Only metainfo files (.torrent) are stored on the server, and they only contain SHA1 hashes which describe the material. All material is located on individual users' computers, so you're perfectly safe in making this argument.

Given that part of the disclaimer, for the love of god don't use your server as a seedbox. That'll just increase your culpability in this sort of thing and will make the bit about copyrighted material a total lie. What's more, running it as a seedbox can have all manner of detrimental effects on your server's performance, not just in terms of bandwidth but also drive speed. I know you probably have a 10 or 100 Mbps line with very low utilization, but resist the temptation.

The site cannot be held accountable for the behavior of its users. It is merely offers an indexing service. It's true. You can make the analogy to Google potentially indexing child pornography sites, but that's a little extreme. The point is that you merely provide a service.

It is the responsibility of the users to ensure that they hold the copyright or attain proper permission prior to uploading material. You may want to hide this statement a little, since some members will inevitably take it to heart. I've definitely had some concerned posts to that effect, to which I've given some rather tongue-in-cheek responses. Obviously, the idea is to turn one big fish into a whole lot of little fishies, reducing your own culpability without greatly affecting the masses.

Finally, you will comply with any and all formal copyright notices sent from the proper jurisdiction (where your host is located, since they shouldn't know where you are personally located). Obviously, you're not hosted in the US... are you? The idea is to make a hell of a lot of work for anyone that would attack you, encouraging them to go after other sites. You also need to provide a clear email address for copyright complaints. Better that they go whining to you than to your web host, which will always have contact information prominently displayed and will (with some exceptions) likely take you offline if a copyright owner says "boo".

It's doubtful that this will do a whole lot of good, but you never know. Once I get some better legal advice, I'll make another post correcting any glaring mistakes...

Site Contingency and Redundancy

Today's post will cover contingency and redundancy, of which we've learned the hard way over time running TorrentFries. Backups are crucial to a holistic approach on contingency and redundancy, but there’s more. By the way, I’m OnionRings, the “fellow admin and general Linux and security geek.”

If you’re just beginning a site, hardware redundancy isn’t possible since you will likely have a shared or dedicated server host, but there are a few other areas of redundancy to think about. Attack issues of redundancy and weaknesses before they occur and in the order of likelihood. In the case of TorrentFries, the most likely thing to happen is for a host to get a copyright infringement notice.

Selecting a host that’s least likely to take fright in a copyright notice is the best choice to reduce risk of downtime, but that can be difficult to know right away. We’ve often asked our hosts what their policy on copyright material was before moving to them. Usually they are happy to take your money up until they receive a notice.

In the past, a host has given us 12 hours notice to remove material before the server would be shut off. That might sound like a lot of time, but when they issue them at 5 am in your time zone, it could be a few hours before you even notice the warning. At that point, you need a new host quickly. Therefore, select a host or two that would be sufficient while you have the time to research it and before you get a takedown notice. It can take days for a hosting company to get you login credentials much less the time it takes to research your best option. Plan ahead.

TorrentFries was once put out of commission after receiving an infringement notice while our host and the domain registrar were the same company. In that case, we were locked from our account and couldn’t even redirect our domain to the new server we purchased. So if uptime means anything to you, register your domain with a different company than you host with.

Short of having an expensive hot spare server wasting CPU cycles somewhere, there are other ways to mitigate risks of takedown. Parceling out the server’s work to subdomains that reside on different hosts is one way to achieve better contingency. In this case, put the tracker announce and scrapes on a separate server than the main site so the tracker doesn’t go down if the main site is taken down. Leveraging this technique can be a handy method to keep staff communications up during downtime where forums, email, or Jabber are critical. If set up with some thought, your database latency won’t take a huge hit and purchasing a couple cheaper servers might be only slightly more expensive than buying one powerful server.

Finally, there’s quite a bit of rework involved in installing and configuring software each time you move to a new server if you’re on a VPS or dedicated server. Even though I’m a Linux geek, I have to look up some things that I don’t use everyday. It’s nice to have a list of the tutorials you’ll be using or a checklist of the things that need to be done starting from disabling root SSH login and ending with the final tweaks in Apache or MySQL configurations. If you’re savvy enough, write up a bash script to handle all those dull tasks.

Hopefully your site will learn from some of our mistakes and you’ll be able to keep your uptime positive.

Backups

Okay, I can't stress this enough: an off-site backup is essential to the operation of any website, especially a tracker. Database backups are the most important, but backing up your files from time to time is necessary as well.

For example, if your thumb-fingered database maintenance *cough* results in a few thousand damaged or deleted rows (done that at least twice) or in the accidental deletion of your privileged user from the database (done that, too), you need a backup to fall back on. It's nice and convenient to have a local backup for that, but you also need to upload that backup to another server (or your home computer) in case the server runs into technical problems or you're suddenly taken down.

I was pretty lazy with backups while running on a shared web host, so I don't have a lot of wisdom to offer there. My host informed me that he ran nightly backups of all databases, so there was no need for me to maintain my own. (My PHP-based backup script had become pretty heavy for his server.) I think this is pretty normal behavior for shared hosts, although I don't know that all of them would give you access to those backups on demand. The point for them is protection from pissed-off customers in case they muck something up on their end or the server crashes or some such. So, when in doubt, back up.

The crux of the matter is that I'm lazy, and you probably are too. If it were left up to me, there'd be thorough backups for a few days, followed by a span of weeks or even months without backup. If you're lucky, that'll be all you need. How lucky do you feel, punk?

Therefore, you need to automate two things: the backup itself, and the process of uploading the backup to an external server. I'm not going to walk you through the setup process since that sort of info is pretty easy to find online, but I'll run over the general concept.

For the backup process, I set up cron jobs with two shell scripts: daily.sh and weekly.sh. When they're set to run is pretty self-evident. daily backs up the database (mysqldump -uroot -prootdatabasepassword --all-databases > backup.sql) and the uploaded metainfo (.torrent) files, while weekly backs up the entire web directory, all the PHP sources and so forth. weekly doesn't have to touch the database, since daily will also be running on that day. Each script then runs tar -czf backup.tar.gz files, with whatever files you intend to compress.

That's all it takes to ensure that you're safeguarded against data corruption. Naturally, there are other pitfalls that you can encounter. Hardware failure, hacking, inept administration, and of course the ever-present DMCA message—all can threaten your data and your backups. For that reason, it's a good idea to maintain off-site backups as well. You can download them manually, of course, but that's not going to be any easier to remember to do on a regular basis than to make the backups in the first place. Instead, I have my backup script upload to an external server via scp. I use that command instead of ftp because it's more secure and easier to automate.

However, scp does require that you have a UNIX box at the end. (Well, there are Windows ssh clients, so maybe there are scp servers as well. Stranger things have happened.) Since I run Mac OS X on my home computer, it's simple to automate the upload to there, or to any other server that I operate. If your shared host gives you rudimentary shell access, it's even okay to use that.

I'm sorry for being so lazy with the tutorial-writing in this one, but this blog entry is already getting rather lengthy and hopefully you can follow me anyway. I just have a few more points to make before I close.

Where one backup is good, archives of backups is better. If you set your script up to name its output files with the creation date, they won't overwrite one another and you'll have a nice collection in case you need to roll back a few days. I have plenty of hard drive space, so I still have backups from 2007. Who knows when you might need them?

Putting all this together, a simple shell script might look something like this (lines are double-spaced to avoid confusion in the case of line breaks):

#!/bin/sh

date=`date -I`

mysqldump --all-databases | gzip > /var/backup/backup-$date.sql.gz

scp /var/backup/backup-$date.sql.gz user@host:/dir/backup-$date.sql.gz

Okay, that's all for now, folks. I meant to throw in some horror stories about maintaining/not maintaining backups, but we're out of time. Maybe I'll get to that later on. Rest assured, such stories do exist, and don't forget: always maintain an off-site backup!

Retrospective 1: Conception

Since starting TorrentFries, I've had a number of people ask me how to start a successful tracker. The first question I ask is always "What sort of material do you want on the site?" Only one person gave me a response other than "everything". Incidentally, I'm now involved in some small way in the development of that other person's tracker. The two aren't directly related (I'm not a sword for hire), but they're not entirely unrelated either.

Listen, I'll let you in on a secret. There are tons of "everything" trackers out there. Come up with something unique to pull people in.

Personally, I believe in unconventional activism. In this case, deep-fried goods are too expensive, so I thought a BitTorrent tracker might get people talking about the problem. Since nobody else had one going, I set out to create one of my own.

The first step in the process was to find a host and a tracker. In this situation, personal connections are nice. I was lucky enough to know a guy that ran a shared web host, so I was able to get things running for $5/month, plus the cost of registering a domain name ($12/year). However, any shared host worth their salt should figure out that someone is running a tracker pretty quickly, so don't count on slipping that under the radar. Instead, you might consider a VPS, which allows you to rent part of a server, rather than having to pay for the whole thing. You should have plenty of control over your server, and get more leeway in terms of traffic. A basic VPS will cost you around $40/month.

However, be aware that there's no way you can buy security for any price you can afford at this point. The best you can hope for is a web host that will look the other way until they hear from a copyright holder. Most dedicated and VPS hosts I've encountered will do this; in fact, some hosts have told me this directly. What you can do is maintain daily off-site backups and have a backup host lined up in advance in case of the worst. We'll talk about contingency planning a bit later on.

Choosing the software involves a fair bit of research. I was pretty lazy on that point, and have been paying for it ever since. The only two trackers I checked out were TBdev and TorrentTrader Classic. TBdev is used by a lot of big-name private trackers such as Torrent-Damage, TorrentLeech, the late OiNK, LearnBits, BitMe, BlackCats, BrokenStones and so forth. However, the administration functions are almost non-existent, and the amount of customization available in TorrentTrader appealed to me. I didn't want to spend much time digging around in the code to get everything to work properly.

I've got a fair bit of experience with graphic and web design, so I took it as given that I'd build a new theme from the ground up. The product was in retrospect not that impressive, but it was certainly usable and unquestionably better than what I'd started with.

Finally, before going live, I asked a friend to take a look at the site. I expected him to spend perhaps 5 minutes and respond with a "looks good", but he ended up giving me a two-page essay including everything down to copy-editing in the FAQ. It was great help, and I was able to refine a lot of features that way.

Tomorrow: details on how to pull members in.