Do developers make good systems administrators?
It's often the case in a small or startup business that the person assigned to perform the general systems administration tasks is the unfortunate developer who asked about the backup strategy first. Whilst this seems like a really good idea at the time, there's a fundamental reason why it isn't.
Simply put, a developer has to be an optimist, always looking forward, seeing if new ideas work. These qualities in a sysadm can be rather dangerous. Risk-taking with the infrastructure is something that tends to shorten the life expectancy of many companies when their gamble doesn't pay off.
No, a sysadm needs to be a pessimist, meticulous in their job. In their eyes, it's not if, but when it's all going to turn to custard, and am I adequately covered when it happens? You need to be prepared for any eventuality you can think of, so you need to constantly monitor your systems as well. This will require resources that can be hard to justify – test servers to ensure your backups work, dedicated ( preferably remote ) hardware to monitor system resources and availability, and of course your time spent monitoring mailing lists and websites for threats and security breaches relevant to your systems.
As a developer, all you really want to do is to move on to the next challenge. If your code's not perfect, then you can go back and fix it, no problem, no harm done, eh? But if that mistake's just trashed your subversion repository, and the backups don't work / haven't been done for a couple of weeks because of these deadlines, then repercussions could be a bit more serious!
OK, so developers and sysadmins require a different outlook, and will often end up on different sides of a debate. Is this a bad thing? I don't think so. Apart from giving each other something to whinge about, a well defined break between development and administration is advantageous to all. The working application provided by the developer needs the cynical eye of the sysadm to check that it will work well in his production environment, organise any new resources and integrate them, and then ensure that it's deployed in a safe and repeatable manner.
As an example, I recently took over a part time sysadm gig where one of the systems was performing abominably – and a glacially slow website doesn't sell anything! Once I'd got my favourite monitoring tools in place ( has anyone knighted Tobi Oetiker yet? His RRD/MRTG tools are just indespensible! ), the problem was narrowed to database performance. The code was beautifully architected, the database normalized and well indexed, the disks performed well, there was plenty of memory available, so what was the problem? It turned out that the developer had used a VIEW to aggregate data to drive the website – so what: it's just shorthand for directly accessing the underlying tables really isn't it? Well no, not when the generation of that view required full table scans of a number of tables, every time a page was accessed. As a fix, a table was created containing a snapshot of the data instead. Instant cure – not even a code change needed - the data in the underlying tables only changed daily, so the snapshot was updated at the same time. As far as the original developer was concerned, his code worked just fine, and was certainly elegant, but it just didn't scale. Now the design is a bit less elegant, but works well in a production environment. Just a few hours work has made huge changes to the bottom line.
So get yourself a dedicated sysadmin to manage your sites. Often it only needs a few hours a week from a specialist company to get your servers covered. It'll look like a waste of money... until the unthinkable happens.
You know it makes sense.