Folding@Home

I've recently joined the folding@home project under the name srkiNZ84. Folding@Home is a hugely distributed computing project with the purpose of doing protein folding (anyone remember seti@home?). I've joined TeamUbuntu and started crunching my way through those work units.

At first what got me intrigued with this project was just trying to benchmark my computer(s), but now what really keeps me interested in it is the competition. You see, with this project you get to keep statistics on how many work units you've completed, in what kind of time frame etc... and every member of the project is ranked. This gives rise to cool personal and team statistics screens like this one from extremeoverclocking or this one from xtreme cpu. Personally I think the competition is one of the main reasons that the project has so many followers.

Right now I'm ranked 455th in my team and 171,330th overall. This isn't too bad considering I've only started. However, the only real way I'm going to get into the top 10% or so of the rankings is by making use of a high performance GPU client. These are versions of the folding@home software that run on your graphics card and make use of it's parallel processing capabilities to complete work units in a much shorter space of time. ATI were the first to produce a client for their line of graphics cards, but were soon followed by nvidia, which blew them away with their much better CUDA performance. I'm not even sure that there is a GPU client for Linux (at least not on the official folding@home website), but I haven't been through the whole recruitment post on ubuntu forums, so I'm hoping there's some clues on there.

Signature Errors on Ubuntu repositories

On all of the Ubuntu installs I've ever had, I almost always eventually run into the following error when running the 'apt-get update' command or hitting the 'Reload' button in Synaptic.


Error:
W: GPG error: http://nz.archive.ubuntu.com hardy-updates Release: The following signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key
W: You may want to run apt-get update to correct these problems


From what I understand the problem comes from apt downloading an incomplete/corrupt signature file, meaning that the signature then doesn't match that of the packages. The fix at the moment is to simply delete the incomplete/corrupt signature file and download it again. To do this run the following command:

sudo rm /var/lib/apt/lists/partial/*

Afterwards running apt-get update doesn't return the same error.

Setting up a VPS - Part 3 - Ruby on Rails

One of the things I wanted to do with this VPS was to have a go at getting a Ruby on Rails environment going and seeing what the hype was all about. This turned out to be more trouble than I originally thought. The problem was that after following the RoR install guide found here, the machine kept on crashing whenever I got to the part about updating RubyGems. i.e.

sudo gem update --system

This was finally traced to the fact that the VPS only had 128MB of memory and no swap space. After adding another 128MB of memory and dedicating 512MB of swap space, the update finished fine and I had a Rails environment. The whole issue did take a while though, mostly because I was busy at work and often couldn't reply to HostingDirect straight away.

Setting up a VPS - Part 2 - Postfix Virtual Domain/Users

The next step in setting up the VPS, was installing and configuring the mail server. For this job, I've gone with the current king of MTA's - Postfix. The basic approach I've gone with when setting this up is to start simple and then add functionality bit by bit. In order to do this I've basically followed the guide found here. In the end I've ended up with support for virtual domains (seperate domains) and virtual users (non-UNIX users) with a flat file backend. I don't have much to add to this tutorial, except to point out that in the setup where it says 'virtual_uid_maps = static:5000' - this means that the process which is trying to deliver the message (i.e. write to disk and create any files/folders necessary) will be running as this user. So, there's no point in setting it to 5000, unless there is a user with that id, which has write access to the virtual domain folder.

I've also had to add some directives to prevent the mail server being flooded with spam. Directives which check that the server comes from a FQDN as well as checking that the IP Address isn't on any blacklists. i.e.


# Wait until the RCPT TO command before evaluating restrictions
smtpd_delay_reject = yes

# Basics Restrictions
smtpd_helo_required = yes
strict_rfc821_envelopes = yes

# Requirements for the connecting server
smtpd_client_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_rbl_client bl.spamcop.net,
reject_rbl_client dnsbl.njabl.org,
reject_rbl_client cbl.abuseat.org,
reject_rbl_client sbl-xbl.spamhaus.org,
reject_rbl_client list.dsbl.org,
permit

# Requirements for the HELO statement
smtpd_helo_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_non_fqdn_hostname,
reject_invalid_hostname,
permit

# Requirements for the sender address
smtpd_sender_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_non_fqdn_sender,
reject_unknown_sender_domain,
permit

# Requirement for the recipient address
smtpd_recipient_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
reject_unauth_destination,
permit


These directives originally came from the email section of an article on howtoforge.com about setting up Mandriva Directory Server.

There's still a lot of work to go with setting up this email server, I haven't even got to setting up Dovecot and SASL. Then I want to set up Amavis and combine it with ClamAV and SpamAssassin(with Baysian filtering and feedback). I also need to setup DKIM, both for signing mail coming from the server and for checking incoming DKIM messages and ofcourse as always there's a need for a decent web front end, to enable you to check your mail. I've been hearing good things about Google Apps, but I don't know anyone that's set it up on their own servers. I wonder whether that's even possible or whether you have to use google's mail servers?

So many technologies, so little time... and this is only setting up the email :)

Setting up a VPS - Part 1 - Hosting, SSH Security and ntp

Got a VPS from an outfit here in NZ called HostingDirect. Opted for Ubuntu 64-bit edition with the Small VPS package (128MB RAM, 10GB disk, 1 IP address). Also got domain registration (cheapest in NZ) and hosting with them which comes with free website hosting, which is nice.

The configurable options in the VPS setup allowed you to select LAMP setup for $150, Email server (SMTP, POP3, IMAP) for $60 and Security Tools for $45. I thought these prices were a bit steep, especially since the Small VPS package only cost $25/month after GST. But then I reminded myself what I charge for setting up such systems and it made sense. I didn't opt for these services, preferring to set them up myself.

So the VPS was provisioned in the afternoon on the 28th but I didn't have time to start configuring it until that night when I came home. By time I started having a look at it, there were already signs of brute force attacks on the ssh server. So the first thing I did was to create a new non-root user and add him to the 'admin' group which was already setup in the sudoers file (mimicking the typical Ubuntu setup). From here I disabled the root ssh login and changed the ssh port to 222. Later I changed the ssh port back to the standard 22 and installed a great new piece of software I found called 'fail2ban' which bans login attempts for a period of time based on the number of unsuccessful login attempts.

Before sorting out the ssh server and fail2ban, I did the obligatory 'apt-get update' followed by an 'apt-get upgrade' which all ran fine. I also did a check on the version of Ubuntu and kernel, with the follwing results:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 8.04.2
Release: 8.04
Codename: hardy

$ uname -a
Linux example.org 2.6.24-23-xen #1 SMP Mon Jan 26 03:09:12 UTC 2009 x86_64 GNU/Linux

So I ended up with Ubuntu 8.04 LTS 64-bit version, which is exactly what I wanted. Shopping around for NZ VPS sellers, I found that a lot of them offered Ubuntu 7.10, which I found strange. I would think more people would prefer the long term release, maybe something to do with stability issues of each distribution running on Xen.

The next thing to set up was the ntp deamon, whch was quite straight forward and only involved adding the line 'server nz.pool.ntp.org' to the '/etc/ntp.conf' file and restarting the ntp daemon.

The VPS also came with access to XenShell, which is a way to administer your VPS through Xen (kind of like VMWare's server console). I've never worked with XenShell before so I'll have to look for a good tutorial to figure out how to make use of this tool.

That's all for today, it's late now and tomorrow I'll start setting up Postfix and all the neccessary extras, a task which it is much better to attempt with a clear head.

Linux Sysadmin Tools

I just found this really useful site 'Ubuntu Server Guide'. It gives a nice general overview of how to set up and configure most of the main uses of your Ubuntu server. There's a few gems in there, like a really useful little package called etckeeper, which is basically a version control system for your '/etc' directory, meaning that you always have backups of your configuration and that you can tell exactly when changes to the config were made and by who. The seems like such a simple, yet brilliant idea, I can't believe the package isn't standard on any distro. It reminds me of Sun's (now Oracle's) ZFS and it's ability to take snapshots and to be able to 'roll-back'.

The other really useful article I came upon when I was investigating a possible break into our servers was this one and this one, explaining how with the use of the 'chattr' command you can make files which are 'immutable' (can't be modified by anyone, including root) or files which are append-only. The append-only would be brilliant for logs, meaning that a person breaking into your machine couldn't just go and modify the logs to cover their tracks. However, this assumes that the person would not have gotten root access, otherwise they could just set the file to 'not-append-only' change the file and then make it look like it hadn't been modified. Also, these tricks apparently only work with ext2/3, so anyone using other file systems is out of luck.

Algorithmic Trading

Apparently there's a way you can get your computer to hook into the worlds financial systems and automatically buy/sell shares. How much attention did you pay in AI class?

Makes me think of the day when all the stock markets will be run and controlled by robots... can't wait