Getting Firefox Sync server running

Have been setting up Firefox Sync, following the instructions found at: https://docs.services.mozilla.com/howtos/run-sync-1.5.html

The instructions were pretty straight-forward, got it up and running in no time (even ran the tests to ensure everything was passing).

However, after reconfiguring the browser to point to the new Sync URL, nothing happened. (In the end it turned out I forgot the "/token/" part of the URL: https://bugzilla.mozilla.org/show_bug.cgi?id=1032039)

Debugging Tips:

* By default if you run the server using "local/bin/pserve syncserver.ini" the server logs to console.

* The database the server saves to (by default) is "syncserver/syncserver.db", which you can view using SQLite Browser.

* To manually start a Sync, you can go to "Tools" -> "Sync Now"

* To view the browser (client) sync logs, enter "about:sync-log" in the URL bar. You can turn on logging on successful sync by toggling the "services.sync.log.appender.file.logOnSuccess" configuration variable.

* At time of writing there's a bug where the URL can be set back to the default in some instances: https://bugzilla.mozilla.org/show_bug.cgi?id=1003708

* You can modify the "services.sync.syncInterval" variable to make the browser attempt to sync more frequently (value is in milliseconds).

Getting Risk of Rain working

I bought the Humble Indie Bundle 13 and downloaded Risk of Rain (non-Steam version). Tried firing it up, when it wouldn't run, giving error:

"error while loading shared libraries: libopenal.so.1"

Found that it required a bunch of 32-bit packages, namely:

* libopenal1:i386
* libxrandr2:i386
* libglu1-mesa:i386

 which you had to install with "sudo apt-get install [package name]"

NOTE: To get "Shadowrun Returns" working, also had to install the "libxcursor1:i386" package.

NOTE2: Am running Ubuntu 14.04LTS 64-bit

Configuring boot services

One of the common tasks when setting up a server is to configure whether a service is set to start up on boot or not. This is handled differently on different versions of Linux.

To list all services and whether they're set to run on boot:

RHEL/CentOS/Fedora

chkconfig --list

Debian/Ubuntu

rcconf

NOTE: this program doesn't come installed by default

Enable a service to run on boot:

RHEL/CentOS/Fedora

chkconfig [service name] on

Debian/Ubuntu

update-rc.d [service name] enable

Disable a service from running on boot

RHEL/CentOS/Fedora

chkconfig [service name] off

Debian/Ubuntu

update-rc.d [service name] disable

Ubuntu - The following packages have been kept back

If you've used Ubuntu for long enough, you'll find that eventually you'll run into a problem when upgrading the installed packages. When running apt-get from the command line, the problem manifests itself as the following:

$ sudo apt-get upgrade
[sudo] password for srdan:
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages have been kept back:
  linux-headers-server linux-image-server linux-server
0 upgraded, 0 newly installed, 0 to remove and 3 not upgrade

The short answer is that you should be able to upgrade by running the "apt-get dist-upgrade" command:

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree      
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  linux-headers-3.2.0-27 linux-headers-3.2.0-27-generic linux-image-3.2.0-27-generic
The following packages will be upgraded:
  linux-headers-server linux-image-server linux-server
3 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 51.2 MB of archives.
After this operation, 217 MB of additional disk space will be used.
Do you want to continue [Y/n]?

The long answer comes from the man page of the "apt-get" command. In particular, if you look at the description of the "upgrade" argument, two sentences stick out:

"under no circumstances are currently installed packages removed, or packages not already installed retrieved and installed."

"New versions of currently installed packages that cannot be upgraded without changing the install status of another package will be left at their current version."

Because you can't upgrade the "linux-image-server" (a.k.a. the kernel) without upgrading the headers as well (technically you can, but it can lead to serious problems) it won't let you upgrade them using the "upgrade" command. Either that, or the "new" linux image package requires that a "new" linux headers package be installed, violating the requirement that packages not already installed not be installed.

The reason that the "dis-upgrade" command works, where the "upgrade" command does not is that "dist-upgrade" takes into account dependencies between packages. Also from the man page:

"dist-upgrade ... intelligently handles changing dependencies with new versions of packages; apt-get has a 'smart' conflict resolution system, and it will attempt to upgrade the most important packages at the expense of less important ones if necessary."

This does beg the question though as to why not just use "dist-upgrade" all the time or incorporate the conflict resolution system into the "upgrade" command?

I suspect the answer has something to do with how each "edition" of a distribution is defined. i.e. Ubuntu 12.04 will ship with version 3.1.13 of the "at" package and all other packages that are a part of this "edition" should work with that version. Having many different packages depend on specific versions of other packages could end up in a situation where it becomes very difficult to upgrade any individual package.

Internal LAN on LXC

When I wrote my previous post about setting up LXC, one of the things I found was that when installing the lxc package, it went ahead and created an "lxcbr0" interface.

It turns out that this interface is actually an "internal network" which you can connect your VM's to if you want them talking to each other directly, as opposed to any network which the host is also on.

To setup my VM's I just added another interface and connected it to the "lxcbr0" bridge, by adding the following lines to the configuration:

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 4a:49:43:49:79:ef

and then configuring the interface in the "interfaces" file:

auto eth1
iface eth1 inet static
    address 10.0.3.2
    netmask 255.255.255.0

Then I did the same to another VM and they were able to talk to each other.

Btrfs snapshots and LXC

In my previous post I talked about LXC, which is a light-weight virtualization technology for Linux. One thing which LXC lacks is the ability to make snapshots. As the VM is running as a regular process in RAM, at the moment it's not possible to just make a copy of the ram file, as it is in VMware etc...

So, in order to work around this, we're only going to be taking snapshots of VM's when they're shutdown and we're going to be making use of the Btrfs snapshot functionality.

First of all, we need to create a Btrfs filesystem. I assume that you have a spare drive or partition which you can use for this:

$ sudo mkfs.btrfs -L btrfs-test /dev/sda8 

WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL 
WARNING! - see http://btrfs.wiki.kernel.org before using 

fs created label btrfs-test on /dev/sda8 nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00GB 
Btrfs Btrfs v0.19

Now we can mount the filesystem:

$ sudo mount /dev/sda8 /lxc

Before we set about creating our VM's, we're going to create some "subvolumes", which we'll be able to snapshot. We're going to use this feature of Btrfs to handle the snapshotting of our VM's.

$ sudo btrfs subvolume create /lxc/vm0
Create subvolume '/lxc/vm0'

Now that we've done this, we can go ahead and create a VM using the template scripts, and configure it, as described in my previous post.

$ sudo /usr/lib/lxc/templates/lxc-ubuntu -p /lxc/vm0 ...

Now that we've created a brand new VM, we're going to create a snapshot of it's "clean" state, so that we can roll back to it, should something go wrong. We do this by creating a snapshot called "clean" of the /lxc/vm0 subvolume.

$ cd /lxc/
$ sudo btrfs subvolume snapshot vm0 vm0-clean
Create a snapshot of 'vm0' in './vm0-clean'
$ sudo btrfs subvolume list /lxc
ID 256 top level 5 path vm0
ID 258 top level 5 path vm0-clean

Now we can start the VM:

$ sudo lxc-start -n vm0 -f /lxc/vm0/config

And install/configure our software:

$ sudo apt-get install apache2 postgresql

Now, suppose that we're humming along nicely, but then realise that we've made a mistake and installed apache2 instead of tomcat6 and postgresql instead of mysql-server and want to start over. All we would need to do is to delete the "vm0" subvolume and rename the "vm0-clean" directory to "vm0":

$ sudo btrfs subvolume delete /lxc/vm0
Delete subvolume '/lxc/vm0'
$ sudo btrfs subvolume list /lxc
ID 258 top level 5 path vm0-clean
$ sudo mv vm0-clean vm0
$ sudo btrfs subvolume list /lxc
ID 258 top level 5 path vm0

Notice how the ID of the snapshot doesn't change, even though we've renamed it.

Now, when we start up the VM, we can see that the "apache2" and "postgresql" packages haven't been installed yet, because we've rolled our VM back to the original snapshot that we've taken.

Now, perhaps the scenario with installing the wrong packages isn't that realistic (it would probably be easier to just remove the packages instead of rolling back the snapshots), however, this was just chosen to demonstrate the capabilities of the technology and you can probably imagine a scenario where a snapshot would be more useful. e.g. testing out a software upgrade, which you're concerned might break some functionality.

NOTE: Another way of using snapshots is to mount them directly, by passing the "-o subvol=..." option at mount time, as described at: http://btrfs.ipv5.de/index.php?title=SysadminGuide#Managing_snapshots

LXC on Ubuntu

This post talks about how to setup LXC (Linux Containers) on Ubuntu 12.04. LXC is an operating system-level virtualization technology, which allow you to run multiple virtual machines on one host.

There are quite a few limitations to this type of virtualization, when compared with the type of full, emulator style virtualization that VMware, VirtualBox etc... use. One of the main ones is that you'll be unable to run different operating systems on the virtualization host. i.e. we can only run Linux VM's on our host.

The big advantage is performance. Because the host doesn't have to bother with all of the code which does virtual hardware emulation, the virtual machines run a lot faster in general.

So, to setup LXC, we first need to install it:

sudo apt-get install lxc lxctl uuid btrfs-tools

The 'lxctl', 'uuid' and 'btrfs-tools' packages aren't really needed, but come recommended, and it doesn't hurt to install them.

Now, at this point I did a reboot, which may not be necessary, and afterwards checked the LXC configuration using:

lxc-checkconfig

You should find that all of the different settings are set to "enabled".

Now that we've got LXC installed, we can go ahead and start creating our first VM. Luckily, the 'lxc' package comes with a set of template scripts, which make setting up a VM easy. These scripts are located under '/usr/lib/lxc/templates' and to create our first VM we run:

sudo /usr/lib/lxc/templates/lxc-ubuntu -p /lxc/vm0/
 
Where '/lxc/vm0' is the path to the VM. Note that you will have to create this directory as it doesn't exist by default.

Once you run that script, you'll see a lot of output from the VM getting created and initialized. From the output and from looking at the template script, it looks like the template takes all of the currently installed packages, copies them over to the VM filesystem and configures them.

Once, this is done, we need to configure the networking, which I found to be the trickiest part of the setup. Because the guest VM is using the same hardware as the host, it has the ability to use the network interface attached to the host. Now, obviously you don't want both the host and the guest using the same interface, as it will lead to IP address and MAC address conflicts. So, the guest should have a distinct MAC and/or IP address.

There are several ways to configure the networking for the guest VM's and there are example configuration files of the many ways that they can be configured under file:///usr/share/doc/lxc/examples/ (note that you can enter this location into your web browser and it should load). For our purposes, we're going to go with the lxc-veth.conf file, which sets up a virtual network interface connected to a network bridge which we have to create.

So, firstly, we need to create a network bridge, which is done by adding the following lines to the /etc/network/interfaces file:

auto br0
iface br0 inet dhcp
bridge_ports eth0

This will create the bridge that we're going to connect the virtual network interface of the VM to. In order to enable it, restart the 'networking' service:

sudo service networking restart

Note that we have to create a bridge, even if we've only got one physical interface to connect to it.

Then add the relevant lines from the example config to the VM configuration file, under /lxc/vm0/config:

...
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 4a:49:43:49:79:bf
lxc.network.ipv4 = 10.0.0.0/24
...

Note that the IP network should match that of your host interface, otherwise you might have some problems getting an IP through DHCP. Also, note that the MAC address is just taken from the example file an was probably randomly generated.

That should complete the configuration. We are now free to start up the VM using the 'lxc-start' command:

sudo lxc-start -n vm0 -f /lxc/vm0/config

This should start up the VM and bring up the console on the terminal screen. For the default Ubuntu template, you can log in using ubuntu/ubuntu username/password pair.

Once you've logged in, you can confirm that the VM has a different IP address to that of the host and start configuring it. 

Funnily enough, just as I've finished writing up this post, I stumbled upon a Launchpad blog post talking about how they're using LXC to speed up their testing: http://blog.launchpad.net/general/parallelising-the-unparallelisable

How to setup Apache as a Tomcat proxy

In this post we're going to setup Apache to act as a proxy for the Tomcat application server on Ubuntu. First off we need to install the "tomcat6" package from the Ubuntu repositories, which is as simple as:

sudo apt-get install tomcat6
and answering "Y" to download Tomcat along with all of its dependencies. To make sure that the Tomcat server is running, try to open up port 8080 on the machine in your browser. If all is well you will see the Tomcat server Welcome page. If not, you may need to start up the server, which can be done with:

sudo service tomcat6 start

Next we need to install the Apache HTTP server, which "apt-get" also makes easy for us:

sudo apt-get install apache2

Again, just enter "Y" when asked whether to download the package and all of its dependencies.

In order to enable Apache to act as a proxy for Tomcat, we're going to need to make use of the "proxy" and "proxy_http" modules. Unfortunately these two modules don't come enabled by default, so we're going to have to enable them and restart apache for the changes to take effect:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo service apache2 restart

Now we need to tell the proxy module how to proxy the requests and where to proxy them to. For this I've created a "tomcat-proxy" file under /etc/apache2/sites-available/, which we're going to enable using Apache's a2ensite command. The file itself looks like the following:

ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 1000
TimeOut 1000
#
# Configure the mod_proxy
#
ProxyPass / http://127.0.0.1:8080/
ProxyPassReverse / http://127.0.0.1:8080/

After editing the file we enable the site and reload Apache's configuration:

sudo a2ensite tomcat-proxy
sudo service apache2 reload

And that's it! If everything's gone to plan we should be able to hit up port 80 on our server and get the Tomcat welcome page.

Note that this isn't the only way to configure Apache as a proxy. A more sophisticated way is to make use of the AJP protocol/module, which is custom designed to work with Tomcat.

To get Apache proxying to Tomcat using the AJP protocol, we have to enable the Apache module and restart Apache for the changes to take effect:

sudo a2enmod proxy_ajp
sudo service apache2 restart
Next we have to enable the AJP connector in Tomcat. This is done in the /etc/tomcat6/server.xml file. If you edit this file you'll need to uncomment a line, which looks like:

<connector port="8009" protocol="AJP/1.3" redirectport="8443"></connector>

and restart Tomcat:

sudo service tomcat6 restart

The next step is to simply go back to our configuration file under /etc/apache2/sites-available/tomcat-proxy and change the protocol and port of the URL's we supplied the ProxyPass and ProxyPassReverse directives:

ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 1000
TimeOut 1000
#
# Configure the mod_proxy
#
ProxyPass / ajp://127.0.0.1:8009/
ProxyPassReverse / ajp://127.0.0.1:8009/

You'll notice we've replaced "http" in the URL with the custom "ajp" protocol. Restart Apache and hitting port 80 on the server should redirect you to the Tomcat welcome page as before.

What's the difference between the two approaches to proxing? Functionally there's not really any difference that the user gets to see. However, the AJP is a binary protocol, compared to the regular HTTP proxy method that the first approach uses. This should mean less data passed between the proxy and application server as well as lower latencies. Look out for a future post benchmarking the two approaches to see what the real-world difference in performance between the two approaches is.

Writeable Rsync server with authentication

In my previous post we talked about how to setup a simple read-only rsync server. In this post,  we'll be taking that simple read-only example and expanding it to allow multiple users, each with their own credentials.

In order to do this, we need to modify the /etc/rsyncd.conf file, and change the following lines:

read only = no
auth users = bozo
secrets file = /etc/rsyncd.secrets

Note that "bozo" is the username of a fake user we're going to create. We can see in this configuration that we're pointing to a file under /etc/rsyncd.secrets. We're going to have to create this file and populate it with the credentials for any users we have created. In this case, we populate it with bozo's username and password:

bozo:clown

We also have to set the permissions on this file to make sure that it's only readable by the root user, using the chmod command:

chmod 600 /etc/rsyncd.secrets

Now, usually we would run the "reload" command to send a message to the server to reload its configuration, but when we do this for rsync, we get the following message:

$ service rsync reload
 * Reloading rsync daemon: not needed, as the daemon
 * re-reads the config file whenever a client connects.

Which is very useful indeed. Now when we connect from the client side, we have to do so using the credentials which we've just created. The command looks like:

rsync -r bozo@192.168.1.10::public/ .


This will ask us for a password, which we know is "clown" from before after which the copy should start as usual. If you want to test out the write capability of the server, we just need to create a file in our current directory and then execute the rsync command going the other way:

rsync -r . bozo@192.168.1.10::public/

In the future we might want to automate the rsync process in order to have it run as a cron job or other automated job. This means that we won't have a human there to enter the password. This can be gotten around with by using the "--password-file" option of the rsync command, like so:

rsync --password-file=~/.rsync_pass -r . bozo@192.168.1.10::public/

Note that as with the rsyncd.secrets file mentioned previously, you'll have to change the permissions on this file to ensure that it's not world readable. The file itself just needs to contain the password to use and nothing else.

Simple read-only rsync server on Ubuntu

If you haven't heard of it rsync is a piece of software which allows you to keep files in sync over a network, while only copying across the "changes" from one copy to the next. The advantages of this are that a lot less data needs to be transferred than would have to be done with something like FTP or SFTP. This attribute of rsync also makes it perfect for things like backups which don't change much from one iteration to the next.

Installing rsync is as simple as:

sudo apt-get install rsync

Although, I've found that with the server version of Ubuntu, it's already installed after installing the OS.

By default, the server doesn't come configured or enabled to start at boot. In order to configure it, we will need to copy across the example rsync configuration into the /etc directory and modify the /etc/default/rsync file:

sudo cp /usr/share/doc/rsync/example/rsyncd.conf /etc/

Modify the /etc/default/rsync file to look like the following:

# defaults file for rsync daemon mode

# start rsync in daemon mode from init.d script?
#  only allowed values are "true", "false", and "inetd"
#  Use "inetd" if you want to start the rsyncd from inetd,
#  all this does is prevent the init.d script from printing a message
#  about not starting rsyncd (you still need to modify inetd's config yourself).
RSYNC_ENABLE=true

# which file should be used as the configuration file for rsync.
# This file is used instead of the default /etc/rsyncd.conf
# Warning: This option has no effect if the daemon is accessed
#          using a remote shell. When using a different file for
#          rsync you might want to symlink /etc/rsyncd.conf to
#          that file.
# RSYNC_CONFIG_FILE=

# what extra options to give rsync --daemon?
#  that excludes the --daemon; that's always done in the init.d script
#  Possibilities are:
#   --address=123.45.67.89 (bind to a specific IP address)
#   --port=8730 (bind to specified port; default 873)
RSYNC_OPTS=''

# run rsyncd at a nice level?
#  the rsync daemon can impact performance due to much I/O and CPU usage,
#  so you may want to run it at a nicer priority than the default priority.
#  Allowed values are 0 - 19 inclusive; 10 is a reasonable value.
RSYNC_NICE=''

# run rsyncd with ionice?
#  "ionice" does for IO load what "nice" does for CPU load.
#  As rsync is often used for backups which aren't all that time-critical,
#  reducing the rsync IO priority will benefit the rest of the system.
#  See the manpage for ionice for allowed options.
#  -c3 is recommended, this will run rsync IO at "idle" priority. Uncomment
#  the next line to activate this.
# RSYNC_IONICE='-c3'

# Don't forget to create an appropriate config file,
# else the daemon will not start.

The only variable that's really changed from the default is the "RSYNC_ENABLED" which has been set to "true".

If we have a look at the config file under /etc/rsyncd.conf, we can see that we're allowing read-only access to the /var/www/pub directory to any user:


# sample rsyncd.conf configuration file

# GLOBAL OPTIONS

#motd file=/etc/motd
#log file=/var/log/rsyncd
# for pid file, do not use /var/run/rsync.pid if
# you are going to run rsync out of the init.d script.
# pid file=/var/run/rsyncd.pid
#syslog facility=daemon
#socket options=

# MODULE OPTIONS

[public]

comment = public access
path = /var/www/pub
use chroot = yes
# max connections=10
lock file = /var/lock/rsyncd
# the default for read only is yes...
read only = yes
list = yes
uid = nobody
gid = nogroup
# exclude =
# exclude from =
# include =
# include from =
# auth users =
# secrets file = /etc/rsyncd.secrets
strict modes = yes
# hosts allow =
# hosts deny =
ignore errors = no
ignore nonreadable = yes
transfer logging = no
# log format = %t: host %h (%a) %o %f (%l bytes). Total %b bytes.
timeout = 600
refuse options = checksum dry-run
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

Now all we need to do is to create the folder and start up the rsync server:

sudo mkdir -p /var/www/pub
sudo service rsync start

In order to test the server, you can just drop any files into the /var/www/pub directory and then download them using:

rsync -r [hostname/IP address]::public/ .


e.g. rsync -r 192.168.1.10::public/ .

This will copy across all of the files from /var/www/public into your current directory. Note that if you leave out the dot at the end, it will merely display the list of files under /var/www/pub. Another thing to note is that by default the Rsync server uses TCP port 873 to communicate with the rsync client, so you may have to open up this port on your firewall if it is blocked.

Etckeeper on Ubuntu

There's a package in the Ubuntu repositories called 'etckeeper', which is a brilliant little tool for tracking changes to your configuration stored under the /etc directory.

The way it works is that it puts the whole of the /etc directory under version control, using one of either Bazaar, Mercurial, Git or Darcs. Then, whenever changes are made, either directly by modifying a file, or indirectly by running something like "apt-get update" it makes a note of who made the changes and what the differences in the files were. It then becomes easy to roll back configuration changes, find out who made the change and what the specific change to the configuration file was.

There's a good writeup on the basic usage on the Ubuntu Server Guide page.

Installing 'etckeeper' is as easy as:

sudo apt-get install etckeeper

It will use Bazaar as the VCS by default and will commit the first revision on installation.

Then, let's say that we wanted to install Apache, we would do this using "apt-get" as per usual, but during the install there'd be an extra section dealing with the commit by etckeeper:


$ sudo apt-get install apache2
[sudo] password for srdan:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libcap2
Suggested packages:
apache2-doc apache2-suexec apache2-suexec-custom
The following NEW packages will be installed:
apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libcap2
0 upgraded, 10 newly installed, 0 to remove and 3 not upgraded.
Need to get 3,248 kB of archives.
After this operation, 11.7 MB of additional disk space will be used.
Do you want to continue [Y/n]? Y
Get:1 http://archive.ubuntu.com/ubuntu/ oneiric/main libcap2 amd64 1:2.21-2 [12.2 kB]
Get:2 http://archive.ubuntu.com/ubuntu/ oneiric/main libapr1 amd64 1.4.5-1 [88.8 kB]
...
...
...
ldconfig deferred processing now taking place
Committing to: /etc/
modified .etckeeper
added apache2
added apache2/apache2.conf
added apache2/conf.d
added apache2/envvars
added apache2/httpd.conf
added apache2/magic
added apache2/mods-available
added apache2/mods-enabled
added apache2/ports.conf
...
added rc6.d/K09apache2
added ufw/applications.d/apache2.2-common
Committed revision 6.


So, we can see that as well as installing Apache, apt-get has committed the changes to the /etc/ directory as well. If you want to see what those changes are, you can using the "bzr log" command as below:


$ sudo bzr log /etc/apache2/httpd.conf
------------------------------------------------------------
revno: 6
committer: srdan
branch nick: etckeeper1 /etc repository
timestamp: Thu 2011-12-29 12:05:11 +1300
message:
committing changes in /etc after apt run

Package changes:
+apache2 2.2.20-1ubuntu1.1
+apache2-mpm-worker 2.2.20-1ubuntu1.1
+apache2-utils 2.2.20-1ubuntu1.1
+apache2.2-bin 2.2.20-1ubuntu1.1
+apache2.2-common 2.2.20-1ubuntu1.1
+libapr1 1.4.5-1
+libaprutil1 1.3.12+dfsg-2
+libaprutil1-dbd-sqlite3 1.3.12+dfsg-2
+libaprutil1-ldap 1.3.12+dfsg-2
+libcap2 1:2.21-2


So from the above output we can see who made the last change to this file, we can see that it was made as a part of an "apt" run and we can see the list of other packages that were installed as a part of this run.

When you make a change to any files under /etc directly, the changes aren't committed straight away, but are rather committed daily (probably by a cron job). To see whether there are any files which have been modified but not committed, use the "bzr status" command:


$ sudo bzr status /etc
modified:
apache2/sites-available/default


And to commit the change manually, use the "etckeeper commit" command:


$ sudo etckeeper commit "Changed the email address of the default webmaster"
Committing to: /etc/
modified apache2/sites-available/default
Committed revision 7.


Hopefully this has been a good introduction to etckeeper and it's use. While it won't stop people breaking things due to bad configuration, at least it can be helpful for preserving a working config and quickly determining what changes were made, by who and for what reason.

LAMP stack with Ubuntu JeOS

JeOS stands for "Just enough operating system" and referrs to an operating system with the bare minimum software installed to run the application required. Ubuntu's server edition makes it easy to install a JeOS minimal system.

To install a minimal LAMP (Linux, Apache, MySQL, PHP) stack on top of an Ubuntu JeOS instance, simply boot off of an Ubuntu server CD and then when it comes to the install screen, highlight the "Install Ubuntu Server" and then press F4 to bring up the "Modes" menu. From this menu select either "Install a minimal system" or "Install a minimal virtual machine", depending on whether you are installing a VM or a physical server.


From there, carry on with the rest of the install process and when it comes time for the software selection, check the "LAMP Stack" option. During the install process you'll be asked for a MySQL root password.

After this is done, you'll end up with an OS that takes up only about 650MB, compared to the regular Ubuntu server option which takes up 2GB. The main advantage of the smaller size is that the server or VM doesn't run any unnecessary services or expend any extra resources in keeping superfluous software updated. In addition, if it is a VM, it makes it much easier to transport and turns it into more of an "appliance"

VirtualBox - Increase screen resolution of guest

In order to get a decent resolution in your VirtualBox (Ubuntu) guest, you need to install the virtualbox guest X11 package:

sudo apt-get install virtualbox-ose-guest-x11



Tomcat6 on Ubuntu

In this post I'm going to talk about getting Tomcat version 6 up and running on Ubuntu. I'm going to be using Ubuntu 10.10. Installing Tomcat is quite simple, due to it being in the repositories. Simply run:

sudo apt-get install tomcat6 tomcat6-admin tomcat6-examples tomcat6-docs

And that's it :) You now have a working install of Tomcat version 6 on your machine. In order to see it working point your browser to http://localhost:8080/. You'll see the Apache "It Works!" message as well as a message about where everything is and links to the docs, examples, manager and host-manager applications.

Now, in order to get access to the manager and host-manager applications you're going to have to add a user. To do this, modify the file at /etc/tomcat6/tomcat-users.xml and add the following section:

<role rolename="admin"/>
<role rolename="manager"/>
<user username="srdan" password="password" roles="admin,manager"/>

After editing the file and restarting the Tomcat server we should be able to log into the manager application by pointing our browser at http://localhost:8080/manager/html and we should be able to log into the host-manager by going to http://localhost:8080/host-manager/html. The manager will allow us to deploy, undeploy, start and stop our applications and the host-manager allows us to declare, remove, start and stop our virtual hosts.

NOTE: From Tomcat 6.0.3 onwards, there is no "manager" role that is recognised by the manager application, rather it has been split up into four roles for security purposes:

manager-gui - allows access to the HTML GUI and the status pages
manager-script - allows access to the text interface and the status pages
manager-jmx - allows access to the JMX proxy and the status pages
manager-status - allows access to the status pages only

To test out the deployment, we're going to create a very simple web application. First, create a new directory and change into it:

mkdir HelloWorld
cd HelloWorld

Next we're going to create a servlet with the following code:

import java.io.*;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;

public class HelloWorld extends HttpServlet {


public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
response.setContentType("text/html");
PrintWriter out = response.getWriter();

out.println("<html>");
out.println("<head>");

out.println("<title>Hello World!</title>");
out.println("</head>");
out.println("<body>");

out.println("<h1>Hello World!</h1>);

out.println("</body>");
out.println("</html>");
}
}
And put it into a file called HelloWorld.java. After this we need to create a WEB-INF and WEB-INF/classes directory. Once we've done this, we compile the above code with:

javac -classpath "/usr/share/tomcat6/lib/*" HelloWorld.java

This should result in a HelloWorld.class file in the same directory. We need to move this file to the WEB-INF/classes directory:

mv HelloWorld.class WEB-INF/classes/

Now we need to add a web.xml file, which is going to tell Tomcat about our application. For our purposes we need to create a file with the following contents:

<?xml version="1.0" encoding="ISO-8859-1"?>

<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">

<description>
Simple Hello World Servlet.
</description>
<display-name>Simple Hello World Servlet</display-name>

<servlet>
<servlet-name>HelloWorld</servlet-name>
<servlet-class>HelloWorld</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>HelloWorld</servlet-name>
<url-pattern>/HelloWorld</url-pattern>
</servlet-mapping>
</web-app>

Once that's done, make sure that the file is in the WEB-INF directory, and we can go ahead and create our war file with:

jar -cvf hello.war .

Now we should be able to go back to the manager application at http://localhost:8080/manager/html, go to the Deploy section, select our war file, upload it and have it deploy straight away.

Assuming everything went fine, we can now go to http://localhost:8080/hello/HelloWorld and see our servlet in action, printing out the famous "Hello World!" message in HTML.

That concludes this post. Hopefully now you know how to install Tomcat on Ubuntu, manage it using the manager and host-manager applications, as well as how to create and deploy a very simple web application to the server.

Changing JRE's in Ubuntu

If you're working with Java on Ubuntu, you have two main choices as to which runtime to use. The open source OpenJDK and the official Sun runtime. Luckily, due to the alternatives system in Ubuntu (probably inherited from Debian) you can install both of these and switch between them as you see fit.

To see your currently installed JRE's run the command:

$ update-java-alternatives -l

and you should see some output like:

java-6-openjdk 1061 /usr/lib/jvm/java-6-openjdk
java-6-sun 63 /usr/lib/jvm/java-6-sun

This assumes that you've installed both of the runtimes. If you've only got one or the other installed, you will only see one line in the above output.

Now that we know which runtimes are installed, it would also be nice to be able to see which one we are using at the moment. Run the command:

$ java -version

If you see the ouput:

java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1~10.04.1)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)

Your are running the OpenJDK runtime. If you see:

java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)

You are running the Sun runtime. As an aside, the OpenJDK version is generally always a few versions behind the official Sun version.

So now that we know which runtimes we have installed and which one we are using at the moment, we can finally switch between them, using the update-java-alternatives command.

To switch to Sun's JRE:

sudo update-java-alternatives --set java-6-sun

To switch to OpenJDK:

sudo update-java-alternatives --set java-6-openjdk

To learn about the OpenJDK project and how it differs from the Sun Java runtime, you can have a look at:

http://openjdk.java.net/
http://en.wikipedia.org/wiki/OpenJDK

The 'update-java-alternatives' command has a lot more options allowing you to choose to switch just the JRE (and keep the JDK the same), or only switch out the browser plugin. To see all of the alternative options, have a look at the man page 'man update-java-alternatives'.

Installing Redmine + SVN on Ubuntu 10.04

I've got a new workstation pc. It's not much (no hardware supported virtualization, onboard graphics) but it does the job. I've decided to take it and basically turn it into the ultimate development machine. That is to say, start right from the beginning and do things properly. The last sentence probably means a lot of different things to a lot of different programmers, so what does it mean to me? Well basically:

* Ability to access the machine remotely (Dynamic DNS + SSH/Web access)
* A Version Control System (Subversion, thank you very much)
* Project management/issue tracking software
* Development web server + database (Tomcat & MySQL)

So, today I will be writing how to get Subversion + Redmine up and running on Ubuntu 10.04. Partly to help someone out there that might be having the same problem and partly because I'll probably be having to install it again in the future and having a written record might save some time :-)

I've installed both Subversion and Redmine from the repositories and generally used the sample configuration which came with the packages, with a little bit of tweaking of course.

Installing Subversion is as easy as running:

sudo apt-get install subversion libapache2-svn

Next, I mostly followed the subversion install instructions found on the ubuntu wiki page, setting up a private repository. The wiki writeup is excellent, telling you how to get up and running quickly, whether you use subversion over HTTP/HTTPS/SVN/SVN+SSH.

Redmine was a little more tricky to install, although still a lot easier than I remember it being before it was in the repositories. Start by running:

sudo apt-get install redmine redmine-mysql

If you don't use MySQL, use one of the alternates (redmine-sqlite, redmine-pgsql). The sample configuration for Apache can be found at /usr/share/doc/redmine/examples/. I used the 'apache2-alias.conf' file by pasting it into /etc/apache2/sites-available/redmine and running:

sudo a2ensite redmine
sudo /etc/init.d/apache2 restart

After this I got an error saying that one of the apache directives was in the wrong place:

... waiting Syntax error on line 14 of /etc/apache2/sites-enabled/redmine:
SocketPath cannot occur within section


I then tried moving the directive to the /etc/apache2/mods-available/fcgid.conf, however this ended up giving me a different set of errors in the apache error log:

[Sun Feb 13 16:23:33 2011] [error] (13)Permission denied: mod_fcgid: couldn't bind unix domain socket /var/run/redmine/sockets/default/6126.28
[Sun Feb 13 16:23:33 2011] [warn] (13)Permission denied: mod_fcgid: spawn process /usr/share/redmine/public/dispatch.fcgi error

Finally, I commented this line out from both sections and was able to get redmine going without fcgi (Later I realised that I had installed passenger and it was running and that this was probably the reason that fcgi kept throwing the errors.

So, then you can navigate to http://localhost/redmine/ and configure the admin user (default username/pass is admin:admin) and you should have a working Redmine install.

Ubuntu 10.04, impressions after a week

As a part of doing a general clean up of my laptop, last Wednesday/Thursday I carefully backed up all of the data accross all of the 5 different partitions (all of which had different operating systems, including three versions of Ubuntu), before deleting the partitions and installing Ubuntu 10.04 LTS as the only OS on my laptop. The process was quite time consuming, having to log into each and every OS and copy anything useful over to the "Data" partition. Ultimately, the effort did pay off in the end though and I'm left a feeling of great satisfaction at the fact that I didn't lose any data and managed to clean up my system.

As for the OS of choice for this rebirth of my laptop, it had to be Ubuntu 10.04. I had used Unetbootin recently to trial both the latest versions of Ubunut and Kubuntu, but ultimately settled on Ubuntu due to its better support of the wireless chipset in my laptop. The wireless is something I've constantly been struggling with over the years, with it behaving strangely, such as reducing power until the connection dies at which point it tries to reconnect. With 10.04 these problems seem largely solved (cross fingers), although my connection still sometimes drops out when I'm on the other side of the house. The other neat little thing is that this newer version of Ubuntu supports the status LED of the wireless chipset.

The few problems which I ran into after the install were to do with third party software and importing items from the previous installs. Google Gears failed to install, and after tracking the problem down it seems that Google is only providing a 64-bit version of the software, meaning that 32-bit users are out of luck. The fix was simply to install the 'xul-ext-gears' package from the repositories, but this could have been better integrated into the interface. The second problem I faced was to do with trying to import all of my old Tomboy notes. I have yet to find a script/program to do this for me and just pointing the synchronisation to the directory containing the old notes doesn't seem to work. It looks like I'm going to have to write some code for this one, not that I mind too much :)

Overall, 10.04 is a very impressive release. I'm more impressed with the stability improvements than any of the new features (social media anyone?).

Eclipse Opening Tutorial woes

I've been looking around for a good all around development environment for a while now and today I've decided to give Eclipse a go. I've installed the eclipse package from the Ubuntu 9.04 repositories and everything seemed to be working well. I was a little disappointed with the fact that the Ubuntu repositories only had version 3.2 of Eclipse, when the main stable release was at 3.5, but this didn't bother me too much.I opened up Eclipse and decided to have a go at the first two tutorials, hoping to get an idea of what this IDE's all about.

The first tutorial goes through the process of creating a Java project with the ambitious aim of printing out 'Hello World!' to the command line. Going through the tutorial was quite straight forward, teaching you how to create a new jave project and add classes to the project.

The second tutorial was the same as the first, except that instead of printing 'Hello World!' to the command line you would print it to a window. In order to do this you use the SWT by downloading the Eclipse project and making it a part of your projects runtime. The tutorial went fine until it came to running the code and I ran into the following error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: no swt-gtk-3550 or swt-gtk in swt.library.path, java.library.path or the jar file
at org.eclipse.swt.internal.Library.loadLibrary(Unknown Source)
at org.eclipse.swt.internal.Library.loadLibrary(Unknown Source)
at org.eclipse.swt.internal.C.(Unknown Source)
...

This is extremely disappointing. I mean, this is meant to be the second tutorial that the user works through, it should be bullet proof. The entry barrier to getting up and running on Eclipse should not be so high, that you have to go out of your way to learn the intricacies of SWT. So far, not a very good start for Eclipse.

Update: Got the second tutorial working. Instead of adding the SWT project off of the official site, I added the 'swt-gtk.jar' library found under /usr/lib/java on my Ubuntu install. Still not too happy about having to go to these kinds of lengths to get the first couple of examples up and running, but at least I'm learning :)

Folding@Home

I've recently joined the folding@home project under the name srkiNZ84. Folding@Home is a hugely distributed computing project with the purpose of doing protein folding (anyone remember seti@home?). I've joined TeamUbuntu and started crunching my way through those work units.

At first what got me intrigued with this project was just trying to benchmark my computer(s), but now what really keeps me interested in it is the competition. You see, with this project you get to keep statistics on how many work units you've completed, in what kind of time frame etc... and every member of the project is ranked. This gives rise to cool personal and team statistics screens like this one from extremeoverclocking or this one from xtreme cpu. Personally I think the competition is one of the main reasons that the project has so many followers.

Right now I'm ranked 455th in my team and 171,330th overall. This isn't too bad considering I've only started. However, the only real way I'm going to get into the top 10% or so of the rankings is by making use of a high performance GPU client. These are versions of the folding@home software that run on your graphics card and make use of it's parallel processing capabilities to complete work units in a much shorter space of time. ATI were the first to produce a client for their line of graphics cards, but were soon followed by nvidia, which blew them away with their much better CUDA performance. I'm not even sure that there is a GPU client for Linux (at least not on the official folding@home website), but I haven't been through the whole recruitment post on ubuntu forums, so I'm hoping there's some clues on there.

Signature Errors on Ubuntu repositories

On all of the Ubuntu installs I've ever had, I almost always eventually run into the following error when running the 'apt-get update' command or hitting the 'Reload' button in Synaptic.


Error:
W: GPG error: http://nz.archive.ubuntu.com hardy-updates Release: The following signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key
W: You may want to run apt-get update to correct these problems


From what I understand the problem comes from apt downloading an incomplete/corrupt signature file, meaning that the signature then doesn't match that of the packages. The fix at the moment is to simply delete the incomplete/corrupt signature file and download it again. To do this run the following command:

sudo rm /var/lib/apt/lists/partial/*

Afterwards running apt-get update doesn't return the same error.