Grails many-to-many deletes

This is an interesting problem I've had recently when trying to delete a Child object that's a member of a many-to-many relationship.

So, the classes are as follows:

class List {

String name
String description

static hasMany = [tasks:Task]
}

class Task {

String name
String description

static belongsTo = List
static hasMany = [lists:List]
}

Pretty simple right? But try calling the delete method of the "Task" controller and you'll soon run into problems. The problem is that when the child object is deleted, it's references in the parent aren't deleted. If you're not sure what this means, just keep an eye on the database contents and you'll see how after deleting the child object you end up with a bunch of foreign key pointers that are invalid now.

The solution is to add some code like the following in the delete method right?

taskInstance.lists.each{
  it.removeFromTasks(taskInstance)
}

However, if you try this you'll get a "java.util.ConcurrentModificationException". Which is telling you that Java doesn't allow you to change the contents of a collection (array, list, set etc...) while iterating over that same collection.

The solution which I found is kind of kludgey, but does the job:

def tmp = []
tmp.addAll(taskInstance.lists)
tmp.each{
it.removeFromTasks(taskInstance)
}

This somehow doesn't "feel" right. It feels like it shouldn't be this hard to delete a child object and have the parent object be updated at the same time.

Multi-boot with MultiSystem

Recently I read about a tool called MultiSystem, which you can get from here. The tool is a live Linux CD which allows you to create multi-bootable USB drives. Why would this be useful? I found a use for it immediately in that it allowed me to boot my desktop PC, which doesn't have a CD/DVD drive, off of the USB and try out the latest version of Ubuntu, make sure that everything works and install the OS. 

How does it work? It's really simple, you just get all of the live CD ISO's that you want to put onto a FAT32 formatted spare USB drive, start up MultiSystem and drag and drop the .iso files onto the interface. Admittedly the interface is somewhat randomly designed, but once you drop the .iso files the rest it taken care of. The next step is to configure your PC to boot off of the USB drive, which may require a change in the BIOS settings. Once that's configured, and you've booted off the USB you should get a GRUB-like menu and have a choice of all of the live CD's which you've configured USB drive with. 

It's a great tool for trying out different flavours of Linux/BSD based systems and there's even an option to configure the USB to be Mac bootable. So, look out for another post down the road about setting up a dual boot system on my Macbook. 

Internal LAN on LXC

When I wrote my previous post about setting up LXC, one of the things I found was that when installing the lxc package, it went ahead and created an "lxcbr0" interface.

It turns out that this interface is actually an "internal network" which you can connect your VM's to if you want them talking to each other directly, as opposed to any network which the host is also on.

To setup my VM's I just added another interface and connected it to the "lxcbr0" bridge, by adding the following lines to the configuration:

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 4a:49:43:49:79:ef

and then configuring the interface in the "interfaces" file:

auto eth1
iface eth1 inet static
    address 10.0.3.2
    netmask 255.255.255.0

Then I did the same to another VM and they were able to talk to each other.

Btrfs snapshots and LXC

In my previous post I talked about LXC, which is a light-weight virtualization technology for Linux. One thing which LXC lacks is the ability to make snapshots. As the VM is running as a regular process in RAM, at the moment it's not possible to just make a copy of the ram file, as it is in VMware etc...

So, in order to work around this, we're only going to be taking snapshots of VM's when they're shutdown and we're going to be making use of the Btrfs snapshot functionality.

First of all, we need to create a Btrfs filesystem. I assume that you have a spare drive or partition which you can use for this:

$ sudo mkfs.btrfs -L btrfs-test /dev/sda8 

WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL 
WARNING! - see http://btrfs.wiki.kernel.org before using 

fs created label btrfs-test on /dev/sda8 nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00GB 
Btrfs Btrfs v0.19

Now we can mount the filesystem:

$ sudo mount /dev/sda8 /lxc

Before we set about creating our VM's, we're going to create some "subvolumes", which we'll be able to snapshot. We're going to use this feature of Btrfs to handle the snapshotting of our VM's.

$ sudo btrfs subvolume create /lxc/vm0
Create subvolume '/lxc/vm0'

Now that we've done this, we can go ahead and create a VM using the template scripts, and configure it, as described in my previous post.

$ sudo /usr/lib/lxc/templates/lxc-ubuntu -p /lxc/vm0 ...

Now that we've created a brand new VM, we're going to create a snapshot of it's "clean" state, so that we can roll back to it, should something go wrong. We do this by creating a snapshot called "clean" of the /lxc/vm0 subvolume.

$ cd /lxc/
$ sudo btrfs subvolume snapshot vm0 vm0-clean
Create a snapshot of 'vm0' in './vm0-clean'
$ sudo btrfs subvolume list /lxc
ID 256 top level 5 path vm0
ID 258 top level 5 path vm0-clean

Now we can start the VM:

$ sudo lxc-start -n vm0 -f /lxc/vm0/config

And install/configure our software:

$ sudo apt-get install apache2 postgresql

Now, suppose that we're humming along nicely, but then realise that we've made a mistake and installed apache2 instead of tomcat6 and postgresql instead of mysql-server and want to start over. All we would need to do is to delete the "vm0" subvolume and rename the "vm0-clean" directory to "vm0":

$ sudo btrfs subvolume delete /lxc/vm0
Delete subvolume '/lxc/vm0'
$ sudo btrfs subvolume list /lxc
ID 258 top level 5 path vm0-clean
$ sudo mv vm0-clean vm0
$ sudo btrfs subvolume list /lxc
ID 258 top level 5 path vm0

Notice how the ID of the snapshot doesn't change, even though we've renamed it.

Now, when we start up the VM, we can see that the "apache2" and "postgresql" packages haven't been installed yet, because we've rolled our VM back to the original snapshot that we've taken.

Now, perhaps the scenario with installing the wrong packages isn't that realistic (it would probably be easier to just remove the packages instead of rolling back the snapshots), however, this was just chosen to demonstrate the capabilities of the technology and you can probably imagine a scenario where a snapshot would be more useful. e.g. testing out a software upgrade, which you're concerned might break some functionality.

NOTE: Another way of using snapshots is to mount them directly, by passing the "-o subvol=..." option at mount time, as described at: http://btrfs.ipv5.de/index.php?title=SysadminGuide#Managing_snapshots

LXC on Ubuntu

This post talks about how to setup LXC (Linux Containers) on Ubuntu 12.04. LXC is an operating system-level virtualization technology, which allow you to run multiple virtual machines on one host.

There are quite a few limitations to this type of virtualization, when compared with the type of full, emulator style virtualization that VMware, VirtualBox etc... use. One of the main ones is that you'll be unable to run different operating systems on the virtualization host. i.e. we can only run Linux VM's on our host.

The big advantage is performance. Because the host doesn't have to bother with all of the code which does virtual hardware emulation, the virtual machines run a lot faster in general.

So, to setup LXC, we first need to install it:

sudo apt-get install lxc lxctl uuid btrfs-tools

The 'lxctl', 'uuid' and 'btrfs-tools' packages aren't really needed, but come recommended, and it doesn't hurt to install them.

Now, at this point I did a reboot, which may not be necessary, and afterwards checked the LXC configuration using:

lxc-checkconfig

You should find that all of the different settings are set to "enabled".

Now that we've got LXC installed, we can go ahead and start creating our first VM. Luckily, the 'lxc' package comes with a set of template scripts, which make setting up a VM easy. These scripts are located under '/usr/lib/lxc/templates' and to create our first VM we run:

sudo /usr/lib/lxc/templates/lxc-ubuntu -p /lxc/vm0/
 
Where '/lxc/vm0' is the path to the VM. Note that you will have to create this directory as it doesn't exist by default.

Once you run that script, you'll see a lot of output from the VM getting created and initialized. From the output and from looking at the template script, it looks like the template takes all of the currently installed packages, copies them over to the VM filesystem and configures them.

Once, this is done, we need to configure the networking, which I found to be the trickiest part of the setup. Because the guest VM is using the same hardware as the host, it has the ability to use the network interface attached to the host. Now, obviously you don't want both the host and the guest using the same interface, as it will lead to IP address and MAC address conflicts. So, the guest should have a distinct MAC and/or IP address.

There are several ways to configure the networking for the guest VM's and there are example configuration files of the many ways that they can be configured under file:///usr/share/doc/lxc/examples/ (note that you can enter this location into your web browser and it should load). For our purposes, we're going to go with the lxc-veth.conf file, which sets up a virtual network interface connected to a network bridge which we have to create.

So, firstly, we need to create a network bridge, which is done by adding the following lines to the /etc/network/interfaces file:

auto br0
iface br0 inet dhcp
bridge_ports eth0

This will create the bridge that we're going to connect the virtual network interface of the VM to. In order to enable it, restart the 'networking' service:

sudo service networking restart

Note that we have to create a bridge, even if we've only got one physical interface to connect to it.

Then add the relevant lines from the example config to the VM configuration file, under /lxc/vm0/config:

...
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 4a:49:43:49:79:bf
lxc.network.ipv4 = 10.0.0.0/24
...

Note that the IP network should match that of your host interface, otherwise you might have some problems getting an IP through DHCP. Also, note that the MAC address is just taken from the example file an was probably randomly generated.

That should complete the configuration. We are now free to start up the VM using the 'lxc-start' command:

sudo lxc-start -n vm0 -f /lxc/vm0/config

This should start up the VM and bring up the console on the terminal screen. For the default Ubuntu template, you can log in using ubuntu/ubuntu username/password pair.

Once you've logged in, you can confirm that the VM has a different IP address to that of the host and start configuring it. 

Funnily enough, just as I've finished writing up this post, I stumbled upon a Launchpad blog post talking about how they're using LXC to speed up their testing: http://blog.launchpad.net/general/parallelising-the-unparallelisable

Programming and premature optimization

I recently watched a lecture by Jonathan Blow, which can be found here. In it he talks about the kinds of challenges that you have to go through being an independent game developer. The main point to take away is that you really have to take a holistic view of the whole "business" and not just the "beauty of computer science" bits which programmers tend to focus on. Once of the main points he covers is to avoid at all costs the sin of premature optimization. The argument goes that if you spend too much time looking for the "perfect" way of doing something, you're likely going to over optimize it and have both wasted your own time on a problem, where you won't get the return on effort that you needed and ended up with code which can't be easily reused.

One of the ways that he gets this message across is to take a look at one of his games, Braid, which it turns out is approximately 90,000 lines of code. He makes the point that the industry average for "lines of code" per programmer, per year is 3,250. At this rate, it would take some ridiculous amount of time, like 28 years to produce a game. So, in order to launch a game by yourself you have to be "super productive" and spending time on anything which won't lead to the game launching (like implementing complex algorithms, which only optimize the game by 0.5%) will without a doubt result in failure.

It's a compelling point and one that can be seen at work in the real world. It's often not the "best" code that ends up being successful, but rather code that ships. The trade-off I guess is that this "ship first" mentality means that a lot of the time, the user ends up with poor quality code, buggy and resistant to upgrades.

Converting param values in Grails

There are some cases in Grails where you want to compare values as Integers instead of the default String objects. There are two ways to do this. The first is to use either the "parseInt" or "valueOf" methods which are original Java methods. The other way to skin this cat is to call the "int()" method on the params object:

if(Integer.parseInt(params.user.id) != user.id){
...
}
if(params.int('user.id') != user.id){
...
}

There are other methods which the params object has to easily convert HTTP post values to well known data types, such as:

param.short(...)
param.byte(...)
param.long(...)
param.double(...)
param.boolean(...)

Another nice feature is that these methods accept a second optional parameter, which is the default that the value is set to, in the case that there is an error in the conversion:

def price = prams.float("seventy", 0.0)

This behaviour is documented in the Grails documentation here. Although, there doesn't seem to be a comprehensive list of the methods available at the time of writing.