Ubuntu - The following packages have been kept back

If you've used Ubuntu for long enough, you'll find that eventually you'll run into a problem when upgrading the installed packages. When running apt-get from the command line, the problem manifests itself as the following:

$ sudo apt-get upgrade
[sudo] password for srdan:
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages have been kept back:
  linux-headers-server linux-image-server linux-server
0 upgraded, 0 newly installed, 0 to remove and 3 not upgrade

The short answer is that you should be able to upgrade by running the "apt-get dist-upgrade" command:

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree      
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  linux-headers-3.2.0-27 linux-headers-3.2.0-27-generic linux-image-3.2.0-27-generic
The following packages will be upgraded:
  linux-headers-server linux-image-server linux-server
3 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 51.2 MB of archives.
After this operation, 217 MB of additional disk space will be used.
Do you want to continue [Y/n]?

The long answer comes from the man page of the "apt-get" command. In particular, if you look at the description of the "upgrade" argument, two sentences stick out:

"under no circumstances are currently installed packages removed, or packages not already installed retrieved and installed."

"New versions of currently installed packages that cannot be upgraded without changing the install status of another package will be left at their current version."

Because you can't upgrade the "linux-image-server" (a.k.a. the kernel) without upgrading the headers as well (technically you can, but it can lead to serious problems) it won't let you upgrade them using the "upgrade" command. Either that, or the "new" linux image package requires that a "new" linux headers package be installed, violating the requirement that packages not already installed not be installed.

The reason that the "dis-upgrade" command works, where the "upgrade" command does not is that "dist-upgrade" takes into account dependencies between packages. Also from the man page:

"dist-upgrade ... intelligently handles changing dependencies with new versions of packages; apt-get has a 'smart' conflict resolution system, and it will attempt to upgrade the most important packages at the expense of less important ones if necessary."

This does beg the question though as to why not just use "dist-upgrade" all the time or incorporate the conflict resolution system into the "upgrade" command?

I suspect the answer has something to do with how each "edition" of a distribution is defined. i.e. Ubuntu 12.04 will ship with version 3.1.13 of the "at" package and all other packages that are a part of this "edition" should work with that version. Having many different packages depend on specific versions of other packages could end up in a situation where it becomes very difficult to upgrade any individual package.

svn:ignore property, STS and Subversive

Working with a combination of Springsource Tool Suite, Subversion and the Subversive plugin, one of the things that really bugs me is the issues I keep on getting as a result of messed up source control of the "target" directory.

This directory typically holds all of the compiled classes as well as a stacktrace log and a few other things. In general it never really needs to be put under version control.

One of the things I keep forgetting to do, that frustrates the hell out of me is to set the svn:ignore property as soon as the repository is created. You see, without this property set, the target directory is submitted (added) to version control and once it's in there, there's no (easy) way of getting it out, as the Subversion documentation says:

"...Once an object is under Subversion's control, the ignore pattern mechanisms no longer apply to it..."

And even if you set the "svn:ignore" property on the directory, it still probably won't work properly. Instead committing all of the class files and so forth that you couldn't give two craps about.

All this is well and good, but could be written off as a minor problem, if it weren't for the fact that when you try to revert your changes, Subversion could very well pick up "conflicts" between the class files, and in fact can put you in a situation where you're struggling to get a functional checkout of the repository, simply because of the "target" directory.

The problem seems to be with the nature of the files being stored in the directory being constantly deleted/recreated etc... which seems to screw up Subversion. At the moment there doesn't seem to be a fix for the issue i.e. every time I see the problem, I just manually manage the conflicting files.

Linux Kernel Swappiness

There is a tendency of the Linux kernel to use memory as file system cache. This generally improves performance and is considered to be a "good thing". However, one thing that the kernel also occasionally does is take the memory allocated to running processes and swap them to disk, in order to use that memory for file systems cache. Now, this can and does result in processes becoming slower, especially if they've been running (sitting in memory) for a while, but haven't been actively used.

Luckily, there is a way in which you can define this behaviour and it's called the kernel "swappiness" value. The value has a range of 0 to 100, with zero roughly meaning that process memory will never get swapped out for the sake of disk caching and a value of 100 means that process memory is very aggressively swapped out, in favour of disk caching. A more in depth explanation of how the kernel manages swappiness, can be found here.

By default, this value is set to 60, which is configured more for server throughput, rather than desktop responsiveness.On my desktops I usually set the value to 10, which seems to be a good fit for desktop responsiveness.

The way to set it on Ubuntu is:

$ sudo sysctl vm.swappiness=10

and to make the changes permanent, just add the following line to /etc/sysctl.conf:

vm.swappiness=10

Apache - Enabling the info module

If you've ever wanted to have something like the phpinfo page for Apache, that showed all of the modules that were enabled as well as configuration and compilation settings, just make use of the mod_info module:

This can be useful for getting a definitive answer on which configuration files are being loaded as well as what modules are being loaded and what they're configured to.

However, as with phpinfo, you have to keep security in mind when using it as a lot of this information can be used by potential attackers. The recommended way of securing it is just modifying the configuration file (/etc/apache2/mods-available/info.conf) to deny access from all but trusted IP addresses:

<ifmodule mod_info.c>

<location /server-info>
    SetHandler server-info
    Order deny,allow
    Deny from all
    Allow from localhost ip6-localhost
#    Allow from .example.com
</location>

</ifmodule>

By default, the only access allowed is from localhost.

Humble Indie Bundle - Bastion

This month being my birthday, I've bought and downloaded the Humble Indie Bundle, which I bought as a present for myself. After seeing the lineup of games for this bundle, I couldn't say no and apparently neither could a lot of other people, the bundle having raked in over $5 million and selling just under 600k bundles.

To be honest, I haven't played any kind of computer games for a while now, so it's been a challenge trying to find the time to play. Out of all the games in the bundle, I started with Bastion, which attracted me to it with its artistic visuals.

The game is good, really good. The only criticism I can make is that it's a little on the difficult side. This  likely has to do with me not having played any video games for a while, however.

Grails many-to-many deletes

This is an interesting problem I've had recently when trying to delete a Child object that's a member of a many-to-many relationship.

So, the classes are as follows:

class List {

String name
String description

static hasMany = [tasks:Task]
}

class Task {

String name
String description

static belongsTo = List
static hasMany = [lists:List]
}

Pretty simple right? But try calling the delete method of the "Task" controller and you'll soon run into problems. The problem is that when the child object is deleted, it's references in the parent aren't deleted. If you're not sure what this means, just keep an eye on the database contents and you'll see how after deleting the child object you end up with a bunch of foreign key pointers that are invalid now.

The solution is to add some code like the following in the delete method right?

taskInstance.lists.each{
  it.removeFromTasks(taskInstance)
}

However, if you try this you'll get a "java.util.ConcurrentModificationException". Which is telling you that Java doesn't allow you to change the contents of a collection (array, list, set etc...) while iterating over that same collection.

The solution which I found is kind of kludgey, but does the job:

def tmp = []
tmp.addAll(taskInstance.lists)
tmp.each{
it.removeFromTasks(taskInstance)
}

This somehow doesn't "feel" right. It feels like it shouldn't be this hard to delete a child object and have the parent object be updated at the same time.

Multi-boot with MultiSystem

Recently I read about a tool called MultiSystem, which you can get from here. The tool is a live Linux CD which allows you to create multi-bootable USB drives. Why would this be useful? I found a use for it immediately in that it allowed me to boot my desktop PC, which doesn't have a CD/DVD drive, off of the USB and try out the latest version of Ubuntu, make sure that everything works and install the OS. 

How does it work? It's really simple, you just get all of the live CD ISO's that you want to put onto a FAT32 formatted spare USB drive, start up MultiSystem and drag and drop the .iso files onto the interface. Admittedly the interface is somewhat randomly designed, but once you drop the .iso files the rest it taken care of. The next step is to configure your PC to boot off of the USB drive, which may require a change in the BIOS settings. Once that's configured, and you've booted off the USB you should get a GRUB-like menu and have a choice of all of the live CD's which you've configured USB drive with. 

It's a great tool for trying out different flavours of Linux/BSD based systems and there's even an option to configure the USB to be Mac bootable. So, look out for another post down the road about setting up a dual boot system on my Macbook.