Having a guest account might be useful on a home computer, but it’s generally not what I want enabled on a notebook.
To disable the default Guest account you need to edit lightdm.conf and add a line (allow-guest=false).
sudo vim /etc/lightdm/lightdm.conf
[SeatDefaults]
greeter-session=unity-greeter
user-session=ubuntu
allow-guest=false
Tested in Ubuntu 12.04 LTS Precise Pangolin & Ubuntu 11.10 Oneiric Ocelot
