Archive for August, 2011

HOWTO Build a base LXC container in Ubuntu 11.04

August 19, 2011

I occasionally build LXC containers to run self contained servers or for testing/demos. Each time I do I end up looking up some references to remind me the details and make sure I don’t forget any steps, so I decided I would put it down in writing here to make it easier to find and follow next time…

The references I generally use are:

Create cgroup
LXC uses cgroups to manage container resources. If cgroups aren’t already mounted, then add them as follows:

  •   Add the following line to /etc/fstab:
none /dev/cgroup cgroup defaults 0 0
  •   Then create the directory and mount:

mkdir /dev/cgroup
mount /dev/cgroup

NOTE: It can be mounted anywhere. I like to put them in /dev/cgroup, other people put them in /cgroup or /var/cgroup.

Install Required Packages
Install some required packages to use LXC:

  • lxc – the LXC tools
  • bridge-utils – for creating linux bridges
  • debootstrap – for creating base systems, required by the template scripts

aptitude install lxc bridge-utils debootstrap # or apt-get install

Networking Setup
There are two types of networking setup I use, depending on how I am going to use the container. The first and most common setup is the bridged configuration. This is required (I believe) if the container is to receive ethernet frames (like DHCPREQUEST packets for a *pxe server). It is also required if the container is to use your network’s DHCP server. This requires creating a bridge device and plugging eth0 (or the primary interface) into it, which can affect the system (ie: if NetworkManager is being used, it can no longer manage the connection; if there are services bound to eth0, they will need to be bound to br0 instead).

The second setup is NAT. A bridge interface is created which is only connected to the container’s network interface. The host then routes packets to/from the container.

(a) Bridged Setup
Add a bridge interface to /etc/network/interfaces file, and set eth0 to manual mode:

vi /etc/network/interfaces

# find the section talking about your physical interface, it's normally
# eth0 or eth1
auto eth0
iface eth0 inet manual # change from 'dhcp' to 'manual'</code>

# add this section
auto br0
iface br0 inet dhcp
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

Then restart networking:

/etc/init.d/networking restart

(b) NAT Setup
Add a bridge interface to /etc/network/interfaces for the container to use. Here I give it a static IP and configure the host to route for it:

vi /etc/network/interfaces

# LXC bridge
auto br-lxc
iface br-lxc inet static

    post-up echo 1 &gt; /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
    pre-down echo 0 &gt; /proc/sys/net/ipv4/ip_forward
    pre-down iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

    bridge_ports none
    bridge_stp off

Then restart networking:

/etc/init.d/networking restart

Setup initial LXC configuration file
This file will be used by the lxc-start command to create the configuration. Replace br-lxc with br0 if using bridged network setup.

vi network.conf = veth = up = br-lxc # (or br0 for bridged) 
# optional stuff (from lxc HOWTO) = {a1:b2:c3:d4:e5:f6} # As appropiate (line only needed if you wish to dhcp later) = {} # (Use if you wish to dhcp later) = eth0 # could likely be whatever you want 
#lxc.mount = {/path/to/fstab/for/CONTAINER_NAME} 
#lxc.rootfs = {/path/to/rootfs/of/CONTAINER_NAME} 

Create the container
Now invoke lxc-create to create the container. I HIGHLY recommend using a template script for this, as it uses debootstrap to download a base system and then fixes the init scripts so that it can be booted in a container (it may also install an ssh server/user, depending on the script). If you don’t use a script, you have to fix these things manually and it can be a pain. The LXC package in Ubuntu 11.04 comes with several template scripts in /usr/lib/lxc/templates, if you need to setup a container on an older Ubuntu (like 10.04) you can copy one of these scripts off an 11.04 install. There are also other scripts on the web, but I have not tried them.

lxc-create -n name -f network.conf -t lucid

Container network configuration
For static NAT configuration, the container’s IP needs to be set before it is booted up (it defaults to dhcp, so it should just work for a bridged network setup).

vi /var/lib/lxc/name/rootfs/etc/network/interfaces

auto lo 
iface lo inet loopback 

auto eth0 
iface eth0 inet static 

Start it!
lxc-start -n name -d #( -d for daemonize)

SSH to it!
The script auto created ssh server, and root’s password is root:

ssh root@

That’s it, a base, (hopefully) working LXC container.
* NOTE: If the nameserver isn’t put in /etc/resolv.conf automatically (you can’t resolve anything), then restart networking INSIDE the container with:

/etc/init.d/networking restart

Some optional steps to make the container a little more usable:

  • Create a user:

  • adduser user
    adduser user GROUP

  • Install some packages:

  • apt-get install aptitude
    aptitude install man bash-completion vim python-software-properties command-not-found sudo


    • python-software-properties – add-apt-repository script
    • command-not-found – suggests which package contains typed missing command