HOWTO Build a base LXC container in Ubuntu 11.04

I occasionally build LXC containers to run self contained servers or for testing/demos. Each time I do I end up looking up some references to remind me the details and make sure I don’t forget any steps, so I decided I would put it down in writing here to make it easier to find and follow next time…

The references I generally use are:
http://www.stgraber.org/2011/05/04/state-of-lxc-in-ubuntu-natty/
http://nigel.mcnie.name/blog/a-five-minute-guide-to-linux-containers-for-debian
http://lxc.teegra.net/

Create cgroup
LXC uses cgroups to manage container resources. If cgroups aren’t already mounted, then add them as follows:

  •   Add the following line to /etc/fstab:
none /dev/cgroup cgroup defaults 0 0
  •   Then create the directory and mount:

mkdir /dev/cgroup
mount /dev/cgroup

NOTE: It can be mounted anywhere. I like to put them in /dev/cgroup, other people put them in /cgroup or /var/cgroup.

Install Required Packages
Install some required packages to use LXC:

  • lxc – the LXC tools
  • bridge-utils – for creating linux bridges
  • debootstrap – for creating base systems, required by the template scripts

aptitude install lxc bridge-utils debootstrap # or apt-get install

Networking Setup
There are two types of networking setup I use, depending on how I am going to use the container. The first and most common setup is the bridged configuration. This is required (I believe) if the container is to receive ethernet frames (like DHCPREQUEST packets for a *pxe server). It is also required if the container is to use your network’s DHCP server. This requires creating a bridge device and plugging eth0 (or the primary interface) into it, which can affect the system (ie: if NetworkManager is being used, it can no longer manage the connection; if there are services bound to eth0, they will need to be bound to br0 instead).

The second setup is NAT. A bridge interface is created which is only connected to the container’s network interface. The host then routes packets to/from the container.

(a) Bridged Setup
Add a bridge interface to /etc/network/interfaces file, and set eth0 to manual mode:

vi /etc/network/interfaces

...
# find the section talking about your physical interface, it's normally
# eth0 or eth1
auto eth0
iface eth0 inet manual # change from 'dhcp' to 'manual'</code>

# add this section
auto br0
iface br0 inet dhcp
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

Then restart networking:

/etc/init.d/networking restart

(b) NAT Setup
Add a bridge interface to /etc/network/interfaces for the container to use. Here I give it a static IP and configure the host to route for it:

vi /etc/network/interfaces

...
# LXC bridge
auto br-lxc
iface br-lxc inet static
    address 192.168.254.1
    netmask 255.255.255.0</code>

    post-up echo 1 &gt; /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
    pre-down echo 0 &gt; /proc/sys/net/ipv4/ip_forward
    pre-down iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

    bridge_ports none
    bridge_stp off

Then restart networking:

/etc/init.d/networking restart

Setup initial LXC configuration file
This file will be used by the lxc-start command to create the configuration. Replace br-lxc with br0 if using bridged network setup.

vi network.conf

lxc.network.type = veth 
lxc.network.flags = up 
lxc.network.link = br-lxc # (or br0 for bridged) 
# optional stuff (from lxc HOWTO) 
#lxc.network.hwaddr = {a1:b2:c3:d4:e5:f6} # As appropiate (line only needed if you wish to dhcp later) 
#lxc.network.ipv4 = {192.168.10.2/24} # (Use 0.0.0.0 if you wish to dhcp later) 
#lxc.network.name = eth0 # could likely be whatever you want 
#lxc.mount = {/path/to/fstab/for/CONTAINER_NAME} 
#lxc.rootfs = {/path/to/rootfs/of/CONTAINER_NAME} 

Create the container
Now invoke lxc-create to create the container. I HIGHLY recommend using a template script for this, as it uses debootstrap to download a base system and then fixes the init scripts so that it can be booted in a container (it may also install an ssh server/user, depending on the script). If you don’t use a script, you have to fix these things manually and it can be a pain. The LXC package in Ubuntu 11.04 comes with several template scripts in /usr/lib/lxc/templates, if you need to setup a container on an older Ubuntu (like 10.04) you can copy one of these scripts off an 11.04 install. There are also other scripts on the web, but I have not tried them.

lxc-create -n name -f network.conf -t lucid

Container network configuration
For static NAT configuration, the container’s IP needs to be set before it is booted up (it defaults to dhcp, so it should just work for a bridged network setup).

vi /var/lib/lxc/name/rootfs/etc/network/interfaces

auto lo 
iface lo inet loopback 

auto eth0 
iface eth0 inet static 
    address 192.168.254.2 
    netmask 255.255.255.0 
    gateway 192.168.254.1 
    dns-nameservers 8.8.8.8 

Start it!
lxc-start -n name -d #( -d for daemonize)

SSH to it!
The script auto created ssh server, and root’s password is root:

ssh root@192.168.254.2

That’s it, a base, (hopefully) working LXC container.
* NOTE: If the nameserver isn’t put in /etc/resolv.conf automatically (you can’t resolve anything), then restart networking INSIDE the container with:

/etc/init.d/networking restart

Optional
Some optional steps to make the container a little more usable:

  • Create a user:

  • adduser user
    adduser user GROUP
    ...

  • Install some packages:

  • apt-get install aptitude
    aptitude install man bash-completion vim python-software-properties command-not-found sudo

    Where:

    • python-software-properties – add-apt-repository script
    • command-not-found – suggests which package contains typed missing command
Advertisements

5 Responses to “HOWTO Build a base LXC container in Ubuntu 11.04”

  1. Ray Says:

    Your 1st and 3rd reference links are dead….

  2. Ray Says:

    The one major confusing thing about some of these LXC tutorials is the fact that I sometimes don’t know where to make changes….in the host or in the containter?

    • bj0z Says:

      The first one still works for me, the third one is dead but this is 3 years old and development on LXC was moving very fast back then so I would expect all of this to be different now. For most cases you can just use docker (www.docker.com), which makes building an lxc container easy.

  3. Davide Says:

    Thanks for the excellent tutorial 🙂

    I’d like to ask why in the bridged setup it is necessary to disable dhcp on the host interface eth0 ?

    I tried both “dhcp” and “manual” and the only thing I figured out is that “dhcp” doesn’t work if br0 is set up. (Indeed I have an active dhcp server wired on the other end of the eth0 card)

    • bj0z Says:

      It was because br0 was to get the dhcp lease, not eth0. This post is very old, though, and LXC has changed so much since then. Nowadays it’s very rare to use LXC directly, since Docker makes it very very easy to spin up containers without having to do all this manual configuration.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: