Skip to content

LXD

Install

Install the package from AUR. 

yay -S lxd

Add custom mapping for uid and gid. 

echo "root:1000000:65536" | sudo tee -a /etc/subuid /etc/subgid
echo "lxd:1000000:65536" | sudo tee -a /etc/subuid /etc/subgid
If you do not you may get Error: Failed container creation: Create container: Create LXC container: LXD doesn't have a uid/gid allocation. In this mode, only privileged containers are supported when launching a container.

Start and enable the service. 

systemctl enable lxd
systemctl start lxd

Add yourself to the lxd group. 

sudo usermod -a -G lxd $USER

Initialize LXD

To create the default network, storage pool and profile. Run as root. 

lxd init

Basic commands

Create a container. 

lxc init images:alpine/3.6/i386 container

Create a new profile. 

lxc profile create name

Create a new profile from another profile. 

lxc profile copy default name

List containers. 

lxc list

List all local images. 

lxc image list

List all available remote images. lxc remote list. 

lxc image list images:
lxc image list ubuntu:
lxc image list server:

Auto update remote images. 

lxc config set images.auto_update_cached true

Create a conatiner 

lxc init ubuntu:16.04 container

Launch a container from a remote image. 

lxc launch images:ubuntu/xenial/amd64 xenial1

Launch an ephemeral container. 

lxc launch --ephemeral ubuntu:18.04 ubuntu

Information about a container. 

lxc info container

Access the container. 

lxc exec container /bin/bash
lxc exec container -- /bin/bash

Stop and start 

lxc start container
lxc stop container
lxc restart container

Delete container. 

lxc delete container

Show storage pools. 

lxc storage list

Create a custom image based on a local container 

lxc publish container --alias customimage

Auto update remote images. 

lxc config set images.auto_update_cached true

Push a file to a container. 

lxc file push /path/in/host container/path/in/container

Pull a file from a container. 

lxc file pull server:container/path/to/file /destination/path

Backup container 

lxc export container container.$(date +'%Y-%m-%d').tar.gz
lxc import container.2020-04-20.tar.gz

Backup image Source 

lxc snapshot container backup
lxc publish container/backup --alias container-backup
lxc image export container-backup .
lxc image delete container-backup

lxc image import backup.tar.gz --alias container-backup
lxc launch container-backup new-container
lxc image delete container-backup

Start on boot 

lxc config set {container} boot.autostart {true|false}
lxc config set {container} boot.autostart.priority integer
lxc config set {container} boot.autostart.delay integer

Network

lxc network create br0
lxc network show br0
lxc network edit br0

Static IP. 

lxc config device set <container> eth0 ipv4.address 192.168.1.100

Bridge br0: 

lxc profile copy default lan
lxc profile edit lan
---
...
    devices:
      eth0:
        name: eth0
        nictype: bridged
        parent: br0
        type: nic
...

Cloud-init

By default only works with images pulled from https://cloud-images.ubuntu.com/releases/ (Ubuntu).

Log available at /var/log/cloud-init.log inside the container.

Create a new profile with ssh keys

Create or clone a new profile. 

lxc profile copy default ssh

Edit the profile with lxc profile edit ssh. Get the key with cat ~/.ssh/lxd.pub. You can create the keys with ssh-keygen -t rsa -b 8128 -C "Keys for LXD". 

config:
  user.user-data: |
    #cloud-config
    users:
      - name: yu
        ssh-authorized-keys:
          - ssh-rsa [...] == Keys for LXD
        sudo: ['ALL=(ALL) NOPASSWD:ALL']
        groups: sudo
        shell: /bin/bash

Launch a container. 

lxc launch --profile ssh ubuntu:18.04 ubuntu

And connect to it over ssh. 

ssh "$(lxc list | grep ubuntu | awk '{print $6}')" -i ~/.ssh/lxd -l yu

Container with custom config file instead of profile

Create a config file config.yml. 

#cloud-config
users:
  - name: yu
    ssh-authorized-keys:
      - ssh-rsa ... == Keys for LXD
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    groups: sudo
    shell: /bin/bash

And create a new container. 

lxc launch ubuntu:18.04 ubuntu --config user.user-data="$(cat lxd_config.yml)"

And connect to it over ssh. 

ssh "$(lxc list | grep ubuntu | awk '{print $6}')" -i ~/.ssh/lxd -l yu

Distrobuilder

Any distro downloaded from https://images.linuxcontainers.org will NOT include cloud-init, so you have to create your own image; you can do it manually (do not) or you can use distrobuilder.

Install dependencies. 

sudo pacman -S go debootstrap rsync gnupg squashfs-tools

Build distrobuilder from source. 

go get -d -v github.com/lxc/distrobuilder
cd $HOME/go/src/github.com/lxc/distrobuilder
make
cd
go get gives an error but it works.

Create a folder. 

mkdir /tmp/centos && cd /tmp/centos

Create a template. Examples available at $HOME/go/src/github.com/lxc/distrobuilder/doc/examples/.

Under files: add the following. 

- name: meta-data.lxd
  path: /var/lib/cloud/seed/nocloud-net/meta-data
  generator: template
  content: |-
    instance-id: {{ container.name }}
    local-hostname: {{ container.name }}
    {{ config_get("user.meta-data", "") }}

- name: network-config.lxd
  path: /var/lib/cloud/seed/nocloud-net/network-config
  generator: template
  content: |-
    {% if config_get("user.network-config", "") == "" %}version: 1
    config:
      - type: physical
        name: eth0
        subnets:
          - type: {% if config_get("user.network_mode", "") == "link-local" %}manual{% else %}dhcp{% endif %}
            control: auto{% else %}{{ config_get("user.network-config", "") }}{% endif %}

- name: user-data.lxd
  path: /var/lib/cloud/seed/nocloud-net/user-data
  generator: template
  content: |-
    {{ config_get("user.user-data", properties.default) }}
 # properties:
 #   default: |
 #     #cloud-config
 #     {}

- name: vendor-data.lxd
  path: /var/lib/cloud/seed/nocloud-net/vendor-data
  generator: template
  content: |-
    {{ config_get("user.vendor-data", properties.default) }}
 # properties:
 #   default: |
 #     #cloud-config
 #     {}

And under - packages: add the following. 

- sudo
- cloud-init
- python
- openssh-server

And create the image for LXD. 

sudo $HOME/go/bin/distrobuilder build-lxd template.yaml

Import the image. 

lxc image import lxd.tar.xz rootfs.squashfs --alias image_name

Create a config file config.yml. 

#cloud-config
users:
  - name: yu
    ssh-authorized-keys:
      - ssh-rsa ... Keys for LXD
    sudo: ['%wheel ALL=(ALL) NOPASSWD: ALL']
    groups: wheel
    shell: /bin/bash

Create a container with a custom cloud-init file config.yml. 

lxc launch image_name container_name --config=user.user-data="$(cat config.yml)"

And log into the container. 

ssh "$(lxc list | grep centos | awk '{print $6}')" -i ~/.ssh/lxd -l yu

Everything I tested seems to work.

Ansible

Example Playbook for Ubuntu. 

---
- hosts: localhost
  connection: local
  tasks:
    - name: Create Ubuntu container in LXD
      lxd_container:
        name: ubuntu
        architecture: x86_64
        state: started
        ephemeral: yes
        config:
          user.network_mode: "link-local"
          raw.idmap: gid 1005 1000 ## Group id 1001 in the container will be 1005 in the host
          boot.autostart: "true"
          limits.cpu: "4"
          limits.cpu.allowance: "50%"
          user.user-data: |
            #cloud-config
            users:
              - name: yu
                ssh-authorized-keys:
                  - "{{ lookup('file', '/path/to/id_rsa.pub') }}"
                sudo: ['ALL=(ALL) NOPASSWD:ALL']
                groups: sudo
                shell: /bin/bash
        devices:
          folder:
            source: "/path/in/host"
            path: "/path/in/container"
            type: disk
          eth0:
            ipv4.address: 10.10.10.57
            name: eth0
            nictype: bridged
            parent: lxdbr0
            type: nic
        source:
          type: image
          mode: pull
          server: https://cloud-images.ubuntu.com/releases/
          protocol: simplestreams
          alias: 18.04
        profiles: ["default"]

Deploy with ansible-playbook playbook.yml.

Ansible modules used for LXD:

Remote instances

Configure the remote server to be accessible. 

lxc config set core.https_address 192.168.1.90:4567
lxc config set core.trust_password something-secure

Add a new remote. 

lxc remote add <name> <IP|FQDN|URL>
lxc remote add server 192.168.1.90:4567 --accept-certificate

List remote servers. 

lxc remote list

List remote containers. 

lxc list server:

Launch a remote container 

lxc launch ubuntu:18.04 server:container

nftables

sudo nft add rule inet filter INPUT iifname "lxdbr0" tcp dport { 53, 67, 68 \} counter accept
sudo nft add rule inet filter INPUT iifname "lxdbr0" udp dport { 53, 67, 68 \} counter accept
sudo nft add rule inet filter FORWARD iifname "lxdbr0" counter accept
sudo nft add rule inet filter FORWARD oifname "lxdbr0" counter accept
sudo nft add rule nat postrouting ip saddr 10.131.131.0/24 oif "enp3s0" snat 10.0.5.12
sudo nft add rule arp filter INPUT iifname "lxdbr0" counter accept

Xorg

Add the display to the container. 

lxc config device add firefox X11 disk source=/tmp/.X11-unix path=/tmp/.X11-unix

On the host allow anyone to connect to the local X11 instance. 

sudo xhost +
sudo xhost inet:192.168.1.100

PulseAudio. Source

Add to /etc/pulse/default.pa 

load-module module-native-protocol-unix auth-anonymous=1 socket=/tmp/pulse-socket

Reload PulseAudio. 

pulseaudio -k
pulseaudio --start

You may need this too. 

rm -rf /tmp/pulse* ~/.pulse* ~/.config/pulse

Configure the container. 

lxc config device add container X11 disk source=/tmp/pulse-socket path=/tmp/pulse-socket