Michael Crosby


the lost packages of docker

Docker is a large project that touches many different layers of the system. Everything from a REST api to low level filesystem and execution calls. Docker is also open source so where are all the awesome packages that make docker dock? Why do the docker developers not contribute back reusable packages that I can import in my own projects? Well we do and they are located in pkg.

While working on docker we try to keep the core as small as possible and break code out into general purpose packages that can be used in many different projects. The hard part about this is that we cannot make separate git repos for the packages because of packaging reasons. Right now it is still hard for distro maintainers to package and build Go applications because of the static nature of Go. Everyone is doing the best that they can right now so while we work on the best way for distros to handle packaging our dependencies we place resuable Go packages in a /pkg folder in the docker repository. So what does docker have to offer?


netlink

We have a pure go implementation of netlink that you can use in your projects. You can use this package to create veth interfaces, bridges, set ip, mtu, and other settings on network interfaces, move network interfaces into different linux namespaces, bla bla bla. This is the same code that handles creating the veth pairs and assigning an ip to each of your docker containers.

Lets use this package to create bridge and assign an ip:

package main

import (
        "github.com/dotcloud/docker/pkg/netlink"
        "log"
        "net"
)

func main() {
        // create a new bridge
        if err := netlink.CreateBridge("mydocker0", false); err != nil {
                log.Fatal(err)
        }
        // get the bridge
        bridge, err := net.InterfaceByName("mydocker0")
        if err != nil {
                log.Fatal(err)
        }

        ip, ipNet, err := net.ParseCIDR("10.0.41.1/16")
        if err != nil {
                log.Fatal(err)
        }

        // add an ip to the bridge
        if err := netlink.NetworkLinkAddIp(bridge, ip, ipNet); err != nil {
                log.Fatal(err)
        }
        // bring the interface up
        if err := netlink.NetworkLinkUp(bridge); err != nil {
                log.Fatal(err)
        }
}

We can run the app above then run ip a to see our new interface:

2: mydocker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether ae:1f:cb:af:f2:54 brd ff:ff:ff:ff:ff:ff
    inet 10.0.41.1/16 scope global mydocker0
    valid_lft forever preferred_lft forever
    inet6 fe80::ac1f:cbff:feaf:f254/64 scope link
    valid_lft forever preferred_lft forever

The netlink package is not feature complete but hopefully not that you know it exists we can all work together on it to make it better.

user

Linux user and group functions that do not rely on external libs.

graphdb

Graphdb is a minimal graph database built ontop of sqlite for use with names and how they relate with one another. It implements a small subset of what most graph databases have but provides a simple interface for expressing relationships between nodes. This is used for container naming and linking inside docker.

package main

import (
        "github.com/dotcloud/docker/pkg/graphdb"
        "log"
)

func main() {
        db, err := graphdb.NewSqliteConn("/root/links.db")
        if err != nil {
                log.Fatal(err)
        }
        defer db.Close()

        if _, err := db.Set("/parent", "momma"); err != nil {
                log.Fatal(err)
        }

        child, err := db.Set("/child", "matt")
        if err != nil {
                log.Fatal(err)
        }
        otherKid, err := db.Set("/otherKid", "koye")
        if err != nil {
            log.Fatal(err)
        }

        // set entity with id of matt to child of the parent
        if _, err := db.Set("/parent/child", child.ID()); err != nil {
                log.Fatal(err)
        }
        if _, err := db.Set("/parent/kidbyaothername", otherKid.ID()); err != nil {
                log.Fatal(err)
        }

        // get all children for the key /parent
        for _, e := range db.List("/parent", -1) {
                log.Println(e.ID())
        }
}

:::bash
~| go run graph.go
2014/04/14 16:16:19 matt
2014/04/14 16:16:19 koye

listenbuffer

Listenbuffer allows you to immediately start listening on a unix, tcp, or udp socket but wait for your application to signal that it is ready before accepting connections.

This is used in docker so that you can start the daemon and a client request from an init script can be sent before the daemon has completed loading. This ensures that the init script does not receive an error and your server can start accepting connections when it is ready. It uses the kernel backlog for buffering the connections and is a super simple package that offers a lot of value if you have a long boot.

mount

Mount is a package to help you work with mounting lots of things. It allows options to be passed fstab style and also offers mount table parsing functions so that your applications can easily find mountpoints.

// do a readonly bind mount
if err := mount.Mount("/myfile", "/myotherfile", "none", "bind,ro"); err != nil {
    log.Fatal(err)
}
defer mount.Unmount("/myotherfile")

term

Term handles terminal sizing, raw mode, and other settings for interacting with a unix terminal on linux, darwin, and bsd.

cgroups

This package is currently in heavy development right now but it provides the cgroup functionality for libcontainer via the raw cgroup filesystem api and also the systemd api where supported.

libcontainer

Pkg is also where libcontainer resides. It is the default execution driver for running docker containers. It sets up namespaces, networking, mounts, drops caps, and manages the container processes. You provide a root filesystem and a configuration on how libcontainer is supposed to execute a container and it does the rest. It allows spawning new containers or attaching to an existing container.


So in the end why did I write this post? I wanted to bring some visibility to these packages. While other projects are able to have separate git repositories for the different dependencies that they use and develop, docker has to keep these in it's main repository causing many people to not know that they exist. So if some of these packages seem interesting to you, feel free to use them in your projects, contribute, write docs, tests, and make them awesome.

If you know of an interesting area currently in the docker code base that is not in a reusable package, submit a PR to make it one. We are always looking for ways to make the docker core smaller and our work useful to others.

comments powered by Disqus