Michael Crosby


Go Dynamic Plugins

For me, my favorite feature of Go is the static binary. I love compiling once and just dropping the binary into a container running the 2mb busybox image and having my app up and running. No pip, no dlls, no problem. However, this does come with a small price. If you want to dynamically extend your Go app at runtime with plugins your SOL.

If you look at the internals of the database/sql package in Go you can see how they handle different database driver support but that is still resolved at build time. A few people are handling plugins for Go via RPC, shelling out to another process, or what not, but all of those solutions still have their own issues to work through. If you want to keep your plugins in the same process and have a natural feel then I think this solution is worth a look.

otto

There is a really cool open source project called otto that implements a javascript runtime in pure Go. This allows you to embed js in your Go apps very easily. Yes, yes, I know it's javascript but it's not that bad when used in moderation.

Anyways, otto allows you to create a js runtime in your Go application and execute js, allow js to call go functions that were added to the runtime, and pass data to and from a javascript function. In the example below I will be making a simple http server that returns a string depending if you are authorized or not. The interesting part is that we will be passing the http.Request to a javascript function to do runtime checks. This allows us to change the code checking the request without stopping and rebuilding our Go code.

The app

Lets start by looking at our main func that will handle the web requests.

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        if checkRequest(r) {
            fmt.Fprintf(w, "Welcome in\n")
        } else {
            w.WriteHeader(http.StatusUnauthorized)
            fmt.Fprintf(w, "Your not allowed!\n")
        }
    })

    if err := http.ListenAndServe(":8080", nil); err != nil {
        panic(err)
    }
}

This is very standard and simple. In the handler we make a call to checkRequest to tell us if the request is valid or not. Now lets look at the checkRequest func.

func checkRequest(r *http.Request) bool {
    runtime := loadPluginRuntime("plugins.js")

    // If we don't have a runtime all requests are accepted
    if runtime == nil {
        return true
    }
    v, err := runtime.ToValue(*r)
    if err != nil {
        log.Fatal(err)
    }

    // By convention we will require plugins have a set name
    result, err := runtime.Call("checkRequest", nil, v)
    if err != nil {
        log.Fatal(err)
    }
    // If the js function did not return a bool error out
    // because the plugin is invalid
    out, err := result.ToBoolean()
    if err != nil {
        log.Fatalf("\"checkRequest\" must return a boolean. Got %s", err)
    }
    return out
}

This is where we interface with the js runtime. We load the runtime in another func that I will show next. If we don't have any plugins we will receive a nil runtime so we just return true. If we do have a runtime then we need to convert our Go type ( the request ) into a js Value/Object so that we can pass it to the plugin method. Right now otto has a issue with pointer values as js Objects so I dereference the request when passing to the ToValue method.

Next we call the js function checkRequest and receive it's return value. In order to call js functions we need to know the name of the function in our Go code. I don't think that this is an issue because when people want to implement plugins for your application they will need to stick to certain naming conventions and interfaces in order for the two applications to interact together.

Now we need to take the return value from the js function and convert it back into a Go type that our program can understand. We expect the checkRequest function to return a bool so we use the ToBoolean method to do the conversion. If we get an err, it is probably because the js function did not return the correct type or nothing at all.

Let's quickly look at the loadPluginRuntime func.

func loadPluginRuntime(name string) *otto.Otto {
    f, err := os.Open(name)
    if err != nil {
        if os.IsNotExist(err) {
            return nil
        }
        log.Fatal(err)
    }
    defer f.Close()
    buff := bytes.NewBuffer(nil)

    if _, err := buff.ReadFrom(f); err != nil {
        log.Fatal(err)
    }
    runtime := otto.New()
    // Load the plugin file into the runtime before we
    // return it for use
    if _, err := runtime.Run(buff.String()); err != nil {
        log.Fatal(err)
    }
    return runtime
}

In the load func we load a file with all the js functions and set them in the runtime. Because this is an example I am run reading the file on each request. In a prod system you will want to do something fancy but for this example it is fine. Anyways, we load the file and then create the new runtime. We use the Run method and pass the contents of the file as a string so that our functions are loaded.

The Plugin

Lets checkout the plugin.js file to see what the javascript implementation is to check the request.

function checkRequest(r) {
    // Only allow GET requests to the API
    return r.Method == "GET";
}

And that is it. We access the Method on the http.Request and check to make sure it is a GET request and not allow anything else. You can change the plugin.js file and watch the responses change as you toggle between true and false.

Otto is still new so I hope people will check out the project and contribute bug fixes and improvements so that otto can be a viable option for dynamic plugins in Go. Of course you can also embed lua or v8 into your Go binary instead. There are many options and this is only one.

I am also interested in embedding lua into Go. I have notice a few large open source projects like Nginx and Redis embedding lua so why not give it a try. Thanks and let me know what you think in the comments below.

Full App

package main

import (
    "bytes"
    "fmt"
    "github.com/robertkrimen/otto"
    "log"
    "net/http"
    "os"
)

// Load the plugin file.  If the file does not exist
// then return a nil runtime
func loadPluginRuntime(name string) *otto.Otto {
    f, err := os.Open(name)
    if err != nil {
        if os.IsNotExist(err) {
            return nil
        }
        log.Fatal(err)
    }
    defer f.Close()
    buff := bytes.NewBuffer(nil)

    if _, err := buff.ReadFrom(f); err != nil {
        log.Fatal(err)
    }
    runtime := otto.New()
    // Load the plugin file into the runtime before we
    // return it for use
    if _, err := runtime.Run(buff.String()); err != nil {
        log.Fatal(err)
    }
    return runtime
}

func checkRequest(r *http.Request) bool {
    runtime := loadPluginRuntime("plugins.js")

    // If we don't have a runtime all requests are accepted
    if runtime == nil {
        return true
    }
    v, err := runtime.ToValue(*r)
    if err != nil {
        log.Fatal(err)
    }

    // By convention we will require plugins have a set name
    result, err := runtime.Call("checkRequest", nil, v)
    if err != nil {
        log.Fatal(err)
    }
    // If the js function did not return a bool error out
    // because the plugin is invalid
    out, err := result.ToBoolean()
    if err != nil {
        log.Fatalf("\"checkRequest\" must return a boolean. Got %s", err)
    }
    return out
}

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        if checkRequest(r) {
            fmt.Fprintf(w, "Welcome in\n")
        } else {
            w.WriteHeader(http.StatusUnauthorized)
            fmt.Fprintf(w, "Your not allowed!\n")
        }
    })

    if err := http.ListenAndServe(":8080", nil); err != nil {
        panic(err)
    }
}
comments powered by Disqus