Michael Crosby

Favorite Tech

2013-04-28T00:00:00Z

I am always working with different projects and technologies so here are some of my favorite projects and tools so far this year.

Vagrant and Puppet

If you are not using virtual machines for development and testing then you are a fool. There is no reason to run code on your local machine when you can reproduce your production configurations in a virtual machine. Vagrant allows you to quickly spin up new boxes and Puppet manages provisioning your boxes and production servers easy and consistently.

When I want to test a new database or mess around with a new package I always spin up a new box with Vagrant and do my testing in a contained environment so that I don't kill my server or development machines.

Vagrant Puppet

Editors

I have been using Sublime Text 2 a lot over the past few months. However I have noticed that when working with a few large projects and a few different windows open with these projects it really slows down. Not just switching back and forth between windows and files but typing; that really bothers me. So I am back in Vim for my editing needs. I don't use gVim but Vim from the terminal because that is where I spend most of my time.

If you take the time to learn and configure Vim then it will be your best friend. You just have to invest in the editor, something that others do not have the time or patience for.

Vim

Redis

Redis is an awesome key value store that keeps the entire dataset in memory. I have been using redis as a work queue because of the awesome PUSH and POP commands for working with lists. You can see how powerful redis is just by viewing the commands. Redis is like a developers swiss army knife, it has many different tools to solve many different problems. It is not only efficient as a data store but also efficient to program for.

You can checkout my redis queue package for Go on github.

Redis

Backbone.js and Bootstrap

Even though I would rather be writing low level code or building apis I still have to write html and js for web projects. Backbone.js makes this simple and structured. Backbone.js is a client side javascript mvc framework for interfacing with a backend RESTful api that adds structure to javascript.

Twitter Bootstrap is a html, js, css framework for building web apps. It is so awesome and helps you produce great looking pages with minimal design skills. It includes many javascript and html components to use that make working on a web project simple while keeping a high level of quality.

This takes me to an aside, I believe the way to build web applications now is with static html and a client side mvc framework with a RESTful api on the backend. Using a client side mvc framework creates a clear separation between a view layer and your backend code. You cannot hack anything into view data, session, or whatever you crappy developers do. On the client side you only have the data and interactions provided by the backend api. Backbone.js allows you to split development between frontend developers and the backend developers working on the domain model and services.

This also makes sites very fast. Instead of rendering templates on the server side, all the templates are rendered client side with simple api calls to the server for CRUD operations. This separation also helps you decouple your backend implementation from your frontend view layer. Because the frontend connects to the backend via a JSON api you could change your entire backend's framework, implementing language, or scale horizontally without the tight coupling that you get when rendering templates server side.

This also makes deployments easy. If you have client side changes you can just push the new html and javascript to the server and you are done. No need to compile your backend code for frontend changes.

Backbone.js Bootstrap

CoffeeScript

CoffeeScript is a simple and elegant language that is compiled into javascript. We all know that javascript sucks and CoffeeScript aims to make it better. Here is some simple CoffeeScript with Backbone.js.

class FeedList extends Backbone.View
    el: '.feedlist'

    render: ->
        view = @
        feeds = new Feeds()

        feeds.fetch(
            success: (feeds) ->
                template = _.template($('#feedlist-template').html(), {feeds:feeds.models})
                view.$el.html(template))

Now lets see what this compiles down to:

FeedList = (function(_super) {
  __extends(FeedList, _super);

  function FeedList() {
    _ref7 = FeedList.__super__.constructor.apply(this, arguments);
    return _ref7;
  }

  FeedList.prototype.el = '.feedlist';

  FeedList.prototype.render = function() {
    var feeds, view;
    view = this;
    feeds = new Feeds();
    return feeds.fetch({
      success: function(feeds) {
        var template;
        template = _.template($('#feedlist-template').html(), {
          feeds: feeds.models
        });
        return view.$el.html(template);
      }
    });
  };

  return FeedList;

})(Backbone.View);

This is a huge difference, which one would you rather write? When you find a part of your job that is a pain, look for tools that will help. Web development is changing a lot and you (or the people in charge) are the only ones prohibiting you from performing your job efficiently and actually having fun. There tools are out to help you write better code, with fewer bugs, and enjoy what you are doing.

CoffeeScript;

Microsoft land

In the Microsoft world I have been using F#. It is really nice to look at problems from the functional perspective. I am super lazy so instead of writing documentation for an assembly I wrote a small class that will iterate over an entire assembly and create instances of any type (no matter if they have complex constructors) and extract documentation and other data from the types. I initially wrote it in C# and it was around 370+ lines of code. I decided to rewrite it in F# and dropped the line count to 85. It is very fast and efficient to code in.

Rant I think that is everything for now. Oh ya, everything on this list is open source and free to use. Fuck vendor lock and drinking the koolaid of using shitty tools because that is just what we use. Use the best tool for the job, shit tools produce shit product and is a pain to use.

Golang JSON RPC

2013-04-23T00:00:00Z

Go's rpc package has an example for the default rpc implementation but nothing for the jsonrpc package. Here is how to get it work. The rpc example uses some helpers built into the rpc package, I just looked at that code in Go's stdlib to figure out the proper way to get the jsonrpc package to work.

package main

import (
    "fmt"
    "log"
    "net"
    "net/rpc"
    "net/rpc/jsonrpc"
    "test/server"
)

func startServer() {
    arith := new(server.Arith)

    server := rpc.NewServer()
    server.Register(arith)

    server.HandleHTTP(rpc.DefaultRPCPath, rpc.DefaultDebugPath)

    l, e := net.Listen("tcp", ":8222")
    if e != nil {
        log.Fatal("listen error:", e)
    }

    for {
        conn, err := l.Accept()
        if err != nil {
            log.Fatal(err)
        }

        go server.ServeCodec(jsonrpc.NewServerCodec(conn))
    }
}

func main() {
    go startServer()

    conn, err := net.Dial("tcp", "localhost:8222")

    if err != nil {
        panic(err)
    }
    defer conn.Close()

    args := &server.Args{7, 8}
    var reply int

    c := jsonrpc.NewClient(conn)

    for i := 0; i < 1; i++ {

        err = c.Call("Arith.Multiply", args, &reply)
        if err != nil {
            log.Fatal("arith error:", err)
        }
        fmt.Printf("Arith: %d*%d=%d", args.A, args.B, reply)
    }
}

Go Lang

2013-04-12T00:00:00Z

If you don't know about Go it's a programming language for now. Most of the languages that we use everyday are old, 10+ years old. After so many years languages need to start to think about news ways to solve the same problems. Go is a new language that takes a different stance on how you write and deploy code. Go is a dynamically typed but statically compiled language. I have "played" with Go pre 1.0 a few times and at the 1.0 release but it never really stuck. I felt like I was writing to much code for what I wanted to accomplish. However, this last time working with Go 1.1beta I have started to do a few projects and packages with Go and I am really impressed with the language.

Abstractions

I write a lot of Python code and I still enjoy writing Python more than any other language so I am used to a fairly high level of abstraction from the language. Even working in C# with tools like LINQ you get trained to expect abstractions for working with collections. Also when used correctly, I really like having generics available in my language.

You miss these things with Go. Go is at a lower level of abstraction in many areas. Looping and working with arrays and slices ( variable sized collections ) is often back to the for i loop. Going from LINQ to anything less hurts. I find my self working with []byte more in Go than I have in any other language. The Go stdlib does provide you with some abstractions in the io packages but you still are working with byte arrays often.

Lets look at reading a file line by line in both Python and Go.

with open('names.txt') as f:
    for line in f.readlines():
        print line

Python is pretty straight forward. Very simple to remember and can be done with no imports from the stdlib. Now lets look at the Go example.

package main

import (
    "bufio"
    "fmt"
    "os"
)

func main() {
    file, err := os.Open("names.txt")
    if err != nil {
        panic(err)
    }

    defer file.Close()

    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        fmt.Println(scanner.Text())
    }
}

So the go example is a little more than just 3 lines of code. We have the package declaration, 3 imports from the stdlib, and the go error handling code in the main func. Even though Go has more code it is still simple and elegant. You do have to learn the stdlib to do simple tasks but once you do, you have a very powerful set of tools. The go example is every more amazing once you learn about implicit interface implementation.

No Classes

I like OOP and modeling classes and their interaction with each other to create a fluent and straight forward API. However, I also feel like OOP is taken too far. People will insist that you have to design a polymorphic interface, provide factories, and strategies when all you really need to a simple function. I believe in design patterns for solving problems in software but I also believe in simplicity, clarity, and elegance in the code that I write. For me, simple wins over "over engineering." Knowing when to design and when to bang out a function comes with experience.

With Go you have types and interfaces but no classes and inheritance. At the beginning this was hard for me. However, after working with packages, types, and interfaces in Go I do not miss classes. I feel like my code is simple and straight forward with only types and packages. It is OOP at it's purest, data with actions. You can still attach methods to your types and you have public and internal access via upper and lower case names. If a type and/or function begins with an upper case it is exported to be used outside of that package, if it is lower case it is internal to that package.

Interfaces in Go are also very modern. You can have a type and implement an interface without explicitly stating that you implement that interface. If we have an interface that has a

Write([]byte) int

then in Go, any type that has a method Write that takes a byte array and returns an int implements that interface, which is exactly what you want. With interfaces you are basically saying that you don't care about the type or other methods, all I care about is that whatever I have passed to me has a Write method that takes bytes and returns an int.

Why Go

So why am I writing a post on a language with no classes and missing certain abstractions that we are used to? Because their some home run features that make Go an amazing language to work with.

Concurrency

There are many articles about concurrency in Go with gorutines and channels so I am not going to say much here except that they are awesome.

Compiled binary

Deploying a python application takes automation. You need to to create a virtual environment, have a requirements file with all of your dependencies, and then finally copy your application code up to the server. There are many tools for deploying python code and it really is not that bad but it is also not simple and there are many places that things can go wrong.

With Go you have a compiled binary with all the dependencies compiled into the binary. You do not even have to have the Go tool chain installed on the computer that you are deploying your go application to run the code. You just drop the executable on the machine and your good to go, nothing else to do. This is amazing.

Also being compiled into native machine code makes go very fast.

Testing and Packages

Testing is simple and easy. Automating testing is simple and easy. Have you tried to automate C# unit tests? Oh, you don't own VS Pro with unit testing?

Installing dependencies is very easy with "go get". You install dependencies exactly the same way as you have your own packages.

Go Doc

All of Go's documentation is auto generated by the commends in the code. This includes packages in the stdlib and your own packages in your GOPATH. I have been doing a lot of work in an area with no Internet access to I really appreciate being able to run the package documentation and other general Go documentation on my local machine.

godoc -http=:8080

This provides me with all the documentation and examples available if I was connected online to the golang.org website.

Parting words

Hopefully this has peaked your interest in Go and maybe you can give it a try. I am going to start using Go as my main language for backend services and processes.

SecureString How To

2013-03-09T00:00:00Z

Working with the SecureString class in .NET is fairly easy. This class allows you to keep string data encrypted in memory. Maybe you load connection strings, passwords, or financial data into memory when your application starts or loads an object so the least you can do is use the SecureString class to keep this information encrypted while the data stays in memory.

Adding your data

Adding a string content to the class is very easy.

public static void Main(string[] args)
{
    var hushhush = "Something I need to keep on the down low";

    using (var secureString = new SecureString())
    {
        foreach (var chr in hushhush.ToCharArray())
        {
            secureString.AppendChar(chr);
        }
        secureString.MakeReadOnly();
    }
    Console.ReadLine();
}

All you need to do is convert your string into a char array and append each char to the secure string. After you have finished writing all data to the secure string you will want to call the MakeReadOnly method so that the instance becomes immutable.

Getting your data back out

Now this is where things get weird. It's nice not to have a convenient method on the SecureString class to get to your data for security reasons but you actually have to work with pointers and unmanaged resources to get to the data.

Because SecureString is a sealed class I made a little extension class that adds a GetString() method for reading the string data from the class.

static class SecureStringExtensions
{
    public static string GetString(
        this SecureString source)
    {
        string result = null;
        int length = source.Length;
        IntPtr pointer = IntPtr.Zero;
        char[] chars = new char[length];

        try
        {
            pointer = Marshal.SecureStringToBSTR(source);
            Marshal.Copy(pointer, chars, 0, length);

            result = string.Join("", chars);
        }
        finally
        {
            if (pointer != IntPtr.Zero)
            {
                Marshal.ZeroFreeBSTR(pointer);
            }
        }

        return result;
    }
}

The Marshal class actually has a BSTR method for use with the SecureString. Just remember when working with unmanaged resources you will need to manually release and free the pointers that you used.

Basically if you have not worked with Marshal and pointers in C# you will have to get the pointer to the char[] from the secure string. Then we use Marshal.Copy to marshal the *char[] to our managed char[] using the length of the string. A simple string.Join and we are home free. Just remember to zero out our pointer to release the resources.

F# Url Shortener

2013-01-27T00:00:00Z

Every year I try to learn a new language. If you are a one trick pony you are not only limiting your self in opportunities but you are also limiting your thinking to the one paradigm that you know.

So this year I wanted to learn something very different from what I am used to in the OO world ( Python, C#, Objective - C). I decided to learn a functional language this year called F#. I looked at a few different functional languages ( Scala, Erlang ) before choosing F#. I felt that it would be the most practical language for what I do but still provide me with the paradigm shift I was seeking. It does have OO parts being part of the .NET family but the functional paradigm still dominates it's landscape.

Usually when I learn a new language I rewrite a program that I have already done before. This helps me to focus on the language and not worry about the problem at hand because I already know the solution. So this program is a url shortener that makes a request to the Google api via JSON. I have written this url shortener in Python, Go, Objective - C, and C# so I know what I need to do and I have a reference. So lets get started.

To the code!

The first thing we need to do to create this program in F# is to "open" our dependencies. So at the top of a blank .fs file add these lines:

#light

open System.Net
open System.IO
open System.Text
open System.Web.Script.Serialization

We need the Net package for the WebRequest, IO for a StreamReader, Text for Encoding, and the Web.Script.Serialization for the JavaScriptSerializer so that we can convert the JSON response into a type that we can use. You will have to add the System.Web.Extensions dll reference to your F# project to use the JavaScriptSerializer.

Next we need to create a new type that the JavaScriptSerializer will deserialize the response to. The response JSON looks like this:

{"id":"short url", "kind": "kind of request", "longUrl": "long url"}

We will have to make this type a class with a default constructor so that the JavaScriptSerializer can create the instance. The members of the type; id, kind, and longUrl will have to be mutable so that the serializer can set the values. By default all types in F# are immutable unless set to mutable.

type Response = class
    val mutable id : string
    val mutable kind : string
    val mutable longUrl : string
    new() = {id = ""; kind = ""; longUrl = ""}
end

So there is the Response type with the default constructor. Next I will create another type that will hold all the request data that is needed. This is more for convenience and not really needed.

type Request = { url: string; requestMethod:string; contentType : string; data : string }

Now lets get the data required and populate a Request type. We will receive the long url via the command line args and the other data such as request method and content type can be hard coded along with the url to the Google api.

let args = System.Environment.GetCommandLineArgs()
let request = {
    url = "https://www.googleapis.com/urlshortener/v1/url";
    requestMethod = "POST";
    data = args.[args.Length - 1];
    contentType = "application/json" }

Functions

We will need a few helper functions that can convert a string in to a byte array, read a Stream into a string, and deserialize the response string into our Response type.

let get_bytes (str:string) =
    Encoding.UTF8.GetBytes(str)

let read_to_string (stream:Stream) =
    use reader = new StreamReader(stream)
    reader.ReadToEnd()

let deserialize_response (stream) =
    let serializer = new JavaScriptSerializer()
    serializer.Deserialize<Response>(read_to_string stream)

These three functions are very straight forward and to the point. One issue that I have ran into is that using classes from the .NET framework often requires a type annotation to be added to the function parameter(s) type inference cannot be used with overloaded methods. It kinda sucks but it is what it is. You will also notice the "use" keyword in the read_to_string function. This is the same as the "using" keyword in C#. Other than that everything is pretty obvious to a .NET developer looking at this code.

Now we will need to create a WebRequest, add the default parameters, write our data to the request stream, and then read the response.

let add_params (webRequest:WebRequest) =
    webRequest.ContentType <- request.contentType
    webRequest.Method <- request.requestMethod
    webRequest

let write_to_stream (webRequest:WebRequest) =
    let stream = webRequest.GetRequestStream()
    let data = get_bytes (sprintf "{\"longUrl\":\"%s\"}" request.data)
    stream.Write(data, 0, data.Length)
    webRequest

let get_response (webRequest:WebRequest) =
    webRequest.GetResponse().GetResponseStream()

let print_response (o:Response) =
    printf "%s" o.id

Easy. One of the things that you will notice is the lack of a return keyword. This is because whatever is on the last line of a function is automatically returned. You can see that in the write_to_stream function, even though it takes a webRequest as an input parameter, I set it on the last line so it can be returned. In C# ( and other languages ) this is often known as a "Fluent api". This will also help us a little bit later when we chain all of these functions together to process the request.

You can also see the "<-" operator when I assign values from our request type to the webRequest. You do not use the "=" operator for member assignment.

Lastly, you may have noticed that I never created the actual webRequest in the above code snippet. That is because in the final function we will create the webRequest and chain all of these functions together.

let get_short_url() =
    HttpWebRequest.Create(request.url)
        |> add_params
        |> write_to_stream
        |> get_response
        |> deserialize_response
        |> print_response

So get_short_url is the final function that ties everything together. We create the webRequest and use it's value to chain all the remaining functions. Each function consumes the result of the previous function this way we can use the "|>" to chain them together. This is very LINQ like but taken to a whole new level in functional programming.

So now all that is left is to get_short_url when a user passes in a long url on the command line. For this we will use a pattern match in F#.

match request.data.StartsWith("http") with
    | true -> get_short_url()
    | false -> printfn "No url passed"

System.Console.ReadLine()

Because our Request type's data member is a string we can use the standard .NET string function "StartsWith" to see if the first chars are "http". Then by using the "match with" we can check for each boolean value to determine our control flow. If true, we will call our get_short_url function, if false, we will tell the user that they did not pass a url. If you are on Windows you will want to add the "Console.ReadLine" call at the end of the file or the command prompt will exit without you seeing your results.

Complete Code

#light

open System.Net
open System.IO
open System.Text
open System.Web.Script.Serialization

type Response = class
    val mutable id : string
    val mutable kind : string
    val mutable longUrl : string
    new() = {id = ""; kind = ""; longUrl = ""}
end

type Request = { url: string; requestMethod:string; contentType : string; data : string }

let args = System.Environment.GetCommandLineArgs()
let request = {
    url = "https://www.googleapis.com/urlshortener/v1/url";
    requestMethod = "POST";
    data = args.[args.Length - 1];
    contentType = "application/json" }

let get_bytes (str:string) =
    Encoding.UTF8.GetBytes(str)

let read_to_string (stream:Stream) =
    use reader = new StreamReader(stream)
    reader.ReadToEnd()

let print_response (o:Response) =
    printf "%s" o.id

let deserialize_response (stream) =
    let serializer = new JavaScriptSerializer()
    serializer.Deserialize<Response>(read_to_string stream)

let add_params (webRequest:WebRequest) =
    webRequest.ContentType <- request.contentType
    webRequest.Method <- request.requestMethod
    webRequest

let write_to_stream (webRequest:WebRequest) =
    let stream = webRequest.GetRequestStream()
    let data = get_bytes (sprintf "{\"longUrl\":\"%s\"}" request.data)
    stream.Write(data, 0, data.Length)
    webRequest

let get_response (webRequest:WebRequest) =
    webRequest.GetResponse().GetResponseStream()

let get_short_url() =
    HttpWebRequest.Create(request.url)
        |> add_params
        |> write_to_stream
        |> get_response
        |> deserialize_response
        |> print_response

match request.data.StartsWith("http") with
    | true -> get_short_url()
    | false -> printfn "No url passed"

System.Console.ReadLine()

So that's it. I am still very fresh to F# and functional programming in general so my code does not look very elegant. It feels a little verbose working with the .NET classes in a functional language but hopefully I will learn more and be able to remove some of the excess and make this program more idiomatic to functional programming.

Infrastructure - Part 1

2012-09-29T00:00:00Z

Servers

Running your own infrastructure is awesome, if your a hacker like me. If your a square and you don't want to mess with your own hardware you have two choices if you are going to host your website or services online. If you don't feel like spending $1k a year you will probably have a VPS or EC2 instance with less than 1GB of RAM.

You can get a shared hosting provider and use what they give you. Basically PHP and Mysql. I run a heavy Python stack and Postgesql. PHP is not an option for me. Also 1GB of ram is not enough. I have to run a Minecraft server for myself and a few friends and I need at least 3-4GB of RAM just for Minecraft. ( This is also a test of how robust my setup is because if Minecraft would go down my friend my hurt himself. ) I also like running other services like jenkins as a build server. You still have to spend some money on things like a battery backup or upgrade your internet connection.

The trick to running your website on your own hardware and internet connection is to host all your content, images, js, css, etc.. on a CDN and only have your servers render the HTML and do database operations. Leave any content on S3. I run my websites on my own server and have S3 host all the images and content. You cannot tell the difference loading my page.

This is the first post of all the shit I have running. I'll try to give a high level overview and for some posts I will go into detail on the setup and settings. Keep an eye out for some of my shell scripts and other cool downloads.

Ajenti

Admin Console

Ajenti is an easy to install Python app that provides information about services running on your server. It also provides a nice UI for reading logs, managing cron jobs and users, and many other tasks.

On the main dashboard I can see the load and memory usage for my main node and the build server.

Ajenti has a nice plugin system. I have not dug into developing a plugin but it looks fairly straight forward. The default selection is great and plugins are fast and easy to install.

Munin

The charts that you are seeing are from Munin. It is a system monitoring application that can monitory local and remote systems. You need one computer as the server and then install munin-node on each computer (including the server) that you want to monitor. Munin is in apt and you only have a few lines to modify in the munin configs after install to get it up and running. You will have to have a HTTP server to serve the munin graphs so that ajenti can read them into the dashboard.

Check out the rest of these screenshots and view the links at the bottom of the page.

Logs

Users

Health

Usage Patterns

2012-09-28T00:00:00Z

Update: I will have updated graphs in a day or two. My time tracking app was only exporting the top 10 applications and I just found the setting to export the top 25.

I always have an app running on my computer that logs all the time I spend in each application. It helps to give some interesting insights how, what, how long, what days, and times do I do certain tasks. You can see what hours of the day that you are productive and what hours you just surf the web.

Usage by App

Just for clarification:

iTerm - terminal app
Forklift - file/ssh/ftp file management
SublimeText 2 - Text Editor

So Chrome wins pretty easy but I do spend a lot of my time in a terminal. I hate pixels and gui's. I wish I could do everything from the command line. The other 21% of my time is spent programming in SublimeText. Even though it is not a large amount 5% of my time sitting at my computer is managing files in Forklift. That is still a lot of time for something like moving a file from one directory to another.

These two graphs show pretty much the same thing. What hours of the day do I must most of my work. 8pm-10pm is when I do most of my work. However I do like the area graph about what apps I am using at what time.

You can see that I have a large spike in the morning with Chrome. This is when I get up and surf the internet looking for news or reading RSS feeds. I have a pretty constant usage of the terminal throughout the entire day but my development really picks up in the later hours with SublimeText and iTerm.

Hours by day

Thursday, Saturday, and Sunday are my big computing days. This mostly because I am at work throughout the week so I do what I want on the weekends.

Finally my usage throughout the past few months. You can read this graph a few different ways. You can look at the usage trends over time and see if I am starting to use a computer more or less. You can also look at how many hours I am on a computer during a single day. On average it's about ~4.5hrs a day. Not too bad at all.

You can study the graphs if you want and see if doing something like this would be advantageous to you. I always enjoy looking over data and seeing trends in my own life and how I use and interact with things. You can use data like this to determine when you are most productive by hour or day, day of week, etc...

Development Infrastructure

2012-09-10T00:00:00Z

Having servers that automate much of your development process is cool. Entities living out on the web just waiting to process any task that you throw their way.

I currently have two servers in my cluster that run my network. Everything from my public web sites, to mongodb and Postgresql, to private git and build servers. Why have all of this? Because it makes me happy when I push changes to my git server and within seconds the project is built, unit tests ran, and a report is emailed to me.

Public Site

I run a Python stack for my web server. Python 2.7 with django as the framework running on-top of lighttpd. Lighttpd also acts as the proxy server for the rest of the services that run on the cluster. Postgresql is the datastore for my sites. I still run mongodb on the server for quick, scheme-less storage.

On Node1 is my git repositories via Stash. Stash is not necessary but it provides a nice UI for managing repositories and users. I could do all of this via ssh but for $10 Starter License it is worth it. You can easily browse and see diffs on the web as well.

The only drawl back to Stash is that is runs on-top of Java and hogs RAM. The Stash instance consumes about 700-900MB of RAM for not doing much. The other memory hog is a minecraft server that I run. Minecraft consumes about 2GB+. This server has to be robust because if the minecraft sever goes down, I will have some angry users texting me until it's back online.

Build server

I run Jenkins-Ci as my build server. I have tried Bamboo for a week but the license was too restrictive for my needs, ( not enough jobs on the starter license ). For my python projects Jenkins has many nice plugins.

Even though the build server is running on Debian it can build both Python and .NET projects thanks to mono ( xbuild ). I am having some issues running nunit tests for a non-root user. I have not been able to figure out what the issue is yet so my .NET projects do not have automated tests.

Server Software

Both servers run Debian and are administered via ssh, no passwords just ssh keys. Each node does it's daily backup to the other with ~3day off-site dumps to Amazon S3.

Node1

Debian 6
lighttpd
git 1.7+ ( built from src )
django
mongodb
postgresql
sun-java
stash
Amazon S3 as CDN for content

Domains Hosted * CodeAssistant

Node2

Debian 6
sun-java
jenkins
mono ( built from git repo )
git 1.7+( built from src )

Future Changes Currently my blog crosbymichael.com ( this site ) is hosted on dreamhost. Also HTTPAssistant is a wordpress site hosted on dreamhost as well. My dreamhost service expires Feb. of next year so I will be moving both of these sites to Node1. I will replace wordpress with a static one page landing page. Hopefully I will have HTTPAssistant 2.0 done and replace that ugly thing.

web.py server setup

2012-08-04T00:00:00Z

Python is my favorite language to work with. It is so versatile. I can automate tasks, write quick scripts, process data, and even run a web application. It is very portable and I love the syntax. For a web framework I use web.py because it stays out of your way and does not impose itself on your application. With other frameworks you have to do it their way or struggle the entire time. It's awesome but setting up a "production" server is a bitch. Tutorials suck and getting Python to run under Apache is a waste of 6+ hours with all the config bullshit. Leave it and do it my way.

What you will learn:

Install lighttpd on Ubuntu/Debian
Install MySQL
Setup web.py to run under lighttpd
Setup MySQLDB for Python to MySQL

Ubuntu Server

Ubuntu or any Debian distro will work with this tutorial. I am currently running my server on Ubuntu Server 11.04. You need to make sure that Apache is not installed on your server. If it is remove everything Apache. Make sure you have shell plus access to install software on your server.

MySQL

Now we will need to install MySQL on your server before we do anything else. Just skip this part if you don't use MySQL. This is very easy, just run these two commands to install MySQL and the development libraries that Python needs.

apt-get install mysql-server libmysqlclient-dev

Remember that you should be running these as root or sudo. If you didn't know this just give up now. You will also need to setup your MySQL username and permissions for your web application to use. I am not going to go over this process because I am lazy. Google it if you don't know how.

lighttpd

Now we can install the lighttpd server.

apt-get install lighttpd

You can go ahead and browse to the site to see if lighttpd is functioning properly.

Python Build Tools and MySQLDB

With our web server and MySQL database setup on the server it is time to get our Python environment setup. Start by installing the buildtools and other Python development tools.

apt-get install python-setuptools python-dev build-essential

This will install "easy_install" and other tools so we can build the python MySQL driver. After this is complete we need to download and install the MySQLDB driver. This is easy if you have the libmysqlclient-dev and setuptools installed.

wget http://downloads.sourceforge.net/project/mysql-python/mysql-python/1.2.3/MySQL-python-1.2.3.tar.gz
tar -zxf MySQL-python-1.2.3.tar.gz

This will download and unzip the driver. Now "cd" in to the folder and run these two commands:

python setup.py build
python setup.py install

Web.py

All we have left to do is install web.py, flup, and configure lighttpd for our application. The first two are easy.

easy_install web.py
easy_install flup

Now lets configure lighttpd.

cd /etc/lighttpd/

Replace the contents of lighttpd.conf with the text below:

server.modules = (
    "mod_access",
    "mod_alias",
    "mod_compress",
    "mod_accesslog",
)

server.document-root        = "/var/www"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"

## Use ipv6 if available
#include_shell "/usr/share/lighttpd/use-ipv6.pl"

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/x-javascript", "text/css", "text/html", "text/plain" )

include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

server.modules   += ( "mod_fastcgi" )
server.modules   += ( "mod_rewrite" )

 fastcgi.server = ( "/index.py" =>
 ("/" => ( "socket" => "/tmp/fastcgi.socket",
    "bin-path" => "/var/www/index.py",
    "max-procs" => 1,
   "bin-environment" => (
     "REAL_SCRIPT_NAME" => ""
   ),
   "check-local" => "disable"
 ))
 )

 url.rewrite-once = (
   "^/favicon.ico$" => "/static/favicon.ico",
   "^/static/(.*)$" => "/static/$1",
   "^/(.*)$" => "/index.py/$1",
 )

Note the server.username and server.groupname. This user/group will need proper permissions to your application files. Study the config above. Everything is simple and you should be able to change whatever settings that you need.

Make sure you updated the server.document-root path to where the location that you are going to be placing your application in. I just use /var/www for my applications. This path will need to be updated in the fastcgi.server section along with your main python file for web.py. I usually call my main application file "index.py" and the above config is setup to reflect that.

Now that we have lighttpd set up we can go into our /var/www directory and set up a sample web.py application ( if you do not already have a working app ). We can just use this simple template. Make a note of the shebang at the top of the file. You will need to add that to your application file. Just save the file as "index.py".

#!/usr/bin/env python

import web

urls = ('/(.*)', 'Index')

application = web.application(urls, globals())
web.config.debug = True

class Index:

    def GET(self, name=''):
        return 'Hello World'

    def POST(self, name=''):
        return 'Hello World'

if __name__ == '__main__':application.run()

Make the file's owner www-data and make it executable. After that restart lighttpd.

chown www-data:www-data index.py
chmod +x index.py
service lighttpd restart

Go ahead and navigate to your site in your browser and you should see "Hello World". If you do not, cat out the lighttpd error log at:

cat /var/log/lighttpd/error.log

That's it. I hope this helps to save hours of your life, hours that I wasted trying to get web.py to run under Apache bullshit. This is the way to go, lighttpd is fast, easy to config, and perfect for web.py.

Objective - C LINQ

2012-07-06T00:00:00Z

I have been working on two iOS applications over the past few days and have been longing for a LINQ like syntax in Objective-C. LINQ is an amazing way to work with collections and it really adds a productivity boost to your coding. Since Objective C does not have LINQ I thought I would code it myself. Right now it only supports non-destructive operations but here it is.

@interface NSArray (IQuery)

- (id) first;
- (id) last;
- (id) firstOrDefault: (int(^)(id)) predicate;
- (id) firstOrDefault;

- (BOOL) any;
- (BOOL) any:(int (^)(id)) predicate;
- (BOOL) all:(int (^)(id)) predicate;
- (NSArray *) ofClass: (Class) class;
- (NSArray *) where: (int (^)(id)) predicate;
- (NSArray *) select: (id (^)(id)) predicate;
- (NSArray *) skip: (id (^)(id)) predicate;
- (void) forEach: (void(^)(id)) action;

@end

This is the current interface. Most methods return BOOL or an NSArray so that the query operations can be chained together.

Code

What we want to do in this example is to take an array with strings and ints, filter it by class, then by strings starting with 'h'. Here is how you do it:

NSArray * array = [[NSArray alloc] initWithObjects:
                          @"hot dog", 
                          @"hamburger", 
                          @"pizza", 
                          @"taco", 
                          @"pop", 
                          0, 
                          5, 
                          3, 
                          nil];

NSArray * filter = [[array 
                           ofClass:[NSString class]]
                           where:^(NSString * item) {
                            return [item characterAtIndex:0] == 'm';
                        }];

[filter forEach:^(NSString * i) { NSLog(i); }];

So that is it. The first IQuery method is "ofClass". This takes a class and returns an NSArray of all the items in the array that are of that type(class). It is the same idea as C#'s OfType.

Next is "where". This takes a predicate block that will return any item that satisfies the condition. The syntax is not as clean as C# implementation but it's a start.

Now after we have this filtered list with the chain of calls to "ofClass" and "where", we can now use the "forEach" method to enumerate the array, passing a block as the action to perform on each item.

Select

One last thing, if you want to select a certain property from an item in the array, you can use the "select" method to do so. Here we select the price for each of the items.

NSArray * array = [[NSArray alloc] initWithObjects:
                         [[Items alloc] initWithNameAndPrice:@"hamburger" :[NSNumber numberWithInt:3]], 
                         [[Items alloc] initWithNameAndPrice:@"hot dog" :[NSNumber numberWithInt:5]],
                         [[Items alloc] initWithNameAndPrice:@"pop" :[NSNumber numberWithInt:2]], 
                       nil];

NSArray * prices = [array select:(id)^(Items * item) {
    return item.price;
}];

This will return an array of NSNumber items. Nothing more.

That's all for now. If you are interested in the project I will have the binary to download soon. Just email me with any comments or questions.

Objective C Design Patterns - Factory

2012-05-21T00:00:00Z

Design patterns can help you solve complex coding problems with proven solutions. This will be a mini series about different design pattern implementations in Objective - C. I will be using a simple console app and updating the code on github with each new pattern.

Factory Pattern

This pattern provides a consistent way for object creation. A factory provides one point of creation for one or a group of objects. It also helps to not code to a concrete class.

When to use it? Many objects need created that share an interface Creation of complex objects Hide concrete classes and implementation from client code Provide one point of entry for the creation of a class or group of classes

Static Factory

Most of the time I find myself using a static factory instead of an abstract factory. A static factory is what we are going to focus on now. In the console app I have an interface for logging text to the console. There are two implementations for this interface. One uses NSLog and the other uses fprintf().

#import <Foundation/Foundation.h>

@protocol CMConsoleOutput <NSObject>

- (void) outputText: (NSString *) content;

@end

//Class one
#import "CMPrintLog.h"

@implementation CMPrintLog

- (void) outputText:(NSString *)content {

    fprintf(stdout, "%s\n", [content UTF8String]);
}

@end

//Class two
#import "CMLogOutput.h"

@implementation CMLogOutput

- (void) outputText:(NSString *)content {
    NSLog(content);
}

@end

Now we need a factory to choose what implementation the client should use without knowing the exact types of objects that are returned. The client does not care what they get as long as it implements the interface. It also helps to cut down the number of imports you have in your header files. You no longer have to import 5 different class headers; you just have to import one factory header. I went ahead an created the factory class file, typedef for identifying what type of output object the client wants, and a static method for creating/returning the object.

typedef enum ouputTypes {
    Timestamp,
    NoTimestamp
} OutputTypes;

@interface CMOutputFactory : NSObject

+ (id<CMConsoleOutput>) create: (OutputTypes) type;

@end

Now the only thing that we need to remember is to autorelease the objects that we create if you are not using ARC.

@implementation CMOutputFactory

+(id<CMConsoleOutput>) create:(OutputTypes)type {
    switch (type) {
        case Timestamp:
            return [[CMLogOutput alloc] init];
        case NoTimestamp:
            return  [[CMPrintLog alloc] init];
    }
}

@end

Finally we just need to use our new factory.

#import <Foundation/Foundation.h>
#import "CMOutputFactory.h"

int main(int argc, const char * argv[])
{

    @autoreleasepool {

        id withTimestamp = [CMOutputFactory create:Timestamp];
        id noTimestamp = [CMOutputFactory create:NoTimestamp];

        [withTimestamp outputText:@"Output without a time stamp"];

        [noTimestamp outputText:@"Without a time stamp"];
    }
    return 0;
}

Very easy, we only have one import and a consistent way to create objects. It is very straight forward for people consuming an API's classes when you provide a factory for object creation. They do not need to know the inner workings of your API, they just want a certain type of functionality and your factory provides that. If you have any questions just ask.

CodeAssistant

2012-05-14T00:00:00Z

My second application was published this week.

CodeAssistant is a simple tool to test code snippets in virtually any language on Windows. It has been in development for four months and has been a great learning experience for domain driven development and .NET's WPF.

The application is build on top of WPF using the MVVM design pattern. I created a MVVM framework called MVVM Async that you can find it on github. WPF and MVVM are still in their infancy and there are some rough edges and verbose syntax that needs fixed ( command delegates anyone? ). I like WPF and MVVM but I wish they would have just used the proved MVC tooling form ASP.NET MVC in WPF. There are a lot of developers familiar with MVC, partials, and the such that would have been a good fit for WPF. But no, they had to build another framework and you have to relearn everything all over again.

This has been the first project where I have built it from the ground up with domain driven design and solid patterns. Each of the layers in the application are completely decoupled from the others and are in their own projects. The view/application layer only has a reference to the ViewModel project and the ViewModel project only references the Domain layer.

Future Projects

I don't have any plans for a future project. I have been wanting to work on a game. Desktop or iOS, I have not decided. I may play with the XNA development toolkit before making a decision. A friend of mine said that Corona is a good platform to build games on. I have a few game ideas but don't know yet.

If anyone has a game that they want to work on, it does not matter what platform, I can code in any language, let me know.

Setup Ruby on Rails with Nginx and Unicorn

2012-02-27T00:00:00Z

I lost an entire day of coding trying to get his to work. We are all lucky that I am a developer and not a systems admin.

However around 9pm I finally have it running. I burned through a lot of screwed up VM's trying to get this to work but finally made it. I tried Apache, Nginx, Passenger, and finally Nginx and Unicorn worked like it should. I don't know what the issues were but the actual rails app was not being ran.

I'll try to go through every single step I make, from installing Debian and the settings that I use to the installing and compiling gems, rails, and Nginx. I will also try to make this into a bash script to automate much of the install incase you f something up.

Requirements

Email
Rails
MySQL
Nginx

Installing Debian 6

I am going to use Debain 6 ( net install ) because it has the bare minimum. Only what we install is what it is going to have. You can also use Ubuntu Server 10.04 or 11.x with these commands. They should work the same. I have tested this on Ubuntu Server 10.04 so I know it works.

Ok boot into your linux install and configure your languages. I set the hostname as:

nginx-rails-server

The domain name to my url.

Create your root and user passwords then for the disk partitioning just use the entire hard drive. I also choose to keep all system and user files on one partition.

Pick one of the package servers for debian. It does not matter.

Install Package Selection

Here is my package selection for install. We do not want a webserver or anything else. I usually install the Mail server so that I don't have to mess with it but since we are going to use Nginx and install mySQL manually all we want is the SSH and Mail servers installed right now.

Install GRUB boot load and restart.

Once your booted up, login really quick and run ifconfig to get the ip of the machine. We will want to SSH into it so we can use our desktop Terminal ( copy and paste ) instead of using a keyboard on the actual machine or through the VM.

Once you have the IP login as root via ssh. Lets nano and paste in the setup script from below. Chmod 755 it and let it rip.

Application Install

So now we need to get our base applications installed on the server. If you are using Ubutnu Server you will want to run:

sudo apt-get update
sudo apt-get upgrade

Also from here on out I will not place sudo infront of all the commands that need it. If you don't konw when you need to run the command with sudo or when not to, just go home. Don't waste my time emailing why when you run a command on Ubuntu it says "permission denied."

Here is the script that I will run to get the base apps installed from apt.

#!/bin/bash

apt-get update
apt-get upgrade -y

apt-get install wget build-essential ruby1.8 ruby1.8-dev irb1.8 rdoc1.8 zlib1g-dev libopenssl-ruby1.8 libopenssl-ruby libzlib-ruby libssl-dev libpcre3-dev -y

apt-get install vim -y
apt-get install git-core -y
apt-get install screen -y
apt-get install ctags -y
apt-get install curl -y
apt-get install libsqlite3-dev -y
apt-get install libcurl4-openssl-dev -y
apt-get install libssl-dev -y

If you notice I did not install mySQL in here because it will ask for the root passwords and I wanted this base install script to be fully automatic.

Now that that is done lets go ahead and isntall mySQL

apt-get install mysql-server libmysqlclient16 libmysqlclient-dev mysql-client mysql-common -y

When asked, pick a good mySQL root password.

Gems

Now that we have everything from apt installed lets get rubygems installed. I don't want to install gems via apt so we are going to grab the tarbal and install that way.

cd /opt/
wget http://production.cf.rubygems.org/rubygems/rubygems-1.5.1.tgz
tar -zxvf rubygems-1.5.1.tgz
cd /opt/rubygems-1.5.1/
ruby setup.rb

This will install gems in:

/usr/bin/gem1.8

Logout of your bash session and them back in and you should be able to type gem1.8 to execute some commands.

Create another bash setup script for all these gem installs. They will take a while so it's easier to just let it roll. I chose not to install the rdoc and ri because this is the server and we don't need it.

#!/bin/bash
gem1.8 install fastthread --no-rdoc --no-ri 
gem1.8 install rake rack --no-rdoc --no-ri 
gem1.8 install rails --no-rdoc --no-ri 
gem1.8 install mysql --no-rdoc --no-ri 
gem1.8 install unicorn --no-rdoc --no-ri 
gem1.8 install sqlite3-ruby --no-rdoc --no-ri 
gem1.8 install execjs --no-rdoc --no-ri

Javascript engine

Rails needs a javascript engine to run. Pick your favorite one and install it. I will isntall node.js for this. I will not use apt or gems to install node.js but will grab the latest stable release from github and compile it.

Lets grab the latest stable release from github and download it in our /opt/ dir.

wget -O nodejs.zip https://github.com/joyent/node/zipball/v0.6.11

One thing that I forgot to install is zip so that we can unzip this file. So quickly install

apt-get install zip

then unzip node.

unzip nodejs.zip

Lets cd into joyent-node-* folder and start building it

./configure
make
make install

Also make sure that you install coffee-rails and jquery-rails. I got this error when trying to make the test app below. It may be a good idea to make a test app as root so bundler to handle these for us.

gem1.8 install coffee-rails --no-rdoc --no-ri
gem1.8 install jquery-rails --no-rdoc --no-ri

Testing rails

Now that we have everything setup lets login to our user account ( not root ) and create a sample rails app and run it with the builtin WEBrick server to make sure that rails works on our server.

In your home dir:

rails new testRails
cd testRails/
rails server

Your server should start up fine so navigate to it's ip and make sure that rails is running and the environment links displays information about your setup.

Hopefully your rails test app looks like this in the browser. If not then don't even try to go forward. Get the WEBrick server working first and then move on.

Nginx

We will go ahead and install nginx via the apt repo. We first need to add the key to the resposity so download it then add it.

wget http://nginx.org/keys/nginx_signing.key
apt-key add nginx_signing.key

Now add the repos to your /etc/apt/sources.list

deb http://nginx.org/packages/debian/ squeeze nginx
deb-src http://nginx.org/packages/debian/ squeeze nginx

Then

apt-get update
apt-get install nginx

It's a really quick install. After the install is finished you can test to make sure that it works by going to your machines ip address. You should see the welcome to nginx text.

User group and dir

Now lets setup a new directory that nginx and our rails apps will be in.

cd /var
mkdir www
usermod -a -G www-data nginx
usermod -a -G www-data YOURUSERNAME

chgrp -R www-data www/
chmod -R 775 www/

Now our user and nginx should be part of www-data group and own /var/www

Nginx Config

Now lets grab the config file form the unicorn repo and place it in the nginx config directory overwriting the default config.

cd /etc/nginx/
cp nginx.conf nginx.conf.old
rm nginx.conf
wget https://raw.github.com/defunkt/unicorn/master/examples/nginx.conf

Now we need to setup our paths in the nginx.conf file.

Uncomment this line:

listen 80 default deferred; # for Linux

Change the root paths to there our apps public dir is:

root /var/www/testapp/public;

Save and restart nginx

/etc/init.d/nginx restart

Now lets get unicorn setup with init.d and make a test app in the www dir via our other user.

Create the test app

rails new testapp

Everything should go smooth if you have things setup right. You don't need root or sudo to make the test app in var/www because we added ourself and nginx to the group that owns the folder. Lets cd into the test app's config dir and download the unicorn config file.

cd testapp/config/
wget -O unicorn.rb https://raw.github.com/defunkt/unicorn/master/examples/unicorn.conf.rb

Vim or nano into it and setup the paths to your app. Here is my config:

# Sample verbose configuration file for Unicorn (not Rack)
#
# This configuration file documents many features of Unicorn
# that may not be needed for some applications. See
# http://unicorn.bogomips.org/examples/unicorn.conf.minimal.rb
# for a much simpler configuration file.
#
# See http://unicorn.bogomips.org/Unicorn/Configurator.html for complete
# documentation.

# Use at least one worker per core if you're on a dedicated server,
# more will usually help for _short_ waits on databases/caches.
worker_processes 4

# Since Unicorn is never exposed to outside clients, it does not need to
# run on the standard HTTP port (80), there is no reason to start Unicorn
# as root unless it's from system init scripts.
# If running the master process as root and the workers as an unprivileged
# user, do this to switch euid/egid in the workers (also chowns logs):
# user "unprivileged_user", "unprivileged_group"

# Help ensure your application will always spawn in the symlinked
# "current" directory that Capistrano sets up.
working_directory "/var/www/testapp" # available in 0.94.0+

# listen on both a Unix domain socket and a TCP port,
# we use a shorter backlog for quicker failover when busy
listen "/tmp/.sock", :backlog => 64
listen 8080, :tcp_nopush => true

# nuke workers after 30 seconds instead of 60 seconds (the default)
timeout 30

# feel free to point this anywhere accessible on the filesystem
pid "/var/www/testapp/shared/pids/unicorn.pid"

# By default, the Unicorn logger will write to stderr.
# Additionally, ome applications/frameworks log to stderr or stdout,
# so prevent them from going to /dev/null when daemonized here:
stderr_path "/var/www/testapp/shared/log/unicorn.stderr.log"
stdout_path "/var/www/testapp/shared/log/unicorn.stdout.log"

# combine Ruby 2.0.0dev or REE with "preload_app true" for memory savings
# http://rubyenterpriseedition.com/faq.html#adapt_apps_for_cow
preload_app true
GC.respond_to?(:copy_on_write_friendly=) and
  GC.copy_on_write_friendly = true

before_fork do |server, worker|
  # the following is highly recomended for Rails + "preload_app true"
  # as there's no need for the master process to hold a connection
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.connection.disconnect!

  # The following is only recommended for memory/DB-constrained
  # installations.  It is not needed if your system can house
  # twice as many worker_processes as you have configured.
  #
  # # This allows a new master process to incrementally
  # # phase out the old master process with SIGTTOU to avoid a
  # # thundering herd (especially in the "preload_app false" case)
  # # when doing a transparent upgrade.  The last worker spawned
  # # will then kill off the old master process with a SIGQUIT.
  # old_pid = "#{server.config[:pid]}.oldbin"
  # if old_pid != server.pid
  #   begin
  #     sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
  #     Process.kill(sig, File.read(old_pid).to_i)
  #   rescue Errno::ENOENT, Errno::ESRCH
  #   end
  # end
  #
  # Throttle the master from forking too quickly by sleeping.  Due
  # to the implementation of standard Unix signal handlers, this
  # helps (but does not completely) prevent identical, repeated signals
  # from being lost when the receiving process is busy.
  # sleep 1
end

after_fork do |server, worker|
  # per-process listener ports for debugging/admin/migrations
  # addr = "127.0.0.1:#{9293 + worker.nr}"
  # server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true)

  # the following is *required* for Rails + "preload_app true",
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection

  # if preload_app is true, then you may also want to check and
  # restart any other shared sockets/descriptors such as Memcached,
  # and Redis.  TokyoCabinet file handles are safe to reuse
  # between any number of forked children (assuming your kernel
  # correctly implements pread()/pwrite() system calls)
end

They are easily found and just point them to your app dir.

![Unicorn Config][8]

Now lets run unicorn and test out our app. Make two new folders in your app before login back in as root and running unicorn.

mkdir /shared
cd shared/
mkdir pids
mkdir log

unicorn_rails -c /var/www/testapp/config/unicorn.rb -D

Now go to your server ip in your browser and you should see the same thing with the ability to click the envirionment link with data displayed. If you do no see any data then unicorn is not running or setup correctly. If you do see it then we are done. It's working and ready to go. You may want to add unicorn to an init.d script so that it starts with your server.

Unicorn init.d

As root:

cd /etc/init.d/
vim unicorn

Here is the init.d script that I am using so just paste it into the file.

#! /bin/sh

# File: /etc/init.d/unicorn

### BEGIN INIT INFO
# Provides:          unicorn
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the unicorn web server
# Description:       starts unicorn
### END INIT INFO

DAEMON=/usr/bin/unicorn
DAEMON_OPTS="-c /var/www/testapp/config/unicorn.rb -D"
NAME=unicorn
DESC="Unicorn app for testapp"
PID=/var/web/testapp/shared/pid/unicorn.pid

case "$1" in
  start)
    echo -n "Starting $DESC: "
    $DAEMON $DAEMON_OPTS
    echo "$NAME."
    ;;
  stop)
    echo -n "Stopping $DESC: "
        kill -QUIT `cat $PID`
    echo "$NAME."
    ;;
  restart)
    echo -n "Restarting $DESC: "
        kill -QUIT `cat $PID`
    sleep 1
    $DAEMON $DAEMON_OPTS
    echo "$NAME."
    ;;
  reload)
        echo -n "Reloading $DESC configuration: "
        kill -HUP `cat $PID`
        echo "$NAME."
        ;;
  *)
    echo "Usage: $NAME {start|stop|restart|reload}" >&2
    exit 1
    ;;
esac

exit 0

Edit to your correct paths then:

chmod +x unicorn
/usr/sbin/update-rc.d -f unicorn defaults

Reboot and see if nginx and unicorn are successfully servering your app.

Everything should work on the defult page via port 80. If you do not see images or the environment info that means unicorn is not running. Check the log files that we setup in the config ( the paths ) and see what is happening. I hope this tutorial works for everyone. I had a lot of trouble setting this up so hopefully you will not repeat my pain.

Multithreading in C#

2012-01-28T00:00:00Z

From my point of view the reason why we create multithreaded applications is so the user experience does not suffer. We have operations that need completed but running these on the main thread will block the UI and piss off our users.

Most of the operations have some similarities. Atleast for myself, the most common use case is the user requests information, we process it, then display the results back. That is the most common use that I use background threads for. When we display these results back, we need to write code that is executed on the Main thread to update the UI.

Working with delegates, callbacks, and IAsyncResults suck. They cause you to write more methods and change your code. There should be an easy way to write code normally and have all the benefits of async programming. That is what I try to accomplish in my WPF MVVM-Async framework. It allows you to write code normally and create multithreading applications with lambdas.

MVVM-Async

So first thing that you need to do to take advantage of the framework is to make your ViewModel inherit from ViewModelBase and pass it a type for the model. By doing this you automatically get access to the Model object.

public class EditorViewModel : ViewModelBase<EditorModel>
...
this.Model.EditorModelProperties...

Async

Now just code and when you want to dispatch a process in a background thread call DispatchAsync() and pass it a lambda.

DispatchAsync(()=> {

    int i = 0;
    bla bla bla

});

The framework will automatically spawn a new thread and execute whatever you placed inside the lambda on that thread. You can even call another method and pass any variables to it. By using lambdas you stay in the scope of the method calling DispatchAsync.

DispatchAsync(() => {

    SomeOtherFunc(name, date, age);

});

Updating the UI

So now we have this lambda executing on a background thread and processing data. We need to take that data and update our UI.

DispatchAsync(() => {

    string fullName = GetFullName(first, last);

    MyViewsTextBoxDataSource = fullName;

});

This is very simple but you get the point. If we were to try to set one of our view's controls from a background thread we would throw an exception because you can only set Controls on the thread that created it. So without using callbacks we can do this easily with the framework.

DispatchAsync(() => {

    string fullName = GetFullName(first, last);

    DispatchMainAsync(() => {

        MyViewsTextBoxDataSource = fullName;

    });

});

How hard was that? If we want to dispatch a lambda to the main thread that updates the UI, we can write it inline, with all our variables in scope, and have that code executed on the main thread. No exceptions.

Easy, simple, and you don't have to have callbacks or other crap to mess with. If you do want a callback when a thread has finished executing the framework does support this.

void DispatchAsyncWithCallback(action, callback);
void DispatchAsyncWithCallback(action, callback, whereTo);

The whereTo tells the framework on what thread do you want the callback executed one. Call me back on a background thread or call me back on the main thread.

DispatchAsyncWithCallback(() => {

    doSomeStuff();

}, () => {

    doSomeStuffOnTheCallback();

}, DispatchOn.Main); //Dispatch the callback on the main thread.

It's all done inline.

Queues

There is also support of Queues. You can load up a DispatchQueue with tasks and invoke the entire queue.

void AddQueue(DispatchQueue queue);
void InvokeQueue(string name);
void ProcessAllQueues();

Timed Execution

You can also tell the framework when to dispatch a thread.

DispatchIn(() => { somethingToDo(); }, "5min");

Or at a specific time.

DateTime someTime;
DispatchAt(() => { somethingToDo(); }, someTime);

Final

I am still working on this and will be releasing the code soon after CodeAssistant is complete. CodeAssistant is using this framework.

tCal - iCal events in terminal

2012-01-14T00:00:00Z

I just puked up some old code for a little app that displays your iCal events in the terminal. I have it as one of my startup scripts so when I start a new bash session I see my events for the next 7 days. It's a very simple but cool app. I am not proud of the design but I just cleaned up the code tonight and threw it on tCal on Github.

Usage

It is very simple to use. Just throw it in a folder that is in your PATH and type tCal to see your events for the next 5 days (default). if you want more or less days then just:

tCal -d 20

There is no config to setup with your calendars, it just works.

Here is a little taste for the code. Head over to Github for it all.

int main(int argc, const char * argv[])
{

    @autoreleasepool {
        CMCalendar * calendar = [[CMCalendar alloc] init];
        CMArgumentParser * parser = [[CMArgumentParser alloc] init];
        NSInteger days;

        @try {
            days = [(NSString *)[parser argumentForName:@"-d"] integerValue];
        }
        @catch (NSException *exception) {
            days = DEFAULTDAYS;
        }
        NSArray * events = [calendar getEventsForDays:days];
        for (id <ITerminalOutput> output in events) {
            [output display];
        }

    }
    return 0;
}

If you have any ideas just send them to me.

Download Binary

Bash Informational Startup

2011-11-11T00:00:00Z

If you use Terminal on your Mac like I do. It is the first application that is run at starup and the last to close. When you start Terminal in a new window or tab that is a great time to review your computer stats, iCal events, and disk sizes. I saw a lifehacker post abut a menu bar app that shows all the disks that are connected to your Mac and their size and free space. I also use todotxt for my todo lists and because it does not support dates for your tasks I keep my dates for events in iCal. I wanted to see all of this whenever I startup a new bash session so here is how I did it.

So this is what I see when I start a new bash session. All my information that I want is there plain text and easy to read.

There is my computers uptime at the very top with load averages.

Next is all the disk space information of my internal hard drive and external USB hard drives. This is a straight forward easy command.

Next up is my iCal events for today and tommorrow. You can easily set how many days forward you want. This is done by a simple Objective - C command line app that I wrote.

Last is todotxt. All my tasks.

So here is the last part of my bash_profile that makes this all happen. You can download all my app that displays your iCal events from my github of dotfiles (it's in the bin dir). You will also find all kinds of awesome stuff in it.

End of my .bash_profile

echo "-----------------------------------------------"
#Display uptime
uptime
echo "-----------------------------------------------"
#Display disk space usage
df -hl
echo "-----------------------------------------------"
echo "Up coming events:"
echo "-----------------"
#Call to my cmd line app to show iCal events
CommandLineAssistant
echo "-----------------------------------------------"
echo "Todo Tasks:"
echo "-----------"
#List my todos 
todo list
echo ""

Compile Vim for Python

2011-10-13T00:00:00Z

Update

I just added the newest MacVim compiled for python using the 10.7 Lion SDK. You can downloading it below.

If you want to use the Python auto complete in vim then you have to compile vim for python on your system. If you are running OSX I have just compiled MacVim for Snow Leopard and I am working on a compiled Lion version but the SL one works fine. Then you can setup an alias for vim so that it points to the binary in the MacVim app so you can still use vim from the command line and as the MacVim app.

It works really well and is fairly easy this way.

MacVim with Python SL MacVim with Python Lion

First download the already compiled MacVim.app from the link above and extract it to your Applications folder.

Then download:

pythoncomplete.vim

vim files and put it in your ~/.vim/autoload/ folder. It should be named:

pythoncomplete.vim

If you need, you can see my .dotfiles on github that already has my vimrc and vim dir in it.

My dotfiles

Then just add:

filetype plugin on

Last thing if you want to run this version form terminal just add:

alias vim=/Applications/MacVim.app/Contents/MacOS/Vim

to your ~/.bash_profile

You can see all these settings and more in my dotfiles.

All done. To use the auto complete just hit control+x then still holding down the control key it o

Python talking to Arduino

2011-10-08T00:00:00Z

I just setup my Arduino with an LCD display. You can follow the tutorial here to get the LCD and board wired up. We will also be using the sample code from that tutorial.

The Problem

The issue that I ran into was displaying messages from the Arduino IDE's SerialMonitor worked fine but when I was trying to display a message that I sent via python, it was not showing on the screen. After some help I finally have it working so this is how you fix it.

Python Code

You will need to install pyserial so that you can talk to your board via USB. It's simple if you have build tools installed so just:

sudo easy_install pyserial

Now if we look at the code that does not work first you can see what you need to change. We need to import serial and then setup the connection to the board.

import serial 
port = serial.Serial('pathtoyourboard', 9600)

You will need to look in your Arduino IDE for the path to your board and the 9600 is the rate that is definded in the Arduino code that we are loading onto it. Now we need to write some data do the Arduino and close the connection.

port.write('sometext')
port.close()

If we load the code from the tutorial that I linked at the beginning of this post onto your board and then ran this python code, we would just see a blank LCD. The probelm is that whenever the serial connection is first made, it takes the Arduino two seconds to get that port open and setup. So for our python code to work we need to import the time module and add a sleep for atleast 2 seconds right after our port = serial.Serial statement. Here is the what the final python code will look like with it working and the message changed.

import serial
import time

port = serial.Serial('/dev/tty.usbmodemfd141', 9600)
time.sleep(2)

message = ['My', 'name', 'is', 'michael']

for word in message:
    port.write(word)
    time.sleep(1)

port.close()

So this code makes the connection and waits 2 seconds for the Arduino to get setup, then starts writing data to it. Everything works find and it's good to go. So in the future when you are working with Python and Arduino via USB, wait 2 seconds before sending data to it or it will not get it.

How to backup gmail the right way

2011-10-04T00:00:00Z

So like many people my life is stored in my gmail. I trust google to keep my data safe but since so many things, from contacts to many different online accounts, are all registered under your gmail address it is important to have that data whenever you need it. There are some services online that will backup your gmail but it is still stored on their servers. I feel a lot better if it was stored locally on my computers so that I have a copy of my emails whenever I need them. So this is how you do it.

Got Your Back

Got Your Back is a python app that will connect to your gmail via oAuth and suck down your messages in simple txt files. They have an *.eml extension but you can open all these files with a text editor and see the contents of your emails. It is fast and also does incremental backups. So after the initial backup of your account, Got Your Back will only download the messages that it doe not have instead of downloading everything each time you do a backup. I use a simple cron job to get all my new messages everyday, automatically without having to think about it.

Installing

You can get to the download page of got your back here. They have downloads for Unix and Windows systems. I have an server setup running debian that handles all my account backups so this tutorial will focus on running gyb (got your back) on Unix systems. So download the source package or use subversion to download the source onto your computer:

svn checkout http://got-your-back.googlecode.com/svn/trunk/ got-your-back-read-only

If you download the zip extract it to whatever folder you want and cd into the directory. Now because gyb uses oAuth to login to your account you will need to initially run this script on a computer with a desktop. Because I use a server over ssh to download my backups I need to first get the oAuth token on a computer with a web browser first.

Getting oAuth Tokens

So now that we are cd into the got-your-back directory run this command to authenticate and get an estimate of the size and number of messages that need to be backed up.

python gyb --email youremail@gmail.com --estimate

So make sure that you input your correct email address. If you are already logged into your email account on your computer and you are trying to backup a different account, you will need to logout first. So gyb will tell you that it will launch a browser, so just hit Enter and you will see a gmail page asking if you want to authenticate gyb to your gmail. Hit yes and then go back to the command line. No hit Enter again if it is still waiting and wait for it to make an estimate of your gmail's size and number of messages.

So now it should give you an estimate. I had 28k messages to download at 800mb. It does not download attachments. If you do an ls now you will see a new file. It will be a file with your email address and an extension of .cfg. This file now holds your oAuth token so if you are going to run the backups on a server over ssh, you will have to copy this file over to your server for it to work. So either zip, tar, or 7z the entire gyb directory with your email's config file in it and transfer it to your server. If you are just going to run it on the same computer then you don't have to do anything yet.

Inital backup

So now that you have the config file and gyb on whatever computer that you want to setup the automated backups on lets run the initial backup. First lets make the gyb.py file runnable.

chmod 755 gyb.py

So run this command to perform the initial backup.

python gyb.py --email youremail@gmail.com --backup

This will perform the initial backup so depending on how many messages your estimate was, this make take awhile.

Cron job for everyday backups

So now that we have an initial backup we will need to do an incremental backup everyday for all of the new messages that we receive. So lets add a cron job with crontab that will run gyb.py everyday. So remember where that file is on your computer because we will need it.

crontab -e

Then input this into your crontab file with the proper changes.

0 2 * * * python /home/michael/got-your-back-read-only/gyb.py --email youremail@gmail.com --backup

So what this does is make a backup at 3am everyday running gyb.py with your specified email. Easy and done. It will now do incremental backups of only your new messages and you will always be able to get that data because it is now backed up locally on your computer or server.

How to manage your website with git

2011-09-28T00:00:00Z

So this is much easier if you have a static site that is generated using pelican or jerkyll. Both of these apps are great for generating blogs out of markdown or other simple text markup language. I use pelican because it is built with python and that is what I develop in.

Local

So on your local machine you will need your basic folder structure setup and if you are not already using git to manage your files, now is the time to start. In terminal cd into your site dir and:

git init

This is create the local git repo that you need. Go ahead and commit the files that you already have done and now we can work on the server.

Server

Make sure that you have ssh access to your web server. The first thing you need to do is cd into your users main directory, not your www or directory that hosts the files that are displayed to your users and we will make a new directory:

mkdir cmsite.git
cd cmsite.git
git init --bare

This will create a bare git repo that will accept the files when you do a push to it. Now we will need to setup a hook that will checkout the latest push into the actual directory that hosts our content. So now we need to add the hook the the hooks directory in cmsite.git in my case.

vim hooks/post-receive
#!/bin/sh
GIT_WORK_TREE=/home/crosbymichael/crosbymichael.com git checkout -f

Make sure that this file is executable so do a quick:

chmod 755 hooks/post-receive

Final Steps

Now we just need to setup our new remote repo on our local machine. So cd into your local git repo and type:

git remote add website ssh://username@yoursite.com/~/cmsite.git

Now all your have to do is push to production when you need to update your site and git does all the rest.

git push website master

Because you can have more than one remote in git, you could have a test site and your production site, and another offsite repo for backups. You can run tests on your test site and then when everything is good to go you just push to production.

AVAudioPlayer Memory Leak

2011-09-11T00:00:00Z

This is a little bug in the AVFoundation framework. When using the AVAudioPlayer in you will get memory leaks when running in the iOS Simulator but not on devices.

It is a pain whenever you do most of your testing on the desktop during development and then start testing on device after you have a working app. So just remember if you are using AVAudioPlayer on the iOS Simulator and you are getting memory leaks that look like this:

Look at the type of leaks. note the responsible library is AudioToolbox not even AVFoundation, go ahead and profile it on a device and see if the leaks go away. If they don't, then it's something that you need to fix in your code, if they do then you are good to go.

My Creations

2011-09-01T00:00:00Z

CodeAssistant

CodeAssistant is a quick and simple desktop application to run code snippets. Have you ever been messing with a new API and you just want to execute one routine to see the output? Just throw your code snippet into CodeAssistant and it will run and display the output. It works on many languages, from C#, C, C++, Go, Python, Ruby, PHP, and it can be easily extended by adding XML config files. CHeck it out and download it for $5.00 here.

HTTP Assistant

HTTP Assistant is a cross-platform REST client for testing web services. You can check out all the features and purchase it for $5.00 here.

Encoder Assistant

This is another cross-platform app that will help you hash and encode text when working with web services. You can grab it here as well.

SVN to GIT and Extracting Individual Projects

2011-08-15T00:00:00Z

This post is still not formatting. I switched from worpress and this is a long post and have not had time to update it.

So here is the issue. You first started to use version control and because you were coding solo and the low learning curve, you started to use Subversion (SVN). It worked great for you because you had your own repository and you added all your projects to it and it was easy. Then you started to work with other people on projects and the very first commit that you made to your svn repo ended in conflicts. Not a good way to start. So you started to use more powerful version controle systems like GIT and now you wonder why you ever used svn in the first place.

However, you have a years worth of revision history sitting in your svn repository that you want out and placed into your shinny new git repo. You have already been working in your git repo for a few months, you just checked out the latest copy from your svn repo and added it to a new git repo, and now you have a branches and commits of all your recent work. It's time to convert your svn to git and unite your past revision history with the present. Lets get to work.

Video is at the bottom if this is confusing.

Problems:

SVN repo has many different projects in the one large repository

SVN has a year worth of revision history with commits from many different projects

You checked out the latest copy from svn and made git repositories for each of your projects

These individual project git repos now have many commits of their own and branches of your recent work

Goal:

Extract individual projects with only their complete revision history from svn and merge that with your current work and revision history in your individual git repos.

Lets look at our starting point. SVN revision history and GIT revision history.

Last svn file change and git's last change. You can see that the files are different. SVN on left and GIT on right.

First we need to suck our svn repo down to a machine where we have shell access. I had a large 4GB svn repo with 25+ projects in it. It had 280 commits from various projects. ie: commit 278 from Encoder Assistant changes, commit 279 from HTTP Assistant changes, and commit 280 from another project. So the svn repo was tracking overall revision history for the entire repo and not individual projects. This was a mistake on my part, I was new to svn and my hosting provider offered free private svn repos to use. So it was a free way for me to have revision history and also offsite backup of my code.

To grab your svn repo from various locations you need to first make a new folder and cd into it. Yes, you will have to use the terminal.

git svn clone url-to-repo

(not the working copy, the actual svn repo)

Local file:///Users/michael/svnrepo

Http http://yoursite.com/svnrepo

SSH ssh://username@yoursite.com/svnrepo

So to cone I will do this:

[highlight]git svn clone file:///Users/michael/Development/Video/WIP/SVNTOGIT/[/highlight]

If you get an error with the wrong url to the svn repo then you will have to delete any folders in your current folder before you can try again.

We can see by the terminal output and then doing an ls that it received 3 revisions(commits) and created a new folder of the svn repo name. Now cd into it and you can see that there are two projects exactly in the same as the svn repo. If we want to check to see if git cloned the full revision history we can type:

[highlight]git log[/highlight]

See it's all there.

If you only had one project in your svn repo then you can skip the next step.

Now we need to extract one project out of this new git repo. We only want one project per repo in our new setup. To do this we need to do a filter.

Lets extract the contents of ProjectOne to the root dir of this new git repo. (If you have more than one project you will want to clone this git repo so that you can keep do this over and over again for each project.)

[highlight]git filter-branch --subdirectory-filter ProjectOne/ -- --all[/highlight]

Now you can see that it rewrote the branch and by doing an ls you can see the contents of ProjectOne are now in the root of your repo. There is only one code.txt file from this repo.

Lets do a git log again and see that only the ProjectOne revision history is extracted with the ProjectOne files.

You can see only two commits, the init and Added more code. These are only the commits that affected ProjectOne. Nothing for ProjectTwo.

Lets make a new dir to keep the same folder structure as your CURRENT git repo that we have been working in.

[highlight]mkdir ProjectOne[/highlight]

Now we need to move the files back into this project one folder.

[highlight]mv * ProjectOne [/highlight]

This will give you an error about moving ProjectOne into it's self but it works and moves every other file and folder into our new ProjectOne folder. You can cd into it and check for your self.

Now we need to commit the changes that we have just done to the repo.

[highlight]git add .[/highlight]

This adds and removed all changes that we have made.

[highlight]git commit -a[/highlight]

This will commit all the changes we made. Give it a message for the commit.

Now we need to merge the revision history, branches, etc from our CURRENT repo with the new git repo's svn revisions. We are still in the same folder of the new git repo that we have created.

Lets create a new branch to add our current work into. We will now refer to our git repo with the svn history as our OLD repo and the git repo with the CURRENT changes as the CURRENT repo. This will get confusing to pay attention.

To create and new branch and check it out at the same time we do:

[highlight]git checkout -b current-repo[/highlight]

Now we need to pull in the changes from the CURRENT repo into the current-repo branch of the OLD repo (the one with the svn revision history and extracted ProjectOne)

git pull pathtorepo

So in my example

[highlight]git pull ~/Development/Video/WIP/GitCopy/svn-repo/[/highlight]

So if you notice, it pulled the CURRENT repo into the OLD repo and when it tried to merge the changes into the branch it received a conflict. Don't worry, resolving merging conflict's in git is easy. If you are on OSX like me or have a merge tool setup, ie: FileMerge all you have to do when you receive a conflict on a merge it type:

[highlight]git mergetool[/highlight]

It will tell you what file you are going to be working with and when you hit enter, it will launch the tool so you can work on the conflict in the file.

Now we can see the code on the left is from the last commit from our svn repo and the code on the right is from our CURRENT git repo with all the changes. Now you just go down through the differences and select the left or right to keep. After you are finished, I am just going to keep all the right side changes, save the file and close your merge tool.

Now if we type:

[highlight]git status [/highlight]

We can see that we have the modified file to commit. If you have untracked files or if you have files with a .orig extension, conflicted files, add or remove those before committing.

[highlight]rm ProjectOne/code.txt.orig[/highlight]

Now commit

[highlight]git commit -a[/highlight]

You can see that git has already added a comment for you stating that this was a merge from another repo and also adds the repo path or url. Write and quit your comment tool and it's done. If we type in:

[highlight]gitk[/highlight]

This will bring up an gui to show our git repos history.

Now you can see that we not only have the Init and Added more code commit's that were originally from the svn repo but also all the commits from the git project added to our new branch. If you want to do another merge to merge the changes from the current-repo branch back into the master branch for the project you can do that and now we can finally call this repository our CURRENT, full revision history, repository of awesomeness.

If you were able to follow along, your welcome.

Setup your own Git Server over SSH

2011-08-08T00:00:00Z

The thing about dvsc that makes me nervous is that all the code and revision history is on your local computer. This makes it very fast when committing, branching, and merging but what if you have a drive failure or worse? All your code is gone if you don't have backups. You have many different ways to backup your git and mercurial repositories. Especially this open-source project that I am working when it is done: DVCS-Backup

If you want to work with others on a project and need a git server setup to be an intermediary, then this is how you can do it as well as having an offsite backup copy of your repos.

First find a server that you are going to use. Setup keyless ssh access to it following my tutorial here

After that ssh into your server and make a new user that will manage the repos if sharing or just user your own account if it's just for you.

Install git on your server. I run debian so it's a simple:

apt-get install git-core

To get git installed.

Now create a new folder for the repo that you want to push to. I have many repos on my development computer that I work from so we are going to push an already populated repo to the server.

mkdir somereponame.git

You add the .git to the end of the folder name to that it's easy to distinguish between other folders. .git means this is a git repo.

Now cd into it and create a bare repo.

git init --bare

Now that server part is done. Easy.

Now go to your local dev computer and cd into the repo that you want to push to the server. Make sure that you don't have a remote origin already setup in this repo. To clear it out just:

git remote rm origin

Now we need to set the origin to your remote server.

git remote add origin ssh://sshusername@yourserver.com/~/myrepo.git

Now your done. Time to push. If you have more than one branch, you can either push one or all the branches to the server. I just push all of them because I want the server to have a full backup or my local repo.

git push origin --all

But for inividual branches:

git push origin branchname

That's it. Now when you want to get data from the server just do a pull.

git pull origin branchname

If you want to clone the repo to a new folder:

git clone ssh://sshusernane@yourserver.com/~/myrepo.git

Or you can make a new dir, cd into it and create a new local repo, set the origin to your server and pull specific branches.

mkdir newrepo
cd newrepo
git init
git remote add origin ssh://sshusername@yourserver.com/~/myrepo.git
git pull origin branchname

Flex 4.5 vs Pure AS3

2011-08-07T00:00:00Z

I have built a few applications for Android using the Flex framework. It is a great framework and I can build, test, and publish apps very quickly. It takes productivity to the max.

However, I have noticed that some of my apps take a noticeable time to launch on my device for the first time. Even simple apps like QuickIP where all it does it check for external and wifi IP address. So I ran a few comparison tests between pure AS3 projects and Flex projects to see three things. Startup time, file (swf) size, and memory usage. All important on a mobile device. Here are the results.

Mobile Pure AS3

Startup Time: 99ms
SWF Size: 1.5kb
Memory Usage*: 6.328mb

Mobile Flex mxml

Startup Time: 653ms
SWF Size: 2.839mb
Memory usage*: 21.397mb

Memory usage is a sample taken after 5 seconds of the app being launched. All the apps are just code to test startup time and memory usage after 5 seconds. The Flex mobile project was a blank project. Each project launched a view that was run on my desktop, not on a device.

So how does this compare?

AS3 over Flex

AS3 launches in 15.16% of the time it takes a Flex project
AS3's file size is .05% of the Flex projects size
AS3 uses 29.58% of the memory that a Flex project uses

As for the desktop, the numbers are closer to the same. Remember that I launched the mobile project on my desktop rather than on a device so the tests have a consistent test environment.

Desktop Pure AS3

Startup Time: 96ms
SWF Size: 1.5kb
Memory Usage*: 6.184mb

Desktop Flex mxml

Startup Time: 556ms
SWF Size: 2.114mb
Memory usage*: 19.021mb

As you can see the desktop Flex project is a little lighter on the resources than a comparable Mobile Flex app. I even did some tests on the desktop such as only using spark and no mx. I have even changed the WindowedApplication to just Application (no visible window as launch) and was able to get the startup time down to 450ms and only using ~15mb of ram. I tried to do tests with a Flex project's initial application file as a .as instead of a .mxml file but they were unsuccessful in launching a visible window. However just launching an initial .as instead of .mxml did make the Flex project have the same ~99ms startup time, ~1.5kb swf size, and ~6mb memory footprint.

So the question is... Is the speed of productivity in building an app on the Flex framework worth the loss of speed running the app? I guess it depends on what type of app you are building and what your user base is. The big difference between the Flex framework and pure As3 is components. You have to build all the components your self in pure AS3 projects. But how will productivity change for pure As3 projects after you already have a library of components built? What if you make all your assets and skin the Flex components anyway with bitmap or fxg assets? Do the components matter then?

These are the questions that I am asking my self currently. I am going to rebuild my QuickIP app with pure AS3 since it has all custom bitmap assets then compare the performance on my Android device and see if the time of development makes a noticeable jump in performance.

Setup Password-less Login over SSH

2011-08-07T00:00:00Z

Everyone needs a fast instance in a datacenter to work with. Amazon's EC2 or any other VPS provider will give you access. Sometimes you need to transfer files around fast or do some data processing. Having a fast instance will help you get your job done faster.

I have a tutorial that I needed to upload to an FTP server and do an FXP copy. The file is resting on another FTP server form a basic hosting provider. I do not have FXP access. So I launched an instance on EC2. Logged in via ssh. wget the file from my hosting server. Then connected to the marketplace FTP and uploaded the file. Easy, fast, and all the transferring was done on someone else's bandwidth.

To make this easier here is how to setup ssh to use a ssh key file instead of a password. This has more advantages than just making it quicker to ssh into my instances. You can use this to setup automated backups using rsync.

How To:

On your local machine you need to jump into terminal for this. Go to your home dir and make a new .ssh folder if it does not exist. To see if it does exist just type:

ls -a

If not exist then:

mkdir .ssh

Now cd into it:

cd .ssh

Now we need to generate an rsa key file:

ssh-keygen -t rsa

Do not enter a passphrase, just leave it blank.

Now you need to ssh into your instance or remote computer and make an .ssh folder in your user home folder. Same as above.

Now cd into and make a new file called:

authorized_keys

In this file, paste the contents of your local computers newly generated id_rsa.pub file into it.

Now, still on your remote computer or instance, you need to change the permission of the authorized_keys file to 644:

chmod 644 authorized_keys

All done. Exit your ssh session and login again and you will notice that you don't have to enter a password now.

I suggest changing your password to something really long and secure since you won't be using it anymore, unless logging into to it from a different computer.

Michael Crosby

Favorite Tech

Vagrant and Puppet

Editors

Redis

Backbone.js and Bootstrap

CoffeeScript

Microsoft land

Golang JSON RPC

Go Lang

Abstractions

No Classes

Why Go

Concurrency

Compiled binary

Testing and Packages

Go Doc

Parting words

SecureString How To

Adding your data

Getting your data back out

F# Url Shortener

To the code!

Functions

Complete Code

Infrastructure - Part 1

Servers

Ajenti

Links

Usage Patterns

Usage by App

Hours by day

Development Infrastructure

Public Site

Build server

Server Software

Node1

Node2

web.py server setup

What you will learn:

Ubuntu Server

Objective - C LINQ

Code

Select

Objective C Design Patterns - Factory

Factory Pattern

Static Factory

CodeAssistant

Future Projects

Setup Ruby on Rails with Nginx and Unicorn

Requirements

Installing Debian 6

Install Package Selection

Application Install

Gems

Javascript engine

Testing rails

Nginx

User group and dir

Nginx Config

Create the test app

Unicorn init.d

Multithreading in C#

MVVM-Async

Async

Updating the UI

Queues

Timed Execution

Final

tCal - iCal events in terminal

Usage

Bash Informational Startup

End of my .bash_profile

Compile Vim for Python

Update

Python talking to Arduino

The Problem

Python Code

How to backup gmail the right way

Got Your Back