Michael Crosby


F# Url Shortener

Every year I try to learn a new language. If you are a one trick pony you are not only limiting your self in opportunities but you are also limiting your thinking to the one paradigm that you know.

So this year I wanted to learn something very different from what I am used to in the OO world ( Python, C#, Objective - C). I decided to learn a functional language this year called F#. I looked at a few different functional languages ( Scala, Erlang ) before choosing F#. I felt that it would be the most practical language for what I do but still provide me with the paradigm shift I was seeking. It does have OO parts being part of the .NET family but the functional paradigm still dominates it's landscape.

Usually when I learn a new language I rewrite a program that I have already done before. This helps me to focus on the language and not worry about the problem at hand because I already know the solution. So this program is a url shortener that makes a request to the Google api via JSON. I have written this url shortener in Python, Go, Objective - C, and C# so I know what I need to do and I have a reference. So lets get started.

To the code!

The first thing we need to do to create this program in F# is to "open" our dependencies. So at the top of a blank .fs file add these lines:

#light

open System.Net
open System.IO
open System.Text
open System.Web.Script.Serialization

We need the Net package for the WebRequest, IO for a StreamReader, Text for Encoding, and the Web.Script.Serialization for the JavaScriptSerializer so that we can convert the JSON response into a type that we can use. You will have to add the System.Web.Extensions dll reference to your F# project to use the JavaScriptSerializer.

Next we need to create a new type that the JavaScriptSerializer will deserialize the response to. The response JSON looks like this:

{"id":"short url", "kind": "kind of request", "longUrl": "long url"}

We will have to make this type a class with a default constructor so that the JavaScriptSerializer can create the instance. The members of the type; id, kind, and longUrl will have to be mutable so that the serializer can set the values. By default all types in F# are immutable unless set to mutable.

type Response = class
    val mutable id : string
    val mutable kind : string
    val mutable longUrl : string
    new() = {id = ""; kind = ""; longUrl = ""}
end

So there is the Response type with the default constructor. Next I will create another type that will hold all the request data that is needed. This is more for convenience and not really needed.

type Request = { url: string; requestMethod:string; contentType : string; data : string }

Now lets get the data required and populate a Request type. We will receive the long url via the command line args and the other data such as request method and content type can be hard coded along with the url to the Google api.

let args = System.Environment.GetCommandLineArgs()
let request = {
    url = "https://www.googleapis.com/urlshortener/v1/url";
    requestMethod = "POST";
    data = args.[args.Length - 1];
    contentType = "application/json" }

Functions

We will need a few helper functions that can convert a string in to a byte array, read a Stream into a string, and deserialize the response string into our Response type.

let get_bytes (str:string) =
    Encoding.UTF8.GetBytes(str)

let read_to_string (stream:Stream) =
    use reader = new StreamReader(stream)
    reader.ReadToEnd()

let deserialize_response (stream) =
    let serializer = new JavaScriptSerializer()
    serializer.Deserialize<Response>(read_to_string stream)

These three functions are very straight forward and to the point. One issue that I have ran into is that using classes from the .NET framework often requires a type annotation to be added to the function parameter(s) type inference cannot be used with overloaded methods. It kinda sucks but it is what it is. You will also notice the "use" keyword in the read_to_string function. This is the same as the "using" keyword in C#. Other than that everything is pretty obvious to a .NET developer looking at this code.

Now we will need to create a WebRequest, add the default parameters, write our data to the request stream, and then read the response.

let add_params (webRequest:WebRequest) =
    webRequest.ContentType <- request.contentType
    webRequest.Method <- request.requestMethod
    webRequest

let write_to_stream (webRequest:WebRequest) =
    let stream = webRequest.GetRequestStream()
    let data = get_bytes (sprintf "{\"longUrl\":\"%s\"}" request.data)
    stream.Write(data, 0, data.Length)
    webRequest

let get_response (webRequest:WebRequest) =
    webRequest.GetResponse().GetResponseStream()

let print_response (o:Response) =
    printf "%s" o.id

Easy. One of the things that you will notice is the lack of a return keyword. This is because whatever is on the last line of a function is automatically returned. You can see that in the write_to_stream function, even though it takes a webRequest as an input parameter, I set it on the last line so it can be returned. In C# ( and other languages ) this is often known as a "Fluent api". This will also help us a little bit later when we chain all of these functions together to process the request.

You can also see the "<-" operator when I assign values from our request type to the webRequest. You do not use the "=" operator for member assignment.

Lastly, you may have noticed that I never created the actual webRequest in the above code snippet. That is because in the final function we will create the webRequest and chain all of these functions together.

let get_short_url() =
    HttpWebRequest.Create(request.url)
        |> add_params
        |> write_to_stream
        |> get_response
        |> deserialize_response
        |> print_response

So get_short_url is the final function that ties everything together. We create the webRequest and use it's value to chain all the remaining functions. Each function consumes the result of the previous function this way we can use the "|>" to chain them together. This is very LINQ like but taken to a whole new level in functional programming.

So now all that is left is to get_short_url when a user passes in a long url on the command line. For this we will use a pattern match in F#.

match request.data.StartsWith("http") with
    | true -> get_short_url()
    | false -> printfn "No url passed"

System.Console.ReadLine()

Because our Request type's data member is a string we can use the standard .NET string function "StartsWith" to see if the first chars are "http". Then by using the "match with" we can check for each boolean value to determine our control flow. If true, we will call our get_short_url function, if false, we will tell the user that they did not pass a url. If you are on Windows you will want to add the "Console.ReadLine" call at the end of the file or the command prompt will exit without you seeing your results.

Complete Code

#light

open System.Net
open System.IO
open System.Text
open System.Web.Script.Serialization

type Response = class
    val mutable id : string
    val mutable kind : string
    val mutable longUrl : string
    new() = {id = ""; kind = ""; longUrl = ""}
end

type Request = { url: string; requestMethod:string; contentType : string; data : string }

let args = System.Environment.GetCommandLineArgs()
let request = {
    url = "https://www.googleapis.com/urlshortener/v1/url";
    requestMethod = "POST";
    data = args.[args.Length - 1];
    contentType = "application/json" }

let get_bytes (str:string) =
    Encoding.UTF8.GetBytes(str)

let read_to_string (stream:Stream) =
    use reader = new StreamReader(stream)
    reader.ReadToEnd()

let print_response (o:Response) =
    printf "%s" o.id

let deserialize_response (stream) =
    let serializer = new JavaScriptSerializer()
    serializer.Deserialize<Response>(read_to_string stream)

let add_params (webRequest:WebRequest) =
    webRequest.ContentType <- request.contentType
    webRequest.Method <- request.requestMethod
    webRequest

let write_to_stream (webRequest:WebRequest) =
    let stream = webRequest.GetRequestStream()
    let data = get_bytes (sprintf "{\"longUrl\":\"%s\"}" request.data)
    stream.Write(data, 0, data.Length)
    webRequest

let get_response (webRequest:WebRequest) =
    webRequest.GetResponse().GetResponseStream()

let get_short_url() =
    HttpWebRequest.Create(request.url)
        |> add_params
        |> write_to_stream
        |> get_response
        |> deserialize_response
        |> print_response

match request.data.StartsWith("http") with
    | true -> get_short_url()
    | false -> printfn "No url passed"

System.Console.ReadLine()

So that's it. I am still very fresh to F# and functional programming in general so my code does not look very elegant. It feels a little verbose working with the .NET classes in a functional language but hopefully I will learn more and be able to remove some of the excess and make this program more idiomatic to functional programming.

comments powered by Disqus