Michael Crosby


Development Infrastructure

Having servers that automate much of your development process is cool. Entities living out on the web just waiting to process any task that you throw their way.

I currently have two servers in my cluster that run my network. Everything from my public web sites, to mongodb and Postgresql, to private git and build servers. Why have all of this? Because it makes me happy when I push changes to my git server and within seconds the project is built, unit tests ran, and a report is emailed to me.

Servers

Public Site

I run a Python stack for my web server. Python 2.7 with django as the framework running on-top of lighttpd. Lighttpd also acts as the proxy server for the rest of the services that run on the cluster. Postgresql is the datastore for my sites. I still run mongodb on the server for quick, scheme-less storage.

On Node1 is my git repositories via Stash. Stash is not necessary but it provides a nice UI for managing repositories and users. I could do all of this via ssh but for $10 Starter License it is worth it. You can easily browse and see diffs on the web as well.

The only drawl back to Stash is that is runs on-top of Java and hogs RAM. The Stash instance consumes about 700-900MB of RAM for not doing much. The other memory hog is a minecraft server that I run. Minecraft consumes about 2GB+. This server has to be robust because if the minecraft sever goes down, I will have some angry users texting me until it's back online.

Build server

I run Jenkins-Ci as my build server. I have tried Bamboo for a week but the license was too restrictive for my needs, ( not enough jobs on the starter license ). For my python projects Jenkins has many nice plugins.

Even though the build server is running on Debian it can build both Python and .NET projects thanks to mono ( xbuild ). I am having some issues running nunit tests for a non-root user. I have not been able to figure out what the issue is yet so my .NET projects do not have automated tests.

Server Software

Both servers run Debian and are administered via ssh, no passwords just ssh keys. Each node does it's daily backup to the other with ~3day off-site dumps to Amazon S3.

Node1

  • Debian 6
  • lighttpd
  • git 1.7+ ( built from src )
  • django
  • mongodb
  • postgresql
  • sun-java
  • stash
  • Amazon S3 as CDN for content

Domains Hosted * CodeAssistant

Node2

  • Debian 6
  • sun-java
  • jenkins
  • mono ( built from git repo )
  • git 1.7+( built from src )

Future Changes Currently my blog crosbymichael.com ( this site ) is hosted on dreamhost. Also HTTPAssistant is a wordpress site hosted on dreamhost as well. My dreamhost service expires Feb. of next year so I will be moving both of these sites to Node1. I will replace wordpress with a static one page landing page. Hopefully I will have HTTPAssistant 2.0 done and replace that ugly thing.

comments powered by Disqus