This document is more notes than a full-blown blog post. It should be treated as a continuous draft.

I've always been a big fan of full VMs, powered by Xen or KVM. Mainly KVM. Recently, I've had an issue with resources - my primary server consists of 64 GB DDR4 memory, and 4TB of storage. This was beefy enough for my needs - 48 VMs about 2GB ram each, and about 80GB of storage for each.

Frankly, 80% of all this could be run in an LXC container, so I moved it all to containers. Fast forward three months, and I've found myself burning file storage for countless OS images than I use. Searching for alternatives, I've always known about Docker, but never taken the time to learn it entirely, I use it for personal development, but never for anything more.

One thing I appreciate about Docker is they can be very lightweight, but my systems usually run Debian Jessie or Stretch, and Ubuntu 16.04 (soon 18.04). This is due to dependency pinning. let's say I'm building Qt applications targeting Ubuntu 16.04, I want a few CI runners to build it. I used KVM then LXC containers for this, and it worked fine, but if I build on 8 containers, I've now got 8 copies of the OS, and 8 copies of my compiled application. I'd much prefer to just dynamically spin images pre-created, runt he task, and remove it automatically once it's pushed to my release server (which, will remain a full KVM container).

Making the switch for everything was long, my current usage of my beefy server is 85% Memory consistently, I've got an entire /26 assigned for VMs, and some for my HAProxy to KVM and LXC containers.

Fast forward 2 weeks, I've made the switch. I've found, I don't need this beefy server anymore. I'm able to downsize from my $150/mo OVH server to a simple Soyoustart server, I'm taking a bandwidth hit (500Mbit to 250Mbit), but I'm OK with that. 32GB of memory is all I need to run my gear, and for any more beefy builds, I invested $150 into a R710 with 64GB memory, and 20TB of storage in a NAS, and this will run any beefy applications I need, the SYS server will remain as my production gear.

I've also spotted some nice additions, now I'm able to run my own private NPM proxy on the production host, and my customers software is downloaded directly from my private repository, allowing me to shield my software in a reasonable manner, and making deployments easier for me!

docker containers lab kvm lxc continous drafts

Mike

Senior Software Engineer, Labber, Sysadmin. I make things scale rapidly. Optimize everything.

Read More