Linux Containers, LXC, FreeBSD Jails, VServer…

These days containerization of work, applications and storage on systems has become a hot topic. Not to say it wasn’t before, but it’s got a boost from the cloud computing segment of the industry. With that I felt the need to write up what I’ve discovered of the history in this industry so far. I’d love feedback and corrections if I’ve got anything out of order here or if – heaven forbid – I’ve got something wrong.

What are Containers?

Before I get into what a container is, it is best to define what operating system-level virtualization is. Sometimes this is referred to as jailed services or apps running in a jail.

This level of virtualization often provides extremely similar functionality as a VMware, Virtual Box or Hyper-V virtual server would provide. The difference however is primarily around the idea that the operating system-level virtualization actually runs as a service, usually protected, that runs apps as if it were an operating system itself.

So what’s a container?

Linux Contains is a feature that allows Linux to run a single or more isolated virtual systems that each have their own network interfaces, computer process threads and namespaces, user namespaces and states.

One of the common abbreviations for Linux Containers you’ll see is LxC. There are however many distinct operating system-level virtualization solutions.

  • Open VZ – this technology uses a single patched Linux kernel, providing the ability to use the architecture and kernel version of the system that is executing the container.
  • Linux V-Server – this technology is a virtual private server implementation that was created by adding operating system-level virtualization to the Linux kerne. The project was started by Jacques Gélinas. It is now maintained by Herbert Pötzl of Austria and is not related to the Linux Virtual Server project. The server breaks things into partitions called security contexts, within that is the virtual private server.
  • FreeBSD Jail – This container technology breaks apps and services into jails.
  • Workload Partitions – This is a technology built for AIX, introduced in AIX 6.1. Workload Partitions breaks things into WPARs. These are software partitions that are created from the resources of a single AIX OS instance. WPARs can be created on any system p (the new old thing, was the RS/6000 tech) hardware that supports AIX 6.1 or higher versions. There are two kinds of WPARs, System WPARs and Application WPARs.
  • Solaris Containers – is a container tech for x86 and SPARC systems. It was first released in February 04′ for Solaris 10. It is also available in OpenSolaris, SmartOS and others as well os Oracle Solaris 11. The Solaris container combines resource controls in seperations referred to as zones. These zones act as completely isolated virtual servers within a OS.

What is so great about a container?

Ok, so I’ve covered what a container is. You’re probably asking, “so what do I do with these containers?” There are a number of things, for starters speed is a huge advantage with containers. You can spool up entire functional application or service systems, like an API facade or something, in seconds. Often times a container will spool up and be ready in less than a second. This provides a huge amount of power to build out flexible, resilient, self-healing distributed systems that otherwise are just impossible to build with slow loading traditional virtual machine technology.

Soft memory is another capability that most containers have. This is the capability of being allocated, or being allocated and running, in memory. As one may already know, if you run something purely out of memory it is extremely fast, often 2-10x faster than running something that has to swap on a physical drive.

Managing crashing services or damaged ecosystem elements. If the containers are running, but one gets hit with an overloaded compute ask, software crashes on it, or one of the many receive some type of blocking state like a DDOS of sorts, just reboot it. Another option is just to kill it and spool up and entirely new instance of the app or service in a container. This ability really is amplified in any cloud environment like AWS where a server instance may crash with some containers on it, but having another instance running with multiple containers on it is easy, and restarting those containers on running instances is easy and extremely fast.

Security is another element that can be assisted with container technology. As I alluded to in the previous point above, if a container gets taken over or otherwise compromised, it’s very easy to just kill it and resume one that is not compromised. Often buying more time to resolve the security concern. Also, by having each container secured against each other container, controlling a container does not result in a compromised physical machine and operating system. This is a huge saving grace when security is breached.

Container Summary

Containers are a hot ticket topic, for good reason. They provide increase management of apps and services, can utilize soft memory, increase security and they’re blazing fast. The technology, albeit having been around for a good decade, is starting to grow in new ways. Containers are starting to also become a mainstay of cloud technology, almost a requirement for effective management of distributed environments.

Next up, I’ll hit on Docker tech from DotCloud and Salomon Hykes @solomonstre.

For now, anybody got some additions or corrections for this short history and definitions of containers?  🙂

Could you answer a few questions about hackathons for me?

Hacking Hackathons
Hacking Hackathons

Recently I was talking to Jeff Martens of CPUsage about Hackathons. A few questions came up about what would make a hackathon fun, exciting and worth your time. Here’s a few questions, that I’d love to have a conversation about…

  • What type of hacking would you or do you prefer to do at a Hackathon? Hardware Hacking? Software Hacking?
  • How important are the prizes?
  • What should the prizes or rewards be?
  • How many people should attend that would actually make the Hackthon fun?
  • Have you attended, and if so, what was the coolest thing about the hackathon to you?
  • Are hackathons better as 1 day or 2 days or more?
  • Should hackathons be during the week or the weekend?

…and as you can imagine, there’s a million other questions. What’s your take on hackathons? Got a few thoughts? Throw down a comment or two, much appreciation for your thoughts, next beer is on me!  😉

Becoming a Better Programmer… Step One, Build a Course

Often a programmer sits back and reflects on, “how can I get better at my job?” A number of months ago I found myself sitting with that same reflection. One of the many ideas that popped into my head was to put together focused material around a topic. Training material, a blog series or something else. I wasn’t sure just yet what it would be, but I knew that was one of the many things I wanted to do. Fast forward a few months.

Fellow Tier 3 coder Richard Seroter, who I’d known for some time via his blog “Richard Seroter’s Architecture Musings” and I had a conversation about what we do, respectively, to keep our skills honed in tech. He brought up he’d be putting together material for Pluralsight for a while now and enjoyed that. It struck me as something I’d like to do too, considering my past reflection, so he made an intro. The rest is history!

Pluralsight Author, Achievement Reached

I’m now a Pluralsight Author (my author’s page) with my first course on Riak Fundamentals. I’m now working on a second course, on Docker Fundamentals. I’ve got a lot more in the queue after that too, so I hope to keep producing a lot of material on everything from the big languages these days like Javascript, Java, Ruby and C# to lesser knowns like Erlang, Go and maybe some others to boot.

For now, check out my Riak Fundamentals Course and some of the other great material that Pluralsight has available. They’ve just acquired TekPub, Peepcode and a number of other companies too, so when you subscribe you don’t just get all the Pluralsight material but also access to all the material at these excellent course creating companies! In the coming days and weeks I’ll have some reviews of other courses I enjoyed. As always, enjoy, subscribe to my blog, subscribe to Pluralsight and cheers!

References:

Getting Docker Installed on Ubuntu 12.04 LTS

A few days ago I posted the blog entry “Using SSH Locally to Work With Ubuntu VM + VMware Tools Installation via Shell“, it was related to getting a clean Ubuntu Server install running with VMware Tools and so that I could use it as a hosted instance. Simply put, being able to SSH into it just as I would a hosted AWS or Windows Azure Ubuntu Server image. Once I had the default virtual machine running 12.04 LTS I went about another installation that is needed to run Docker. Docker will have issues with anything pre-3.8 kernel. Running the command below will show that kernel 3.5 is the current kernel in 12.04 LTS.

[sourcecode language=”bash”]
apt-cache search linux-headers-$(uname -r)
[/sourcecode]

To update to the 3.8 kernel I ran the following command and then rebooted.

[sourcecode language=”bash”]
sudo apt-get install linux-image-generic-lts-raring linux-headers-generic-lts-raring
sudo shutdown -r now
[/sourcecode]

With the reboot complete, I checked the kernel version again and 3.8 was installed successfully.

[sourcecode language=”bash”]
@ubuntu:~$ apt-cache search linux-headers-$(uname -r)
linux-headers-3.8.0-33-generic – Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
[/sourcecode]

To get Docker installed (as of 0.6) run the following command.

[sourcecode language=”bash”]
sudo sh -c "wget -qO- https://get.docker.io/gpg | apt-key add -"
sudo sh -c "echo deb http://get.docker.io/ubuntu docker main\
> /etc/apt/sources.list.d/docker.list"
[/sourcecode]

Next update the sources, then install lxc-docker.

[sourcecode language=”bash”]
sudo apt-get update
sudo apt-get install lxc-docker
[/sourcecode]

To verify that docker is installed I executed the following command and…

[sourcecode language=”bash”]
sudo docker run -i -t ubuntu /bin/bash
[/sourcecode]

…see similar results just after issuing the command.

[sourcecode language=”bash”]
Unable to find image ‘ubuntu’ (tag: latest) locally
Pulling repository ubuntu
8dbd9e392a96: Download complete
b750fe79269d: Download complete
27cf78414709: Download complete
[/sourcecode]

After that displays then I typed exit to leave docker. I now have a running version of docker on the Ubuntu 12.04 LTS instance ready for testing and hacking with docker.

Using SSH Locally to Work With Ubuntu VM + VMware Tools Installation via Shell

I do a lot of work with Ubuntu, 90% or so of that work is from an Ubuntu instance. Often that instance happens to be a local VM running in VMware Fusion (or sometimes Virtual Box). Often I’ll start with a base server image which isn’t entirely setup for SSHing into the instance. These are the steps to get that installed and ready to go.

First install the image, in this particular situation I’m using the Ubuntu 12.04 LTS Server image.

Ubuntu 12.04 Server. Click for full size image.
Ubuntu 12.04 Server. Click for full size image.

That will take a few minutes to install, on machines these days I’ve experience just about 8-15 minutes. There are a million other options to do this too, such as starting with a clean Ubuntu image using Vagrant, which takes all of about 1-2 minutes, sometimes a bit more if you have to download the image. But either way, get one built and running.

Installing Ubuntu using VMware Fusion. Click for full size image.
Installing Ubuntu using VMware Fusion. Click for full size image.

Once the image is installed, login and install openssh-server and openssh-client.

[sourcecode language=”bash”]
sudo apt-get install openssh-server openssh-client
[/sourcecode]

Once that’s installed I pull up my IP address with ifconfig.

[sourcecode language=”bash”]
ifconfig
[/sourcecode]

The ifconfig command shows a lot of information regarding the network configuration associated with the various network adapters in the machine that it is executed on. In the image I’ve circled the local IP address that is assigned to the instance.

The local IP address using the ifconfig command. Click for full size image.
The local IP address using the ifconfig command. Click for full size image.

Now that you have the local IP of the instance, bring up a local terminal (in this case I’m on OS-X, but if you’re on Windows pull up Putty or on Linux or another *nix variant pull up a shell). In the terminal you can now enter the follow SSH command to log in from the local machine versus the running instance. This comes in handy when you want to treat the machine like an actual hosted machine somewhere, in which you wouldn’t be directly logged into the server.

[sourcecode language=”bash”]
ssh username@192.168.77.197
[/sourcecode]

Logged In.
Logged In.

Getting VMware Tool Installed

This assumes that you mount the installation files (aka the cdrom) via the built into mount option in the VMware Fusion menu.

Selecting 'Reinstall VMware Tools' to mount the installation files. Click for full size image.
Selecting ‘Reinstall VMware Tools’ to mount the installation files. Click for full size image.

Once that’s mounted, the machine is ready to install the tools on. However, there are a few other things to install just before installing these. First get the latest updates for apt-get with the update command.

[sourcecode language=”bash”]
sudo apt-get update
[/sourcecode]

Now install the latest gcc, make, kernel headers and other important tools.

[sourcecode language=”bash”]
sudo apt-get install gcc make build-essential
sudo apt-get install linux-headers-$(uname -r)
[/sourcecode]

In the above, everything can be put on one line, but I separated the linux-headers just for extra clarity. I can now via remote SSH on the local machine or directly into the virtual machine and run the following commands to install the VMware Tools.

[sourcecode language=”bash”]
sudo mkdir /mnttools
sudo mount /dev/cdrom /mnttools
tar xzvf /mnttools/VMwareTools-x.x.x-xxxx.tar.gz -C /tmp/
cd /tmp/vmware-tools-distrib/
sudo ./vmware-install.pl -d
[/sourcecode]

Finish everything up with a good reboot.

[sourcecode language=”bash”]
sudo shutdown -r now
[/sourcecode]

Now I have the VMware Tools installed and able to SSH remotely, giving me the ability to use the virtual machine as I would an actual hosted instance.