Containers in the Data Center

What are containers?

Containers have invaded the data center, pushed there by developer insistence. Containers promise to deliver the process isolation and packaging of a virtual machine together with a developer-friendly build process. The container hypothesis says, “I’ll never again have to ask if production matches QA.”

Are containers VM?

Containers in the data center act a lot like virtual machines in the cloud (see Virtual Machines in the Cloud). Any individual container only has a short-lived identity. As a result, it should not be configured on a per-instance basis. This can cause interesting effects with older monitoring systems that need to be reconfigured and bounced every time a machine is added or removed.

A container won’t have much if any, local storage, so the application must rely on external storage for files, data, and maybe even cache.

The challenge with containers

The most challenging part of running containers in the data center is definitely the network. By default, a container doesn’t expose any of its ports (on its own virtual interface) on the host machine. We can selectively forward ports from the container to the host, but then we still have to connect them from one host to another. One common pattern that’s developing is the overlay network. This uses virtual LANs (VLANs), see Virtual LANs for virtual machines, to create a virtual network just among the containers. The overlay network has its own IP address space and does its own routing with software switches running on the hosts. Within the overlay network, some control plane software manages the whole ensemble of containers, VLANs, IPs, and names.

A close second for “hardest problem in container-world” is making sure enough container instances of the right types are on the right machines. Containers are meant to come and go. Part of their appeal is their very fast startup time (think milliseconds rather than minutes). But that means container instances will be like quantum foam burbling across all our hosts. Manually operating containers would be absurd. Instead, we delegate that job to another bit of control plane software. We describe our desired load out of the containers, and the software spreads container meringue across the physical hosts. The control software should know something about the geographic distribution of the hosts as well. That way it can allocate instances regionally for low latency while maintaining availability in case we lose a data center.

It seems natural that the same software should schedule container instances and manage their network settings, right? Solutions for running containers in data centers are emerging. None are dominant at this time, but packages like Kubernetes, Mesos, and Docker Swarm are attacking both the networking and allocation problem. Whichever one solves this problem first will be able to truly claim the title of “operating system for the data center.”

Virtual LANs for Virtual Machines

As if there weren’t enough ways for a packet to hit a pocket on a socket on a port, we’ve got virtual LANs (VLANs) and virtual extensible LANs (VXLANs) to contend with. The idea of a VLAN is to multiplex Ethernet frames on a single wire but let the switch treat them like they came in from totally separate networks. The VLAN tag is a number from 1 to 4,094 that nestles into the physical routing portion of the header.

Every network we encounter will support VLANs. The operating system that runs a NIC can create a virtual device assigned to a virtual LAN. Then all the packets sent by that device will have that VLAN ID in them. That also means the virtual device must have its own IP address in a subnet assigned to that VLAN. VXLAN takes the same idea but runs it at layer 3, meaning it’s visible to IP on the host. It also uses 24 more bits in the IP header, so a physical network can have more than 16 million VXLANs riding its wires. At one time this was all the province of network engineers pulling cables around the data center.

Virtualization and containers increasingly rely on software switches to handle dynamic updates. It will be common to see software switches running on the hosts, presenting a complete network environment to the containers that does the following:

• Allows containers to believe they’re on isolated networks
• Supports load-balancing via virtual IPs
• Uses a firewall as a gateway to the external network

While this technology matures, our container systems have to provide their own load- balancing and need to be told which IP addresses and ports their peers are on.

Designing apps for containers

When we design an application for containers, keep a few things in mind. First, the whole container image moves from environment to environment, so the image can’t hold things like production database credentials. Credentials all have to be supplied to the container. A 12-factor app handles this naturally. If we’re not using that style, think about injecting configuration when starting the container. In either case, look into password vaulting.

The second thing to externalize is networking. Container images should not contain hostnames or port numbers. Again, that’s because the setting needs to change dynamically while the container image stays the same. Links between containers are all established by the control plane when starting them up.

Things to remember when working with containers

Containers are meant to start and stop rapidly. Avoid long startup or initialization sequences. Some production servers take many minutes to load reference data or to warm up caches. These are not suited for containers. Aim for a total startup time of one second.

Finally, it’s notoriously difficult to debug an application running inside a container. Just getting access to log files can be a challenge. Don’t even bother trying to figure out why some socket is being held open for too long. Containerized applications, even more than ordinary ones, need to send their telemetry out to a data collector.