Running Marathon and Mesos with Panamax
Technology Overview
Panamax is a new tool that allows you to manage multiple Docker containers and to link them together. In this post, I’ll talk about creating a Panamax template which will allow you to run Marathon and Mesos in Docker containers. Mesos is a cluster manager, which allows you to run many jobs in a fault-tolerant manner. It can scale to thousands of machines and is well suited for running large jobs like Hadoop or running many different services in a microservice architecture. Marathon is a Mesos framework which provides a UI for scheduling jobs on Mesos. Marathon and Mesos both rely on a distributed application called Zookeeper to store configuration information. Panamax is very helpful in wiring together Marathon, Mesos masters, Mesos slaves, and Zookeeper instances.
Running Panamax
Panamax has some great installation instructions. Locally it depends on Vagrant and VirtualBox to create a CoreOS instance on which to run the Docker containers. I got a bit hung up on running it for the time and the VM wouldn’t start. I debugged this problem by opening the VirtualBox UI and running the VM manually. It turns out that I didn’t have virtualization extensions turned on in my BIOS on this computer yet, so I got the error message “VT-x is disabled in the BIOS.” Most computers have VT-x disabled by default as a security precaution, so if you’ve never turned VT-x on, you’ll have to do so.
Creating a Panamax application
The first step of creating a Panamax application is to find Docker containers to use. This part was trickier than I imagined given that this was my first time using Docker. I first tried to use thefactory/marathon Docker image. However, it turned out that the version they published did not match what was in the Docker description because the DockerHub automated build didn’t build one of their commits, and so Marathon wouldn’t actually run. I filed a bug on this issue and it has since been fixed, so it would be a great image to try again. It’s always good to review the docker images you use. E.g. I ended up using the redjack/mesos-master and saw that it was doing some of its software installation over insecure HTTP, so I sent them a commit that they merged to change it to HTTPS. I also saw that it was using Ubuntu 14.04, but using the Mesos install for 12.04, so I also sent a pull request to have it use the correct Mesos install and upgrade it to Mesos 0.20 at the same time.
One problem with the way I set things up was that the initial download of all the images takes a long time. I used images from a few different sources and they all used slightly different base OS images. They’re quite big nearing 1GB each and need to be downloaded. If they used the same base file then it’d only have to be downloaded once. Now that the issue the thefactory images is fixed, it’d probably be nice to try to give those images another shot in order to speed up usage of the Panamax template.
One of the things you’ll have to figure out is how to pass configuration information to your docker containers. I passed some command line flags directly in the Docker run command. Another great strategy is to run services with wrapper script that reads config from environment variables as is done in this script in the CenturyLink MySQL Docker image.
Running the template
You can find my template by searching for “Marathon on Mesos and Zookeeper” from the Panamax Contest Templates. It has some great instructions for getting started, so I won’t rehash them here. After the various images are up and running and you’ve set the required settings, you should be able to see a Marathon screen like the following:
Things to watch out for
Panamax seems to struggle with being disconnected from the internet while downloading an image, so be sure you have time to wait for your downloads to complete. As long as you’re plugged in and not going anywhere you shouldn’t have any problems. The other issue I had was a hard time saving my Panamax template because it wasn’t dealing well with GitHub accounts with lots of repos. That issue has already been fixed, which is evidence of how quickly this project is moving. I also wasn’t sure if it was possible to test local Docker images as part of a Panamax application, so it seems like you’ll want to publish any images you plan to use.
You’ll also have to be careful to create good documentation for your Panamax templates and to use templates with good docs. I saw that someone else posted a Mesos template, so I tried it out to see how it would compare to mine, but was unable to run it. I thought for awhile that it was broken and wouldn’t work, but I think now that it’s probably a case of missing documentation instead. However, those missing docs could cause hours of debugging. Panamax is really easy to use and has a nice UI, but there’s still technology under the covers that has to be configured correctly when using it.
Future improvements
The thing I’d most like to see change is for Marathon to offer better authentication and authorization support. I’ve submitted a pull request to the Chaos Web Framework, which was created for use by Marathon and Chronos, to make this possible.
Marathon on Panamax template interest
This blog post was mentioned in the CenturyLink Labs newsletter. I tweeted about this template and it was favorited or retweeted by several folks including Marc Averitt (Managing Director of Okapi Venture Capital) and AllThingsMesos. Ross Jimenez (Director of Software at CenturyLink Labs) tweeted as well and was retweeted and favorited by several folks including Florian Leibert (Founder of Mesosphere), the Panamax Project, and Lucas Carlson (CIO of CenturyLink Labs and CEO of AppFog). GrĂ©gory Horion said this was his favorite Panamax template (besides the Locomotive CMS template he created đŸ™‚ and this Tweet was favorited by the engineering team at Twitter. Seen this template mentioned other places? Let me know!
What’s next for Panamax
Panamax is a very cool project. One of the biggest things that the Panamax team is working on is support for multiple hosts. Things will really start to get fun then. It will be very cool to see this deployed in production. I can see web hosts really loving something like this since it’s great for running software like WordPress where there are multiple components that need to linked together such as PHP, Apache, and MySQL.
Leave A Comment