MesosCon Europe 2017

The Mesos executor is a part of Mesos that could be replaced with custom implementation. The executor controls tasks lifecycle. In this talk I will present the benefits that comes from writing a custom executor.

Yelp was an early adopter of Mesos and Marathon, building PaaSTA, a PaaS that provides an easy way for developers to deploy their services and batches. As we migrated more parts of the infrastructure to run on PaaSTA, we had to figure out how to maximize cluster utilization and minimize costs. In this talk, I'll discuss how Yelp autoscales both services and servers, shuffling tasks around our Mesos cluster to improve utilization, whilst dealing with the extra volatility caused by running on AWS Spot Fleet. I’ll tell stories of outages, strategies for improving resilience against AWS pulling the plug on instances with 2 minutes warning and gracefully migrating services actively serving traffic, and discuss how we decide when to increase and decrease cluster capacity.

Marathon and Chronos have been the core and essential frameworks for most Apache Mesos clusters. Marathon ensures long running tasks are running in the datacenter and is commonly used to maintain high availability of other frameworks. Chronos is the fault tolerant cron of the datacenter. Not long ago, Mesosphere introduced Metronome as a replacement for Chronos based on the Marathon code base. There has been significant change in Marathon to leverage the latest features of Apache Mesos, shift the framework to a reliable and stable state and improve its overall performance and scalability. This session will provide an overview of all the new features, a reasoning about how and why Marathon will grow, and will conclude with a look at the roadmap for Marathon in particular and frameworks launched via Marathon in general.

At Yelp, we all strive to increase developer productivity by decreasing the time to test, deploy, and monitor changes. To enable developers to push code safely, we run more than 20 million tests every day. Yelp has a monolith web application which has 100,000 tests; running them sequentially takes approximately 2 days to finish. We built an in-house distributed system called Seagull, which splits these 100,000 tests into smaller chunks(bundles) using our bespoke algorithm and runs these chunks in parallel on the compute cluster to finish all the tests in less than 10 mins. Seagull uses Apache Mesos to schedule these run test bundles on AWS spotfleet. We have written a custom autoscaler for AWS spotfleet which dynamically adjusts the cluster capacity based on different utilization metrics for optimal use of resources. Seagull runs more than 2 million ephemeral docker containers every day.

Chartbeat, a real time web publishing analytics platform made the decision last year to migrate the bulk of our workload from puppet managed AWS EC2 instances to Aurora. Today, the majority of our migration is done and this talk will focus on what we learned and the decisions made along the way, including what not to migrate to Aurora. We chose to adopt Aurora for a variety of reasons: cost savings from better resource utilization, consistent deployments and monitoring of services, streamlining of the development workflow and the ability to approach scaling our platform holistically. Aurora has helped us to reign in server sprawl and get a much better handle on our footprint. We don't claim to have done everything "right", but it works for us and that's all that matters.

Enterprises large and small want to have a disaster recovery plan at the ready when they are running distributed databases. The simplest such setup uses two database clusters in two different data centers with asynchronous replication of all updates. However, already this seemingly simple approach causes considerable head scratching for architects and developers of a distributed data store, considering fault tolerance, network failures, automatic failover and write load spikes.

This talk explains how ArangoDB implements asynchronous DC to DC replication between clusters using the Kafka message broker on both sides together with an incremental replication protocol and automatic write ahead log tailing. Apache Mesos plays a vital role in managing the deployment, managing resources, improve utilization, ensure scalability and ensuring liveness of the whole system.