October 25-27, 2017 - Prague, Czech Republic
Click Here For Information & Registration
View analytic
Friday, October 27 • 14:00 - 14:50
Seagull: A Distributed, Fault Tolerant, Concurrent Task Runner - Sagar Patwardhan, Yelp Inc.

Sign up or log in to save this to your schedule and see who's attending!

At Yelp, we all strive to increase developer productivity by decreasing the time to test, deploy, and monitor changes. To enable developers to push code safely, we run more than 20 million tests every day. Yelp has a monolith web application which has 100,000 tests; running them sequentially takes approximately 2 days to finish. We built an in-house distributed system called Seagull, which splits these 100,000 tests into smaller chunks(bundles) using our bespoke algorithm and runs these chunks in parallel on the compute cluster to finish all the tests in less than 10 mins. Seagull uses Apache Mesos to schedule these run test bundles on AWS spotfleet. We have written a custom autoscaler for AWS spotfleet which dynamically adjusts the cluster capacity based on different utilization metrics for optimal use of resources. Seagull runs more than 2 million ephemeral docker containers every day.


Sagar Patwardhan

Software Engineer, Yelp inc.
My name is Sagar Patwardhan and I am a software engineer at Yelp Inc. I am part of the distributed systems(Infrastructure) team. I have been with Yelp for a little over two years; I am currently working on building mesos infrastructure and next-gen batch processing infrastructure... Read More →

Friday October 27, 2017 14:00 - 14:50
Congress Hall 2

Attendees (21)