The focus of this hackathon, while not excluding other ideas, is improving Apache Mesos and its ecosystem.
There will be a number of experienced contributors and committers helping the participants to get started, give direct feedback and if possible even commit code.
Potential areas include:
All other proposals are welcome as well and will be discussed at the beginning of the hackathon.
You just shipped all your features, and you got your service running in your development environment. You know how to run tasks on a Mesos cluster, but this ain't your first rodeo...
It's well understood that in order to move your applications to production, you need to make them highly available. But what about making them highly debuggable? Once you've got the highly available part right, how can you be sure that your app runs as expected? And how can you ensure that you are well-equipped to debug problems when things inevitably go wrong?
In this session we'll talk about how to architect your highly-available applications for maximum debuggability.
In this workshop, participants will build their own microservice application and connect it to a Fast Data Pipeline consisting of Apache Spark, Mesos, Cassandra, and Kafka. They will then deploy this pipeline on top of DC/OS, and explore the operational aspects of updating, scaling, and monitoring the data pipeline.
Participants will learn about:
Best practices for setting up Fast Data Pipelines
The components of the SMACK stack and respective alternatives.
E.g., using Apache Flink for stream processing
How to deploy such stack in an efficient and fault-tolerant way
How to operate and monitor the stack once it is set up:
Monitoring
Upgrades
Debugging
With the rise of machine learning and artificial intelligence, organizations are looking to adopt more GPUs (Graphics Processing Units) as they can be orders of magnitudes faster than standard CPUs. With recent advance on deep learning models in self-driving car areas such as lane-detection, perception and so on, it is important to enable distributed deep learning with large-scale GPU clusters.
GPU-enabled clusters are usually dedicated to a specific team or shared across teams. These two scenarios mean that GPUs are either underutilized or overutilized during peak times, leading to increased delays and a waste of precious time for the data science team and cloud resources. Existing tools do not allow dynamic allocation of resources while also guaranteeing performance and isolation
This workshop will show how DC/OS supports allocating GPUs and Machine learning frameworks to different services and teams.
Participants will learn hands-on with pre-provisioned cluster about:
Setting up GPU isolation in DC/OS
Deploying different Tensorflow instances on DC/OS utilizing these GPU resources
Deploying a complete pipeline for Twitter sentiment analysis with Tensorflow on DC/OS
