Observing and Controlling Distributed Systems with Cross-Cutting Tools

It is notoriously difficult to understand, troubleshoot, and enforce distributed systems behaviors, because unlike standalone programs, they lack a central point of visibility and control.  This impacts a range of important distributed systems management tasks, such as resource management, performance, security, accounting, and many more.  In this talk I will outline several techniques and abstractions that let us re-establish this missing visibility and control, and reason about systems end-to-end.  To demonstrate, I will present two management tools that I have developed in my research: Retro, which measures resource usage and co-ordinates scheduler parameters to achieve end-to-end performance goals; and Pivot Tracing, which dynamically monitors and correlates metrics across component boundaries.  Together, these tools illustrate some of the common challenges and potential solutions when developing and deploying tools for distributed systems.

Bio:
Jonathan Mace is a Ph.D. candidate in the Computer Science department at Brown University.  His research centers on how to understand and enforce end-to-end behaviors in distributed systems.  During his Ph.D. he was awarded the Facebook Fellowship in Distributed Systems, and he received an SOSP Best Paper Award for his work on Pivot Tracing.