Cornell Systems Lunch

In this talk we give a survey of solutions -- and especially, discuss the underlying design principles -- that we developed in recent years to achieve extremely high packet processing rates in commodity operating systems, for both bare metal and virtual machines.

By using simple abstractions, and resisting the temptation to design systems around performant but constraining assumptions, we have been able to build a very flexible framework that addresses the speed and latency communication requirements of both bare metal and virtual machines up to the 100 Gbit/s range. Our goal is not to provide the fastest framework in the universe, but one that is easy to use and rich of features (and still, amazingly fast), to taking away concerns on the dataplane's speed from (reasonable) network applications.

Our NETMAP framework, opensource and BSD licensed, runs on 3 OSes (FreeBSD, Linux, Windows); provides access to NICs, host stack, virtual switches and point-to-point channels (netmap pipes), running between 20 and over 100Mpps on software ports, and saturating NICs with just a single core (reported up to 45 Mpps on recent 40G nics); achieves bare-metal speed on VMs thanks to a virtual passthrough mode (available for Qemu and bhyve); and can be used with no modifications by libpcap clients.

How can a machine specify a computation to another one and then, without executing the computation, check that the remote machine carried it out correctly? And how can this be done without assumptions about the performer (replication, etc.) or restrictions on the computation? This is a classic problem in systems security, and is relevant in multiple contexts: cloud computing, an untrusted hardware supply chain, etc. For decades, it has been known that this problem can be solved in theory, using probabilistically checkable proofs (PCPs) coupled with cryptographic tools. The rub was practicality: if implemented naively, the theory would be astronomically more expensive than simply executing the computation locally.

Over the last several years, a number of projects have brought this theory to near practicality. In this emerging area of _proof-based verifiable computation_, the pace of progress has been rapid, and there have been many encouraging developments: factors-of-a-trillion speedups, compilers that transform from high-level programs to executables that implement the protocol entities, and sophisticated implementations.

I will discuss the general area and my group's efforts in refining theory and building systems. A notable recent result is our Buffet system. Buffet has very good expressivity (it handles almost the entire C programming language as well as setups such as remote MapReduce jobs, database queries, etc.) and achieves the best performance in the area by multiple orders of magnitude. I will also cover the remaining obstacles to practicality in this research area; my hope is to communicate cautious optimism, with the emphasis on cautious.

In this talk, I will present two flavors of our hybrid clocks: hybrid logical clocks (HLC) and hybrid vector clocks (HVC). HLC finds applications in multiversion distributed database systems and enable efficient querying of consistent snapshots. HVC finds applications in debugging for concurrency race conditions and in lightweight causal delivery of messages. For all practical applications and deployment environments, the size of HVC remains only as a couple entries and substantially less than N, the number of nodes. I will also talk about higher-level auditability and availability primitives, detectors and correctors, that hybrid clocks enable.

The ability to identify authors of computer programs based on their coding style is a direct threat to the privacy and anonymity of programmers. Previous work has examined attribution of authors from both source code and compiled binaries, and found that while source code can be attributed with very high accuracy, the attribution of executable binary appears to be much more difficult. Many potentially distinguishing features present in source code, e.g. variable names, are removed in the compilation process, and compiler optimization may alter the structure of a program, further obscuring features that are known to be useful in determining authorship.

We examine executable binary authorship attribution from the standpoint of machine learning, using a novel set of features that include ones obtained by decompiling the executable binary to source code. We show that many syntactical features present in source code do in fact survive compilation and can be recovered from decompiled executable binary. This allows us to add a powerful set of techniques from the domain of source code authorship attribution to the existing ones used for binaries, resulting in significant improvements to accuracy and scalability. We demonstrate this improvement on data from the Google Code Jam, obtaining attribution accuracy of up to 96% with 20 candidate programmers. We also demonstrate that our approach is robust to a range of compiler optimization settings, and binaries that have been stripped of their symbol tables. Finally, for the first time we are aware of, we demonstrate that authorship attribution can be performed on real world code found "in the wild" by performing attribution on single-author GitHub repositories.