SFI Summer School
June 24, 2011, 6:07 pm
Filed under: Uncategorized | Tags: , , , ,

If you don’t live in New Mexico [1], you’ve probably never heard of a mangonada.  Basically, it’s frozen mango pulp on a stick, served in a cup that contains lime juice, chili and salt.  You dip the mango popsicle in the cup as you eat it.  A mangonada is simultaneously too sweet, too salty, too spicy and too sour, but somehow, after a few licks, it starts to make sense.

At its best, the Santa Fe Institute is kind of like a mangonada.  It’s aplace where people simultaneously do research in economics, physics, sociology, biology and computation.  The research agenda is ridiculously broad and ambitious, but sometimes it works out.

This summer, I had the pleasure of lecturing at the Santa Fe Institute Complex System Summer School – a program that brings together about 100 graduate students in many disciplines from all over the world.

This was a great chance for me to think about what is fundamental about distributed algorithms, what are some of the key ideas and problems that students in economics, math, biology, physics, and sociology need to know.  The links below give the material that I chose to talk about.  Basically I spent the first lecture on “traditional” distributed computing problems, where we can tell each node exactly what algorithm to run.  Then I spent the second lecture on the harder models where only some of the nodes run our algorithm, or worse, no node will run our algorithm unless it’s in that nodes best interest.

The students seemed particularly engaged in the second lecture, and I was surprised about the level of interest in Byzantine agreement, which I almost didn’t include at first, since it seemed a bit esoteric.  I talked to several students after the lectures, including a biologist and sociologist  who were interested in connections between these types of problems and the problems faced by social insects and social groups.  I always find it very energizing to talk to bright young scholars and I’m looking forward to next summer.

Here are my lectures:


Special thanks to Sriram Pemmaraju  who helped me out with the distributed graph coloring material, and to Roger Wattenhoffer for his excellent class notes on distributed graph coloring.

[1] or old Mexico I assume 🙂


Consensus[1] is arguably one of the most fundamental problem in distributed computing.  The basic idea of the problem is: n players each vote on one of two choices, and the players  want to hold an election to decide which choice “wins”.  The problem is made more interesting by the fact that there is no leader, i.e. no single player who everyone knows is trustworthy and can be counted on to tabulate all the votes accurately.

Here’s a more precise statement of the problem.  Each of n players starts with either a 0 or a 1 as input.  A certain fraction of the players (say 1/3) are bad, and will collude to thwart the remaining good players.  Unfortunately, the good players have no idea who the bad players are.  Our goal is to create an algorithm for the good players that ensures that

  • All good players output the same bit in the end
  • The bit output by all good players is the same as the input bit of at least one good player

At first, the second condition may seem ridiculously easy to satisfy and completely useless.  In fact, it’s ridiculously hard to satisfy and extremely useful.  To show the problem is hard, consider a naive algorithm where all players send out their bits to each other and then each player outputs the bit that it received from the majority of other players.  This algorithm fails because the bad guys can send different messages to different people.  In particular, when the vote is close, the bad guys can make one good guy commit to the bit 0 and another good guy commit to the bit 1.

The next thing I want to talk about is how useful this problem is.   First, the problem is broadly useful in the engineering sense of helping us build tools.  If you can solve consensus, you can solve the problem of how to build a system that is more robust than any of its components.  Think about it: each component votes on the outcome of a computation and then with consensus it’s easy to determine which outcome is decided by a plurality of the components (e.g. first everyone sends out their votes, then everyone calculates the majority vote, then everyone solves the consensus problem to commit to a majority vote.)   It’s not surprising then that solutions to the consensus problem have been used in all kinds of systems ranging from flight control, to data bases, to peer-to-peer; Google uses consensus in the Chubby system; Microsoft uses consensus in Farsite; most structured peer-to-peer systems (e.g. Tapestry, Oceanstore) use consensus in some way or another.  Any distributed system built with any pretension towards robustness that is not using consensus probably should be.

But that’s just the engineering side of things.  Consensus is useful because it allows us to study synchronization in complex systems.  How can systems like birds, bees, bacteria, markets come to a decision even when there is no leader.  We know they do it, and that they do it robustly, but exactly how do they do it and what is the trade-off they pay between robustness and the time and communication costs of doing it?  Studying upper and lower bounds on the consensus problem gives insight into these natural systems.  The study of how these agreement building processes, or quorum sensing, occur in nature has become quite popular lately, since they occur so pervasively.

Consensus is also useful because it helps us study fundamental properties of computation.  One of the first major result on consensus due to Fischer, Lynch and Patterson, in 1982, was that consensus is impossible for any deterministic algorithm with even one bad player (in the asynchronous communication model).  However, a follow up paper by Ben-Or showed that with a randomized algorithm, it was possible to solve this problem even with a constant fraction of bad players, albeit in exponential time.  This was a fundamental result giving some idea of how useful randomization can be for computation under an adversarial model.  As a grad student, I remember taking a class with Paul Beame telling us how impressed he was by what these two results said about the power of randomness when they first came out.  Cryptography was also shown to be useful for circumventing the Fischer, Lynch and Patterson result, and I’ve heard of several prominent cryptographers  who were first drawn to that area at the time because of its usefulness in solving consensus.

In the next week or two, I’ll go into some of the details of recent results on this problem that make use of randomness and cryptography.  Early randomized algorithms for consensus like Ben-Or’s used very clever tricks, but no heavy duty mathematical machinery.  More recent results, which run in polynomial time, make use of more modern tricks like the probabilistic method, expanders, extractors, samplers and connections with error-correcting codes, along with assorted cryptographic tricks.  I’ve been involved on the work using randomness, so I’ll probably start there.

[1] The consensus problem is also frequently referred to as the Byzantine agreement problem or simply agreement.  I prefer the name consensus, since it is more succinct and descriptive. While the research community has not yet reached a “consensus” on a single name for this problem, in recent years, the name consensus is being used most frequently.

July 17, 2009, 4:48 pm
Filed under: Uncategorized | Tags: , , ,

Welcome! I am Jared Saia, a professor of computer science at U. New Mexico. This is my new blog containing meandering thoughts on order, disorder and robustness in complex systems, with an eye towards connections with computer science theory, distributed computing and security.