Big Red Bits » probability http://www.bigredbits.com Theory, Distributed Systems, and Other Random Bits Thu, 29 Sep 2011 07:13:29 +0000 en hourly 1 http://wordpress.org/?v=3.3.1 Bayesian updates and the Lake Wobegon effect http://www.bigredbits.com/archives/643 http://www.bigredbits.com/archives/643#comments Mon, 26 Sep 2011 01:48:27 +0000 renatoppl http://www.bigredbits.com/?p=643 We seem to have a good mathematical understanding of Bayesian updates, but somehow a very poor understanding of its practical implications. There are many situations in practice that we easily perceive as irrational, one of the most famous is the so called Lake Wobegon effect, named after the fictional town in Minnesota, where “all the women are strong, all the men are good looking, and all the children are above average”. It is described as a cognitive bias where individuals tend to overestimate their own capabilities. In fact, when drivers are asked to rate their own skilled compared to the average in three groups: low-skilled, medium-skilled and high-skilled, most rate themselves above the average.

In fact, the behavioral economics literate is full of examples like this where the observed data is far from what you would expect to observe if all agents were rational – and those are normally attributed to cognitive biases. I was always a bit suspicious of such arguments: it was never clear if agents were simply not being rational or whether their true objective wasn’t being captured by the model. I always thought the second was a lot more likely.

One of the main problems of the irrationality argument is that they ignore the fact that agents live in a world where its states are not completely observed. In a beautiful paper in Econometrica called “Apparent Overconfidence“, Benoit and Dubra argue that:

“But the simple truism that most people cannot be better than the median does not imply that most people cannot rationally rate themselves above the median.”

The authors show that it is possible to reverse engineer a signaling scheme such that the data is mostly consistent with the observation. Let me try to give a simple example they give in the introduction: consider that each driver has one of three types of skill: low, medium or high: \{L,M,H\} and \mathbb{P}(L) = \mathbb{P}(M) = \mathbb{P}(H) = \frac{1}{3}. However, they can’t observe this. They can only observe some sample of their driving. Let’s say for simplicity that they can observe a signal A that says if they caused an accident or not. Assume also that the larger that skill of a driver, the higher it is his probability of causing an accident, say:

\mathbb{P}(A \vert L) = \frac{47}{80}, \mathbb{P}(A \vert L) = \frac{9}{16}, \mathbb{P}(A \vert L) = \frac{1}{20}

Before observing A each driver things of himself as having probability $\frac{1}{3}$ of having each type of skill. Now, after observing A, they update their belief according to Bayes rule, i.e.,

\mathbb{P}(s \vert A) = \frac{ \mathbb{P}(A \vert s) \mathbb{P}(s) }{ \sum_{s'} \mathbb{P}(A \vert s') \mathbb{P}(s') }

doing the calculations, we have that \mathbb{P}(A) = \frac{2}{5} and for the \frac{3}{5} of the drivers that didn’t suffer an accident, they’ll evaluate \mathbb{P}(L \vert \neg A) = \frac{11}{48}, \mathbb{P}(M \vert \neg A) = \frac{35}{144}, \mathbb{P}(H \vert \neg A) = \frac{19}{36}, so:

\mathbb{P}(H \vert \neg A) > \mathbb{P}(L \cup M \vert \neg A)

and therefore will report high-skill. Notice this is totally consistent with rational Bayesian-updaters. The main question in the paper is: “when it is possible to reverse engineer a signaling scheme ?”. More formally, let \Theta be a set of types of users and let \theta \sim H \in \Delta(\Theta), i.e., H is a distribution on the types which is common knowledge. Now, if we ask agents to report their type, their report is some H' \in \Delta(\Theta), H' \neq H. Is there a signaling scheme S which can be interpreted as a random variable correlated with \theta such that H' is the distribution rational Bayesian updaters would report based on what they observed from S ? The authors give necessary and sufficient condition on when this is possible given H,H'.

—————————–

A note also related to the Lake Wobegon effect: I started reading a very nice book by Duncan Watts called “Everything Is Obvious: *Once You Know the Answer” about traps of the common-sense. The discussion is different then above, but it also talks about the dangers of applying our usual common sense, which is very useful to our daily life, to scientific results. I highly recommend reading the intro of the book, which is open in Amazon. He gives examples of social phenomena where, once you are told them, you think: “oh yeah, this is obvious”. But then if you were told the exact opposite (in fact, he begins the example by telling you the opposite from the observed in data), you’d also think “yes, yes, this is obvious” and come up with very natural explanations. His point is that common sense is very useful to explaining data observations, specially observations of social data. On the other hand, it is performs very poorly on predicting how the data will look like before actually seeing it.

]]> http://www.bigredbits.com/archives/643/feed 0 MHR, Regular Distributions and Myerson’s Lemma http://www.bigredbits.com/archives/539 http://www.bigredbits.com/archives/539#comments Mon, 30 May 2011 10:46:08 +0000 renatoppl http://www.bigredbits.com/?p=539 Monotone Hazard Rate (MHR) distributions and its superclass regular distributions keep appearing in the Mechanism Design literature and this is due to a very good reason: they are the class of distributions for which Myerson’s Optimal Auction is simple and natural. Let’s brief discuss some properties of those distributions. First, two definitions:

  1. Hazard rate of a distribution f : h(z) = \frac{f(z)}{1-F(z)}
  2. Myerson virtual value of a distribution f : \phi(z) = z - \frac{1-F(z)}{f(z)}

We can interpret the hazard rate in the following way: think of T \sim f as a random variable that indicates the time that a light bulb will take to extinguish. If we are in time t and the light bulb hasn’t extinguished so far, what is the probability it will extinguish in the next \delta time:

\mathbb{P}[T \leq t+\delta \vert T > t] \approx \frac{f(t) \delta}{1-F(t)}

We say that a distribution is monotone hazard rate, if h(z) is non-decreasing. This is very natural for light bulbs, for example. Many of the distributions that we are used to are MHR, for example, uniform, exponential and normal. The way that I like to think about MHR distributions is the following: if some distribution has hazard rate h(z), then it means that F'(z) = (1-F(z)) h(z). If we define G(z) = 1-F(z), then (log G(z))' = \frac{G'(z)}{G(z)} = -h(z), so:

F(z) = 1-\text{exp}(-\int_0^z h(u) du)

From this characterization, it is simple to see that the extremal distributions for this class, i.e. the distributions that are in the edge of being MHR and non-MHR are constant hazard rate, which correspond to the exponential distribution F(z) = 1-e^{-\lambda z} for z \in [0,\infty). They way I like to think about those distributions is that whenever you are able to prove something about the exponential distribution, then you can prove a similar statement about MHR distributions. Consider those three examples:

Example 1: \mathbb{P}[\phi(z) \geq 0] \geq \frac{1}{e} for MHR distributions. This fact is straightforward for the exponential distribution. For the exponential distribution \phi(z) = z-\lambda^{-1} and therefore

\mathbb{P}[\phi(z) \geq 0] \geq \mathbb{P}[z > \lambda^{-1}] = 1-F(\lambda^{-1}) = e^{-1}

but the proof for MHR is equally simple: Let r = \inf \{z; \phi(z) \geq 0\}, therefore r h(r) \leq 1.
P\{ \phi(v) \geq 0\} = P\{ v \geq r \} = 1 - F(r) = e^{-\int_0^r h(u) du} \geq e^{-r h(r)} \geq e^{-1}

Example 2: Given z_1, z_2 \sim f iid where f is MHR and v_1 = \max \{z_1, z_2\} and v_2 = \min \{z_1, z_2\}, then \mathbb{E}[v_2] \geq \frac{1}{3} \mathbb{E}[v_1]. The proof for the exponential distribution is trivial, and in fact, this is tight for the exponential, the trick is to use the convexity of z \mapsto \int_0^z h(u) du. We use that \int_0^{2z} h \geq 2 \int_0^z h in the following way:

\mathbb{E} [v_2] = \int_0^\infty (1 - F(z))^2 dz = \int_0^\infty e^{-2 \int_0^z h} dz

\geq \int_0^\infty e^{-\int_0^{2z} h} dz= \frac{1}{2} \int_0^\infty 1 - F(z) dz = \frac{1}{2} \mathbb{E} [z]

Since \mathbb{E} [v_1 + v_2] = \mathbb{E} [z_1 + z_2] = 2 \mathbb{E} [z], we have that \mathbb{E}[v_1] = 2 \mathbb{E}[z] - \mathbb{E}[v_2] \leq \frac{3}{2} \mathbb{E}[z]. This way, we get: \mathbb{E}[v_2] \geq \frac{1}{2}\mathbb{E}[z] \geq \frac{1}{2} \cdot \frac{2}{3} \mathbb{E}[v_1] = \frac{1}{3} \mathbb{E}[v_1]

Example 3: For MHR distributions, there is a simple lemma that relates the virtual value and the real value and this lemma is quite useful in various settings: let r = \inf \{z; \phi(z) > 0 \}, then for z \geq r, \phi(z) \geq z - r. Again, this is tight for exponential distribution. The proof is quite trivial:

x - \phi(x) = \frac{1-F(x)}{f(x)} \leq \frac{1-F(r)}{f(r)} = r

Now, MHR distributions are a subclass of regular distributions, which are the distributions for which Myerson’s virtual value \phi(z) is a monotone function. I usually find harder to think about regular distributions than to think about MHR (in fact, I don’t know so many examples that are regular, but not MHR. Here is one, though, called the equal-revenue-distribution. Consider z \in [1, \infty) distributed according to f(z) = 1/z^2. The cumulative distribution is given by F(z) = 1-1/z. The interesting thing of this distribution is that posted prices get the same revenue regardless of the price. For example, if we post any price r \in [1,\infty), then a customer with valuations z \sim f buys the item if z > r by price r, getting  revenue is r (1-F(r)) = 1. This can be expressed by the fact that \phi(z) = 0. I was a bit puzzled by this fact, because of Myerson’s Lemma:

Myerson Lemma: If a mechanism sells to some player that has valuation v \sim f with probability x(v) when he has value v, then the revenue is \mathbb{E} [x(v) \phi(v)].

And it seemed that the auctioneers was doomed to get zero revenue, since \phi(z) = 0. For example, suppose we fix some price r and we sell the item if v \geq r by price r. Then it seems that Myerson’s Lemma should go through by a derivation like that (for this special case, although the general proof is quite similar):

\mathbb{E} [x(v) \phi(v)] = \int_r^\infty \phi(z) f(z) dz = \int_r^\infty z f(z) - (1-F(z)) dz =

= \int_r^\infty [ z f(z) - \int_z^\infty f(u) du ] dz = \int_r^\infty z f(z) dz - \int_r^\infty \int_r^u f(u) dz du

= r (1-F(r))

but those don’t seem to match, since one side is zero and the other is 1. The mistake we did above is classic, which is to calculate \infty - \infty. We wrote:

\mathbb{E}[\phi(v)] = \int_r^\infty z f(z) dz - \int_r^\infty 1-F(z) dz

but both are infinity! This made me realize that Myerson’s Lemma needs the condition that \mathbb{E}[z] < \infty, which is quite a natural a distribution over valuations of a good. So, one of the bugs of the the equal-revenue-distribution is that \mathbb{E}[z] = \infty. A family that is close to this, but doesn’t suffer this bug is: f(z) = \frac{\alpha-1}{z^\alpha} for z \in [1,\infty), then F(z) = 1 - z^{1-\alpha}. For \alpha > 2 we have \mathbb{E}[v] < \infty, then we get \phi(z) = \frac{\alpha-2}{\alpha-1} z.

]]> http://www.bigredbits.com/archives/539/feed 0 DP and the Erdős–Rényi model http://www.bigredbits.com/archives/443 http://www.bigredbits.com/archives/443#comments Mon, 16 May 2011 21:41:28 +0000 renatoppl http://www.bigredbits.com/?p=443 Yesterday I was in a pub with Vasilis Syrgkanis and Elisa Celis and we were discussing about how to calculate the expected size of a connected component in G(n,p), the Erdős–Rényi model. G(n,p) is the classical random graph obtained by considering n nodes and adding each edge (i,j) independently with probability p. A lot is known about its properties, which very interestingly change qualitatively as the value of p changes relativeto n. For example, for p <\frac{1}{n} then there is no component greater than O(\log n) with high probability. When p = \frac{c}{n}, c>1 and n \rightarrow \infty, then the graph has a giant component. All those phenomena are very well studied in the context of probabilistic combinatorics and also in social networks. I remember learning about them in Jon Kleinberg’s Structure of Information Networks class.

So, coming back to our conversation, we were thinking on how to calculate the size of a connected component. Fix some node u in G(n,p) – it doesn’t matter which node, since all nodes are equivalent before we start tossing the random coins. Now, let C_u be the size of the connected component of node u. The question is how to calculate C(n,p) = \mathbb{E} [C_u].

Recently I’ve been learning MATLAB (actually, I am learning Octave, but it is the same) and I am very amazed by it and impressed about why I haven’t learned it before. It is a programming language that somehow knows exactly how mathematicians think and the syntax is very intuitive. All the operations that you think of performing when doing mathematics, they have implemented. Not that you can’t do that in C++ or Python, in fact, I’ve been doing that all my life, but in Octave, things are so simple. So, I thought this was a nice opportunity for playing a bit with it.

We can calculate C(n,p) using a dynamic programming algorithm in time O(n^2) – well, maybe we can do it more efficiently, but the DP I thought was the following: let’s calculate \mathcal{C}(n,s,p) where it is the expected size of the u-connected component of a random graph with n nodes where the edges between u and other nodes have probability p' = 1 - (1-p)^s and an edge between v_1 and v_2 have probability p. What we want to compute is C(n,p) = \mathcal{C}(n,1,p).

What we can do is to use the Principle of Deferred Decisions,  and toss the coins for the n-1 edges between u and the other nodes. With probability bin(k,n,p') = {n \choose k} (p')^k (1-'p)^{n-k}, there are k edges between u and the other nodes, say nodes w_1, \hdots, w_k. If we collapse those nodes to u we end up with a graph of n-k nodes and the problem is equivalent to k plus the size of the connected component of u in the collapsed graph.

One difference, however is that the probability that the collapsed node u is connected to a node v of the n-1-k nodes is the probability that at least one of w_i is connected to v, which is 1-(1-p)^k. In this way, we can write:

\mathcal{C}(n,s,p) = 1 \cdot bin(0,n-1,p') + \sum_{k=1}^{n-1} bin(k,n-1,p') [ k + \mathcal{C}(n-k,k,p)]

where p' = 1-(1-p)^s. Now, we can calculate C(n,p) by using DP, simply by filling an n \times n table. In Octave, we can do it this way:


function component = C(N,p)
  C_table = zeros(N,N);
  for n = 1:N for s =1:N
    C_table(n,s) = binopdf(0,n-1,1-((1-p)^s)) ;
    for k = 1:n-1
      C_table(n,s) += binopdf(k,n-1,1-((1-p)^s)) * (k + C_table(n-k,k));
    end
  end end
  component = C_table(N,1);
endfunction

And in fact we can call C(n,p) for say n = 200 and p = 0.01 .. 0.3 and see how C(n,p) varies. This allows us, for example, to observe the sharp transition that happens before the giant component is formed. The plot we get is:


Erdős–Rényi model

]]> http://www.bigredbits.com/archives/443/feed 0 Probability Puzzles http://www.bigredbits.com/archives/278 http://www.bigredbits.com/archives/278#comments Wed, 17 Feb 2010 02:54:53 +0000 renatoppl http://www.bigredbits.com/?p=278 Today in a dinner with Thanh, Hu and Joel I heard about a paradox I haven’t heard so far. Probability is full of cute problems that challenge our understanding of the basic concepts. The most famous of them is the Monty Hall Problem, which asks:

You are on a TV game show and there are {3} doors – one of them contains a prize, say a car and the other two door contain things you don’t care about, say goats. You choose a door. Then the TV host, who knows where the prize is, opens one door you haven’t chosen and that he knows has a goat. Then he asks if you want to stick to the door you have chosen or if you want to change to the other door. What should you do?

Probably you’ve already came across this question in some moment of your life and the answer is that changing doors would double your probability of getting the price. There are several ways of convincing your intuitions:

  • Do the math: when you chose the door, there were three options so the prize is in the door you chose with {\frac{1}{3}} probability and in the other door with probability {\frac{2}{3}} (note that the presenter can always open some door with a goat, so conditioning on that event doesn’t give you any new information).
  • Do the actual experiment (computationally) as done here. One can always ask a friend to help, get some goats and perform the actual experiment.
  • To convince yourself that “it doesn’t matter” is not correct, think {100} doors. You choose one and the TV host open {98} of them and asks if you want to change or stick with your first choice. Wouldn’t you change?

I’ve seen TV shows where this happened and I acknowledge that other things may be involved: there might be behavioral and psychologic issues associated with the Monty Hall problem – and possibly those would interest Dan Ariely, whose book I began reading today – and looks quite fun. But the problem they told me about today in dinner was another: the envelope problem:

There are two envelopes and you are told that in one of them there is twice the amount that there is in the other. You choose one of the envelopes at random and open it: it contains {100} bucks. Now, you don’t know if the other envelope has {50} bucks or {200} bucks. Then someone asks you if you wanted to pay {10} bucks and change to the other envelope. Should you change?

Now, consider two different solutions to this problem: the first is fallacious and the second is correct:

  1. If I don’t change, I get {100} bucks, if I change I pay a penalty of {10} and I get either {50} or {200} with equal probability, so my expected prize if I change is {\frac{200+50}{2}-10 = 115 > 100}, so I should change.
  2. I know there is one envelope with {x} and one with {2x}, then my expected prize if I don’t change is {\frac{x + 2x}{2} = \frac{3}{2}x}. If I change, my expected prize is {\frac{x + 2x}{2} - 10 < \frac{3}{2}x}, so I should not change.

The fallacy in the first argument is perceiving a probability distribution where there is no one. Either the other envelope contains {50} bucks or it contains {200} bucks – we just don’t know, but there is no probability distribution there – it is a deterministic choice by the game designer. Most of those paradoxes are a result of either an ill-defined probability space, as Bertrand’s Paradox or a wrong comprehension of the probability space, as in Monty Hall or in several paradoxes exploring the same idea as: Three Prisioners, Sleeping Beauty, Boy or Girl Paradox, …

73a_humpty-dumpty

There was very recently a thrilling discussion about a variant on the envelope paradox in the xkcd blag – which is the blog accompaning that amazing webcomic. There was a recent blog post with a very intriguing problem. A better idea is to go there and read the discussion, but if you are not doing so, let me summarize it here. The problem is:

There are two envelopes containing each of them a distinct real number. You pick one envelope at random, open it and see the number, then you are asked to guess if the number in the other envelope is larger or smaller then the previous one. Can you guess correctly with more than {\frac{1}{2}} probability?

A related problem is: given that you are playing the envelope game and there are number {A} and {B} (with {A < B}). You pick one envelope at random and then you are able to look at the content of the first envelope you open and then decide to switch or not. Is there a strategy that gives you expected earnings greater than {\frac{A+B}{2}} ?

The very unexpected answers is yes !!! The strategy that Randall presents in the blog and there is a link to the source here is: let {X} be a random variable on {{\mathbb R}} such that for each {a<b} we have {P(a < X < b) > 0}, for example, the normal distribution or the logistic distribution.

Sample {X} then open the envelope and find a number {S} now, if {X < S} say the other number is lower and if {X > S} say the other number is higher. You get it right with probability

\displaystyle P(\text{picked }A) P(X > A) + P(\text{picked }B) P(X < B) = \frac{1}{2} (1 + P(A < X < B))

which is impressive. If you follow your guess, your expected earning {Y} is:

\displaystyle \begin{aligned} &P(\text{picked }A) \mathop{\mathbb E}[Y \vert \text{picked }A] + P(\text{picked }B) \mathop{\mathbb E}[Y \vert \text{picked }B] = \\ & = \frac{1}{2} [P(X<A) A + P(X>A) B] + \frac{1}{2} [P(X<B) B + P(X>B) A] \\ &= \frac{1}{2}[A [P(X<A) + P(X>B)] + B [P(X>A) + P(X<B)]] > \frac{A+B}{2} \\ \end{aligned}

The xkcd pointed to this cool archive of puzzles and riddles. I was also told that the xkcd puzzle forum is also a source of excellent puzzles, as this:

You are the most eligible bachelor in the kingdom, and as such the King has invited you to his castle so that you may choose one of his three daughters to marry. The eldest princess is honest and always tells the truth. The youngest princess is dishonest and always lies. The middle princess is mischievous and tells the truth sometimes and lies the rest of the time. As you will be forever married to one of the princesses, you want to marry the eldest (truth-teller) or the youngest (liar) because at least you know where you stand with them. The problem is that you cannot tell which sister is which just by their appearance, and the King will only grant you ONE yes or no question which you may only address to ONE of the sisters. What yes or no question can you ask which will ensure you do not marry the middle sister?

copied from here.

]]> http://www.bigredbits.com/archives/278/feed 0 Looking at probability distributions http://www.bigredbits.com/archives/231 http://www.bigredbits.com/archives/231#comments Fri, 13 Nov 2009 03:16:27 +0000 renatoppl http://www.bigredbits.com/?p=231 I’ve been taking two classes in probability this semester and in those I saw the proofs of a lot of interesting theorems which I knew about previously but I have never seen the proof, as the Central Limit Theorem, the Laws of Large Numbers and so on… Also, some theory which is looks somewhat ugly in the undergrad courses becomes very clear with the proper formal treatment. Today I was thinking what was the main take-home message that a computer scientist could take from those classes and. at ;east for me, this message is the various ways of looking to probability distributions. I’ve heard about moments, Laplace transform, Fourier transform and other tools like that, but I never realized before their true power. Probably still today, most of their true power is hidden from me, but I am starting to look at them in a different way. Let me try to go over a few examples of different ways we can look at probability distributions and show cases where they are interesting.

Most of ways of looking at probability distributions are associated with multiplicative system: a multiplicative system {Q} is a set of real-valued functions with the property that if {f_1, f_2 \in Q} then {f_1 \dot f_2 \in Q}. Those kinds of sets are powerful because of the Multiplicative Systems Theorem:

Theorem 1 (Multiplicative Systems Theorem) If {Q} is a multiplicative system, {H} is a linear space containing {1} (the constant function {1}) and is closed under bounded convergence, then {Q \subseteq H} implies that {H} contains all bounded {\sigma(Q)}-measurable functions.

The theorem might look a bit cryptic if you are not familiar with the definitions, but it boils down to the following translation:

Theorem 2 (Translation of the Multiplicative Systems Theorem) If {Q} is “general” multiplicative system, {X} and {Y} are random variable such that {\mathop{\mathbb E}{f(X)} = \mathop{\mathbb E}{f(Y)}} for all {f \in Q} then {X} and {Y} have the same distribution.

where general excludes some troublesome cases like {Q = \{1\}} or all constant functions, for example. In technical terms, we wanted {\sigma(Q) = \sigma\{f^{-1}((-\infty,a]); a\in {\mathbb R}, f \in Q \}} to be the Borel {\sigma}-algebra. But let’s not worry about those technical details and just look at the translated version. We now, discuss several kinds of multiplicative systems:

  1. The most common description of the a random variable is by the cummulative distribution function {F(u) = P\{X \leq u\}}. This is associated with {Q = \{ 1_{(-\infty,u]}(x); u \in {\mathbb R} \}} notice that simply {F(u) = \mathop{\mathbb E}{1_{(-\infty,u]}}}.
  2. We can characterize a random variable by its moments: the variable {X} is characterized by the set {M_n = \int_{\mathbb R} x^n \mu_X(dx)}. Given the moemnts {M_1, M_2, \hdots}, the variable is totally characterized, i.e., if two variables have the same moments, then they have the same distribution by the Multiplicative Systems Theorem. This description is associated with the system {Q = \{x^n; n = 1, 2, \hdots\}}
  3. Moment Generating Function: If {X} is a variable that assumes only integer values, we can describe the it as {p_0, p_1, p_2, \hdots}, where {p_n = P\{ X = n \}}. An interesting way of representing those probabilities is as the moment generating function {\psi_X(z) = \mathop{\mathbb E}{z^X} = \sum_{n=0}^\infty p_n z^n}. This is associated with the multiplicative system {Q = \{z^x, 0 \leq z < 1\}}.Now suppose we are given two discrete independent variables {X} and {Y}. What do we know about {X+Y}. It is easy to know its expectation, its variance, … but what about more complicated things? What is the distribution of {X+Y} ? Moment generating functions answer this question very easily, since:

    \displaystyle \psi_{X+Y} (z) = \mathop{\mathbb E} [z^{X+Y}] = \mathop{\mathbb E} [z^{X}] \cdot \mathop{\mathbb E}[z^{Y}] = \psi_X(z) \cdot \psi_Y(z)

    If we know moment generating functions, we can calculate expectation very easily, since {\mathop{\mathbb E}[X] = \psi'_X(1)}. For example, suppose we have a process like that: there is one bacteria in time {t = 0}. In each timestep, either this bacteria dies (with probability {p_0}), continues alive without reproducing (with probability {p_1} or has {k} offsprings (with probability {p_{1+k}}). In that case {\sum_0^\infty p_n = 1}. Each time, the same happens, independently with each of the bacteria alive in that moment. The question is, what is the expected number of bacteria in time {t} ?

    It looks like a complicated problem with just elementary tools, but it is a simple problem if we have moment generating functions. Just let {X_{ti}} be the variable associated with the {i^{th}} bacteria of time {t}. It is zero if it dies, {1} if it stays the same and {k+1} if it has {k} offsprings. Let also {N_t} be the number of bacteria in time {t}. We want to know {\mathop{\mathbb E} N_t}. First, see that:

    \displaystyle N_t = \sum_{i=1}^{N_{t-1}} X_{ti}

    Now, let’s write that in terms of moment generating functions:

    \displaystyle \psi_{N_t}(z) = \mathop{\mathbb E}{z^{X_{t1} + \hdots + X_{tN_t}}} = \sum_{k=0}^\infty P\{N_{t-1} = k\} \cdot \mathop{\mathbb E}[z^{X_{t1} + \hdots + X_{tN_t}} \vert N_t = k]

    which is just:

    \displaystyle \psi_{N_t}(z) = \sum_{k=0}^\infty P\{N_{t-1} = k\} \cdot \mathop{\mathbb E}[z^{X_{t1} + \hdots + X_{tk}}] = \sum_{k=0}^\infty P\{N_{t-1} = k \} \cdot \psi_X (z)^k

    since the variables are all independent and identically distributed. Now, notice that:

    \displaystyle \psi_{N_{t-1}}(z) = \sum_{k=0}^\infty P\{N_{t-1} = k\} \cdot z^k

    by the definition of moment generating function, so we effectively proved that:

    \displaystyle \psi_{N_{t}}(z) = \psi_{N_{t-1}}(\psi_X(z)) = \psi_X \psi_X \hdots \psi_X(z)

    We proved that {\psi_{N_{t}}(z)} is just {\psi(z)} iterated {t} times. Now, calculating the expectation is easy, using the fact that {\psi_X(1) = 1} and {\psi'_X(1) = \mathop{\mathbb E} X}. Just see that: {\mathop{\mathbb E} N_t = \psi_{N_{t}}'(1) = \psi_{N_{t-1}}'(\psi_X(1)) \cdot \psi'_X(1) = \psi_{N_{t-1}}'(1) \cdot \mathop{\mathbb E} X }. Then, clearly {\mathop{\mathbb E} N_t = (\mathop{\mathbb E} X)^t}. Using similar technique we can prove a lot more things about this process, just by analyzing the behavior of the moment generating function.

  4. Laplace Tranform: Now, moving to continuous variables, if {X} is a continuous non-negative variable we can define its Laplace tranform as: {\phi_X(u) = \mathop{\mathbb E} [e^{u X}] = \int_0^\infty e^{ux} \mu_X(dx)}, where {\mu_X(dx)} stands for the distribution of {X}, for example, {\rho_X(x) dx}. This is associated with the multiplicative system {Q = \{e^{ux}; u \geq 0\}}. Again, by the Multiplicative Systems Theorem, if {\phi_X = \phi_Y}, then the two variables have the same distribution. The Laplace tranform has the same nice properties as the Moment Generating Function, for example, {\phi_{X+Y}(u) = \mathop{\mathbb E}[e^{(X+Y)u}] = \mathop{\mathbb E}[e^{Xu}] \cdot \mathop{\mathbb E}[e^{Yu}] = \phi_X(u) \cdot \phi_Y(u)}.And it allows us to do similar tricks than the one I just showed for Moment Generating Functions. One common trick that is used, for example, in the proof of Chernoff bounds is, given independent non-negative random variables:

    \displaystyle P\left\{\sum_i X_i > u\right\} = P\left\{e^{\sum_i X_i} > e^u\right\} \leq \frac{\mathop{\mathbb E}[e^{\sum_i X_i} ]}{e^u} = \frac{\prod_i \mathop{\mathbb E}[e^{X_i} ]}{e^u}

    where we also used Markov Inequality: {\mathop{\mathbb E}[X] \geq \beta P\{X \geq \beta\}}. Passing to the Laplace transform is the main ingredient in the Chernoff bound and it allows us to sort of “decouple” the random variables in the sum. There are several other cases where the Laplace transform proves itsself very useful and turns things that looked very complicated when we saw in undergrad courses into simple and clear things. One clear example of that is the motivation for the Poisson random variable:

    If {T_i} are independend exponentially distributed random variables with mean {\lambda}, then {\rho_{T_i}(t) = \lambda e^{- \lambda t}}. An elementary calculation shows that its laplace transform is {\phi_{T_i}(u) = \frac{\lambda}{\lambda+u}}. Let {S_n = T-0 + T_1 + T_2 + \hdots + T_n}, i.e., the time of the {(n+1)^{th}} arrival. We want to know what is the distribution of {S_n}. How to do that?

    \displaystyle \phi_{S_n}(u) = \mathop{\mathbb E}[e^{u(T_0 + \hdots + T_n)}] = \phi_T(u)^{n+1} = \left( \frac{\lambda}{\lambda+u} \right)^{n+1}

    Now, we need to find {\rho_{S_n}} such that {\int_0^\infty \rho_{S_n}(t) e^{iu} dt = \left( \frac{\lambda}{\lambda+u} \right)^{n+1}}. Now it is just a matter of solving this equation and we get: {\rho_{S_n}(t) = \frac{\lambda (\lambda t)^n}{n!} e^{-\lambda t}}. Now, the Poisson varible {N_t} measures the number of arrivals in {[0,t]} and therefore:

    \displaystyle \begin{aligned} P\{N_t = n\} & = P\{S_{n-1} < t < S_n\} = P\{S_n > t\} - P\{S_{n-1} \geq t\} \\ & = \int_t^\infty \rho_{S_n}(t) dt - \int_t^\infty \rho_{S_{n-1}}(t) dt = \frac{(\lambda t)^n}{n!} e^{-\lambda t} \end{aligned}

  5. Characteristic Function or Fourier Tranform: Taking {Q = \{e^{i\lambda x}; x \in \mathbb{C}\}} we get the Fourier Transform: {f_X(\lambda) = \mathop{\mathbb E} e^{i \lambda x}} which also has some of the nice properties of the previous ones and some additional ones. The characteristic functions were the main actors in the development of all the probability techniques that lead to the main result of 19th century Probability Theory: the Central Limit Theorem. We know that moment generating functions and Laplace transforms completely characterize the distributions, but it is not clear how to recover a distribution once we have a transform. For Fourier Transform there is a cleas and simple way of doing that by means of the Inversion Formula:

    \displaystyle \rho_X(x) = \frac{1}{2 \pi} \int_{\mathbb R} e^{-i \lambda x} f_X(\lambda) d\lambda

    One fact that always puzzled me was: why is the normal distribution {\rho_N (x) = \frac{1}{\sqrt{2 \pi}} e^{-x^2/2} } so important? What does it have in special to be the limiting distribution in the Central Limit Theorem, i.e., if {X_1, X_2, \hdots, X_n} is a sequence of independent random variables, {S_n = \sum_1^n X_i - \sum_1^n \mathop{\mathbb E} X_i} then {S_n / \sqrt{var S_n} \rightarrow N(0,1)} under some natural conditions on the variables. The reason the normal is so special is because it is a “fixed point” for the Fourier Transform. We can see that {f_N(\lambda) = e^{-\lambda_2/2}}. And there we have something special about it that makes me believe the Central Limit Theorem.

————————-

This blog post was based on lectures by Professor Dynkin at Cornell.

]]> http://www.bigredbits.com/archives/231/feed 0 Random Spanning Trees http://www.bigredbits.com/archives/226 http://www.bigredbits.com/archives/226#comments Wed, 04 Nov 2009 04:51:59 +0000 renatoppl http://www.bigredbits.com/?p=226 BigRedBits is again pleased to have Igor Gorodezky as a guest blogger directly from UCLA. I leave you with his excelent post on the Wilson’s algorithm.

——————————————

Igor again, with another mathematical dispatch from UCLA, where I’m spending the semester eating and breathing combinatorics as part of the 2009 program on combinatorics and its applications at IPAM. In the course of some reading related to a problem with which I’ve been occupying myself, I ran across a neat algorithmic result – Wilson’s algorithm for uniformly generating spanning trees of a graph. With Renato’s kind permission, let me once again make myself at home here at Big Red Bits and tell you all about this little gem.

The problem is straightforward, and I’ve essentially already stated it: given an undirected, connected graph {G}, we want an algorithm that outputs uniformly random spanning trees of {G}. In the early ’90s, Aldous and Broder independently discovered an algorithm for accomplishing this task. This algorithm generates a tree {T} by, roughly speaking, performing a random walk on {G} and adding the edge {(u,v)} to {T} every time that the walk steps from {u} to {v} and {v} is a vertex that has not been seen before.

Wilson’s algorithm (D. B. Wilson, “Generating random spanning trees more quickly than the cover time,” STOC ’96) takes a slightly different approach. Let us fix a root vertex {r}. Wilson’s algorithm can be stated as a loop-erased random walk on {G} as follows.

Algorithm 1 (Loop-erased random walk) Maintain a tree {T}, initialized to consist of {r} alone. While there remains a vertex {v} not in {T}: perform a random walk starting at {v}, erasing loops as they are created, until the walk encounters a vertex in {T}, then add to {T} the cycle-erased simple path from {v} to {T}.

We observe that the algorithm halts with probability 1 (its expected running time is actually polynomial, but let’s not concern ourselves with these issues here), and outputs a random directed spanning tree oriented towards {r}. It is a minor miracle that this tree is in fact sampled uniformly from the set of all such trees. Let us note that this offers a solution to the original problem, as sampling {r} randomly and then running the algorithm will produce a uniformly generated spanning tree of {G}.

It remains, then, to prove that the algorithm produces uniform spanning trees rooted at {r} (by which we mean directed spanning trees oriented towards {r}). To this we dedicate the remainder of this post.

1. A “different” algorithm

Wilson’s proof is delightfully sneaky: we begin by stating and analyzing a seemingly different algorithm, the cycle-popping algorithm. We will prove that this algorithm has the desired properties, and then argue that it is equivalent to the loop-erased random walk (henceforth LERW).

The cycle-popping algorithm works as follows. Given {G} and {r}, associate with each non-root vertex {v} an infinite stack of neighbors. More formally, to each {v \neq r} we associate

\displaystyle \mathcal S_v = [u_{0}, u_{1}, \dots ]

where each {u_{i}} is uniformly (and independently) sampled from the set of neighbors of {v}. Note that each stack is not a random walk, just a list of neighbors. We refer to the left-most element above as the top of {\mathcal S_v}, and by popping the stack {\mathcal S_v} we mean removing this top vertex from {\mathcal S_v}.

Define the stack graph {G_{\mathcal S}} to be the directed graph on {V} that has an edge from {v} to {u} if {u} is at the top of the stack {\mathcal S_v}. Clearly, if {G} has {n} vertices then {G_{\mathcal S}} is an oriented subgraph of {G} with {n-1} edges. The following lemma follows immediately.

Lemma 1 Either {G_{\mathcal S}} is a directed spanning tree oriented towards {r} or it contains a directed cycle.

If there is a directed cycle {C} in {G_{\mathcal S}} we may pop it by popping {\mathcal S_v} for every {v \in C}. This eliminates {C}, but of course might create other directed cycles. Without resolving this tension quite yet, let us go ahead and formally state the cycle-popping algorithm.

Algorithm 2 (Cycle-popping algorithm) Create a stack {\mathcal S_v} for every {v \neq r}. While {G_{\mathcal S}} contains any directed cycles, pop a cycle from the stacks. If this process ever terminates, output {G_{\mathcal S}}.

Note that by the lemma, if the algorithm ever terminates then its output is a spanning tree rooted at {r}. We claim that the algorithm terminates with probability 1, and moreover generates spanning trees rooted at {r} uniformly.

To this end, some more definitions: let us say that given a stack {\mathcal S_v}, the vertex {u_i} is at level {i}. The level of a vertex in a stack is static, and is defined when the stack is created. That is, the level of {u_i} does not change even if {u_i} advances to the top of the stack as a result of the stack getting popped.

We regard the sequence of stack graphs produced by the algorithm as leveled stack graphs: each non-root vertex {v} is assigned the level of its stack. Observe that the level of {v} in {G_{\mathcal S}} is the number of times that {\mathcal S_v} has been popped. In the same way, we regard cycles encountered by the algorithm as leveled cycles, and we can regard the tree produced by the algorithm (if indeed one is produced) as a leveled tree.

The analysis of the algorithm relies on the following key lemma (Theorem 4 in Wilson’s paper), which tells us that the order in which the algorithm pops cycles is irrelevant.

Lemma 2 For a given set of stacks, either the cycle-popping algorithm never terminates, or there exists a unique leveled spanning tree {T} rooted at {r} such that the algorithm outputs {T} irrespective of the order in which cycles are popped.

Proof: Fix a set of stacks {\{ \mathcal S_v \}_{v \neq r}}. Consider a leveled cycle {C} that is pop-able, i.e.~there exist leveled cycles {C_1, C_2, \dots, C_k=C} that can be popped in sequence. We claim that if the algorithm pops any cycle not equal to {C}, then there still must exist a series of cycles that ends in {C} and that can be popped in sequence. In other words, if {C} is pop-able then it remains pop-able, no matter which cycles are popped, until {C} itself is actually popped.

Let {C'} be a cycle popped by the algorithm. If {C'=C_1} then the claim is clearly true. Also, if {C'} shares no vertices with {C_1, \dots, C_k}, then the claim is true again. So assume otherwise, and let {C_i} be the first in the series {C_1, \dots, C_k} to share a vertex with {C'}. Let us show that {C'=C_i} by contradiction.

If {C_i \neq C'}, then {C_i} and {C'} must share a vertex {w} that has different successors in {C'} and {C_i}. But by definition of {C_i}, none of the {C_1, \dots, C_{i-1}} contain {w}, and this implies that {w} has the same level in {C'} and {C_i}. Therefore its successor in both cycles is the same, a contradiction. This proves {C_i=C'}.

Moreover, the argument above proves that {C_i} and {C'} are equal as leveled cycles (i.e.~every vertex has the same level in both cycles). Hence

\displaystyle C'=C_i, C_1, C_2, \dots, C_{i-1}, C_{i+1}, \dots, C_k=C

is a series of cycles that can be popped in sequence, which proves the original claim about {C}.

We conclude that given a set of stacks, either there is an infinite number of pop-able cycles, in which case there will always be an infinite number and the algorithm will never terminate, or there is a finite number of such cycles. In the latter case, every one of these cycles is eventually popped, and the algorithm produces a spanning tree {T} rooted at {r}. The level of each non-root vertex in {T} is given by (one plus) the number of popped cycles that contained {v}. \Box

Wilson summarizes the cycle-popping algorithm thusly: “[T]he stacks uniquely define a tree together with a partially ordered set of cycles layered on top of it. The algorithm peels off these cycles to find the tree.”

Theorem 3 The cycle-popping algorithm terminates with probability 1, and the tree that it outputs is a uniformly sampled spanning tree rooted at {r}.

Proof: The first claim is easy: {G} has a spanning tree, therefore it has a directed spanning tree oriented towards {r}. The stacks generated in the first step of the algorithm will contain such a tree, and hence the algorithm will terminate, with probability 1.

Now, consider a spanning tree {T} rooted at {r}. We’ll abuse notation and let {T} be the event that {T} is produced by the algorithm. Similarly, given a collection of leveled cycles {\mathcal C}, we will write {\mathcal C} for the event that {\mathcal C} is the set of leveled cycles popped by the algorithm before it terminates. Finally, let {\mathcal C \wedge T} be the event that the algorithm popped the leveled cycles in {\mathcal C} and terminated, with the resulting leveled tree being equal to {T}.

By the independence of the stack entries, we have {\Pr[\mathcal C \wedge T] = \Pr[\mathcal C] \cdot p}, where {p} is the probability that the algorithm’s output is a leveled version of {T}, a quantity which a moment’s reflection will reveal is independent of {T}. Now,

\displaystyle \Pr[T] = \sum_{\mathcal C} \Pr[\mathcal C \wedge T] = p \sum_{\mathcal C} \Pr[\mathcal C]

which, as desired, is independent of {T}. \Box

2. Conclusion

We have shown that the cycle-popping algorithm generates spanning trees rooted at {r} uniformly. It remains to observe that the LERW algorithm is nothing more than an implementation of the cycle-popping algorithm! Instead of initially generating the (infinitely long) stacks and then looking for cycles to pop, the LERW generates stack elements as necessary via random walk (computer scientists might recognize this as the Principle of Deferred Decisions). If the LERW encounters a loop, then it has found a cycle in the stack graph {G_{\mathcal S}} induced by the stacks that the LERW has been generating. Erasing the loop is equivalent to popping this cycle. We conclude that the LERW algorithm generates spanning trees rooted at {r} uniformly.

]]> http://www.bigredbits.com/archives/226/feed 0 Entropy http://www.bigredbits.com/archives/145 http://www.bigredbits.com/archives/145#comments Fri, 28 Aug 2009 01:28:54 +0000 renatoppl http://www.bigredbits.com/?p=145 Today was the first day of classes here at Cornell and as usual, I attend to a lot of different classes to try to decide which ones to take. I usually feel like I wanted to take them all, but there is this constant struggle: if I take too many classes I have no time to do research and to read random things that happen to catch my attention at that moment, and if I don’t take many classes I feel like not learning a lot of interesting stuff I wanted to be learning. The solution in the middle of the way is to audit a lot of classes and start dropping them as a start needing more time: what happens usually quickly. This particular fall I decided that I need to build a stronger background in probability – since I am finding a lot of probabilistic stuff in my way and I have nothing more than my undergrad course and things I learned on demand. I attended at least three probability classes with different flavours today and I decided to blog about a simple, yet very impressive result I saw in one of them.

Since I took a class on “Principles of Telecommunications” in my undergrad, I became impressed by Shannon’s Information Theory and the concept of entropy. There was one theorem that I always heard about but never saw the proof. I thought it was a somewhat complicated proof, but it turned out not to be that much.

Consder an alphabet {\Omega} and a probability distribution over it. I want to associate to each {\omega \in \Omega} a string {c(\omega)} of {k(\omega)} {\{0,1\}}-digits to represent each simbol of the alphabet. One way of allowing the code to be decodable is to make them a proper code. A proper code is a code such that given any {\omega_1} and {\omega_2}, {c(\omega_1)} is not a prefix of {c(\omega_2)}. There are several codes like this, but some are more efficient then others. Since the letters have different frequencies, it makes sense to code a frequent letter (say ‘e’ in English) with few bits and a letter that doesn’t appear much, say ‘q’ with more bits. We want to find a proper code to minimize:

\displaystyle \mathop{\mathbb E}[k(\omega)] = \sum_{\omega \in \Omega} k(\omega) p(\omega)

The celebrated theorem by Shannon shows that for any proper code (actually it holds more generally for any decodable code), we have {\mathop{\mathbb E}[k(\omega)] \geq H} where {H} is the entropy of the alphabet, defined as:

\displaystyle H = - \sum_{\omega} p(\omega) \log_2 p(\omega)

even more impressive is that we can achieve something very close to it:

Theorem 1 There is a code such that {\mathop{\mathbb E}[k(\omega)] \leq H + 1}.

With an additional trick we can get {H + \epsilon} for any {\epsilon > 0}. The first part is trickier and I won’t do here (but again, it is not as hard as I thought it would be). For proving that there is a code with average length {\leq H + 1} we use the following lemma:

Lemma 2 There is a proper code for {\Omega} with code-lengths {k(\omega)} if and only if {\sum_\omega 2^{-k(\omega)} \leq 1}

Proof: Let {N = \max_\omega k(\omega)} and imagine all the possible codewords of length {\leq N} as a complete binary tree. Since it is a proper code, no two codes {c(\omega_1)} and {c(\omega_2)} are in the same path to the root. So, picking one node as a codeword means that we can’t pick any node in the subtree from it. Also, for each leave, the is at most one codeword in its path to the root. Therefore we can assign each leaf of the tree to a single codeword or to no codeword at all. It is easy to see that a codeword with size {k(\omega)} has associated with it {2^{N - k(\omega)}} leaves. Since there are {2^N} leaves in total, we have that:

\displaystyle \sum_\omega 2^{N-k(\omega)} \leq 2^N

what proves one direction of the result. Now, to prove the converse direction, we can propose a greedy algorithm: given {\Omega} and {k(\omega)} such that {\sum_\omega 2^{-k(\omega)} \leq 1}, let {N = \max_\omega k(\omega)}. Now, suppose {k(\omega_1) \leq k(\omega_2) \leq k(\omega_3) \leq \hdots}. Start with {2^N} leaves in a whole block. Start dividing them in {2^{k(\omega_1)}} blocks and assign one to {\omega_1}. Now we define the recursive step: when we analyze {\omega_j}, the leaves are divided in {2^{k(\omega_j-1)}} blocks, some occupied, some not. Divide each free block in {2^{k(\omega_j) - k(\omega_j-1)}} blocks and assign one of them to {\omega_j}. It is not hard to see that each block corresponds to one node in the tree (the common ancestor of all the leaves in that block) and that it corresponds to a proper code. \Box

Now, using this we show how to find a code with with {\mathop{\mathbb E}[k(\omega)] \leq H + 1}. For each {\omega}, since {p(\omega) \in (0,1]} we can always find {k(\omega)} such that {\frac{1}{2} p(\omega) \leq 2^{-k(\omega)} \leq p(\omega)}. Now, clearly:

\displaystyle \sum_\omega 2^{-k(\omega)} \leq \sum_\omega p(\omega) = 1

and:

\displaystyle \mathop{\mathbb E}[k(\omega)] = \sum_\omega k(\omega) p(\omega) \leq \sum_\omega [1 - \log_2 p(\omega)] p(\omega) = H + 1

Cool, but now how to bring it to {H + \epsilon} ? The idea is to code multiple blocks at the same time (even if they are independent, we are not taking advantage of correlation between the blocks). Consider {\Omega^k} and the probability function induced on it, i.e.:

\displaystyle p_k (\omega_1, \hdots, \omega_k) = \prod_{i=1}^k p(\omega_i)

It is not hard ot see that {\Omega^k} with {p_k} has entropy {kH} because:

\displaystyle \begin{aligned} \sum_{\omega_1, \hdots, \omega_k} p_k(\omega_1, \hdots, \omega_k) \log_2 p_k(\omega_1, \hdots, \omega_k) =\\ = \sum_{\omega_1, \hdots, \omega_k} \prod_i p(\omega_i) \sum_i \log_2 p(\omega_i) =\\ = \sum_i \sum_\omega p(\omega) \log_2 p(\omega) = kH \end{aligned}

and then we can just apply the last theorem to that: we can find a function that codifies {k} symbols {\omega = (\omega_1, \hdots, \omega_k)} with {l(\omega)} symbols such that:

{kH \leq \mathop{\mathbb E}[l(\omega)] \leq kH + 1} since {l(\omega)} codifies {k} symbols, we are actually interested in {\mathop{\mathbb E}[l(\omega)/k]} and therefore we get:

\displaystyle H \leq \mathop{\mathbb E}\left[\frac{l(\omega)}{k}\right] \leq H + \frac{1}{k} ]]> http://www.bigredbits.com/archives/145/feed 0