How to summarise a collection of trees that came from a Bayesian analysis

After running a Bayesian phylogenetic analysis we are usually left with a large collection of trees, that came from the posterior distribution of the model given our data. Then if we want to work with a single tree - that is, to have a point estimate of this posterior distribution of trees - the most usual ways are to calculate the consensus tree or to select the most frequent tree. There are other ways, but let's fix on those by now.

We might not be aware of it, but when we choose for one or another summary we are in fact deciding for the tree estimate that minimizes its distance to all other trees in the set, and in expectation this will be the closest to the true tree under this distance metric (the so called Bayes estimator). This depends on what exactly do we mean by "distance" between trees, and that's what the article "Bayes Estimators for Phylogenetic Reconstruction" (doi 10.1093/sysbio/syr021) is about.

For example, the majority-rule consensus tree is the best we can get if we assume that the Robinson-Foulds distance (RF distance) is a good way of penalizing trees far away from the true one (I won't dwell into the meaning of "truth"; for us, the True tree^® is the one that originated the data). To be more explicit, the consensus tree is the one whose RF distance to all trees in the sample is the shortest possible. This will be the closest we can get to the true tree for this sample, if by "close" we mean "with a small RF distance".

Now suppose I don't like the RF metric because I can only count to two: if the trees are the same the distance is zero, but if they are different then the distance is not zero, and I don't care how different they are (think of apples and oranges). In this case the best representative of my sample is the one that appears more often, known as modal value or Maximum A Posteriori (MAP) value, since our sample comes from a posterior distribution. Is it the closest I can get to the true tree for this distribution? Yes, for this particular definition of distance: the MAP tree is the tree that maximizes the expected coincidence with the true tree.

In the article they also mention that if you want to find the tree that minimizes the expected quartet distance to the true value, then the quartet puzzling method will find this tree for you. But the quartet puzzling tree is not as easy to calculate as the consensus or MAP tree, and there is no straightforward way to find the tree that minimizes other distances in general (e.g. the dSPR, the geodesic distance or the Gene Tree Parsimony). Therefore the authors offer the well-known hill-climbing heuristics for finding the best tree, and use the squared path difference as an example of distance.

Below you can find the presentation I gave to my group last week about this paper, it contains basically some background information and a summary of their method. One thing that is absent from the slides are the results, which I briefly summarize below:

• their method (called "Bayes" in the figures or "BE") always used the path difference as distance measure; this is the overall distance they were trying to minimize.
• they simulated many data sets with several levels of sequence divergence, and reconstructed the phylogeny using Maximum Likelihood, Neighbor-Joining, and Bayesian analysis. From the Bayesian posterior distribution they elected as point estimates the consensus tree, MAP tree, and used their method to find the BE under the path difference.
• Figures 3 and 5 show the distance between the inferred and the true trees, where on figure 3 this distance is the path difference and in figure 5 it is the RF distance. As expected,  the Bayes estimator is better than any other measure at minimizing the path difference distance to the true tree, while the consensus tree wins if we want the closest in terms of RF distance.
• this result is rephrased in figures 8 and 9, which now look specifically at the distances between BE or MAP trees and the true tree. What they plot is distance(BE, true) - distance(MAP, true) for a different definition of distance(,) in each case. The MAP tree is correlated to the consensus tree (if the MAP frequency is larger than 50% they are equal, for instance). Therefore it should come as no surprise that if we define closeness to the true tree in terms of RF distance, the MAP tree will be closer than the BE as shown in figure 9. Because BE assumes that closeness to true is calculated in terms of the path difference, which is reinforced in figure 8.
• The authors wisely avoid offering the "best" Bayes estimator, since it depends on your judgment of how to penalize trees different from the true one.

Journal Club @ UVigo 2011.07.22

View more presentations from Leonardo de Oliveira Martins

OBS: This was my first time using beamer for Latex (after all these years, I know), so the slides are not prime time material. This is also my first submission to slideshare, and I like the idea of an embedded presentation within the blog post. I use latex a lot, and I think it would be easier for me to prepare a post with figures, equations and text within a presentation, and then simply embed it here with a minimum of extra text.

Maybe I'll try this next time, a presentation but with much more text than the recommended - in real life presentations the slides should support and complement but not replace the lecturer. Then you tell me if you would like to read on such a format or if you prefer a more traditional article-ish post.

Reference:
Huggins, P., Li, W., Haws, D., Friedrich, T., Liu, J., & Yoshida, R. (2011). Bayes Estimators for Phylogenetic Reconstruction Systematic Biology, 60 (4), 528-540 DOI: 10.1093/sysbio/syr021

New article describing priors on biomc2

Our new paper is out!

Leonardo de Oliveira Martins and Hirohisa Kishino (2010) Distribution of distances between topologies and its effect on detection of phylogenetic recombination. Annals of the Institute of Statistical Mathematics 62(1): 145--159. doi:10.1007/s10463-009-0259-8

Unfortunately I had to transfer the copyright to the ISM, but I am still allowed to maintain the accepted manuscript at my personal homepage [1]. In this article we describe in more details theoretical aspects that we couldn't explore before (because it would divert the audience's attention). The novelties are:

• describing the topology distance parameter as an augmented variable. It seems contradictory that our method assumes that the number of recombination break-points is variable, and at the same time we claim that the number of parameters is constant. This is because we have a variable (we could think of it as an indicator variable) that tells if there is recombination or not. Actually it is not just an indicator variable since it doesn't simply say if there is recombination or not, but actually approximates the amount of recombination. I like the analogy with linear models (the infamous Y=B0 + B1 X1 + B2 X2 + ... + Bn Xn), where despite the number of variables is constant (n+1) those parameters too close to zero mean that the "effective" number of variables is smaller.
• describing the mini-sampler procedure. There is a strategy in reversible-jump MCMC (rjMCMC) that allows the chain to walk a little bit before accepting/rejecting, that we refer to as the mini-sampler. As we just saw our model doesn't need a rjMCMC since the number of parameters is constant, but still we employ the same strategies, making sure that the moves respect the detailed balance of the chain. This mini-sampler is necessary because on the one hand each step changes the topology just a little, and on the other hand we want to be able to handle cases where neighboring topologies are very different.
• showing the importance of the modified Poisson distribution as a prior for the distances. We explicitly work with two scenarios:
  
  1. forcing the penalty hyperparameter to a fixed value. This shows the relevance of the hierarchical modelling.
     
  2. using a simplified distance that can only detect presence/absence of recombination witout being able to quantify it. This is a model analogous to other procedures that don't explicitly take the distance into account, and is equivalent to the "indicator variable" described above.
  
  
• description of the ensemble of mosaics and how to choose the most representative one. Each MCMC sample is one set of topologies (one per segment) that we call the mosaic structure. We then devise the calculation of a distance between these mosaics to quantify how similar two samples are, and find the centroid mosaic - the sample most similar to all other samples. It is worth noticing that
  
  1. this is done after the MCMC sampling is finished, and is not part of the Bayesian model per se
  2. this distance between samples is completely unrelated to the distance between topologies (that we call dSPR).
     
  
  

In the meanwhile I'm updating the software site and upgrading the program: nothing special, but I realized that many libraries were unnecessary and that biomc2.summarise was painfully slow for large topologies. Now it is only slow.

[1] the first time that you try to access any file hosted on https://corn.ab.a.u-tokyo.ac.jp/ your browser will complain about my self-signed certificate - some lobby is not happy about it. Please neglect the terrorist warnings.