Created: 01 Oct 2012 | Modified: 23 Jul 2020 | BibTeX Entry | RIS Citation |
Montgomery Slatkin proposed an “exact test” (in the sense of enumeration of exact probability for an observation) for neutrality of a sample of N individuals, in which K alleles are observed (Slatkin 1994, 1996). The test calculates the probability of getting a sample of alleles “more extreme” than the observed data from the Ewens Sampling Formula (or Distribution). By way of review, the ESD gives the distribution of alleles in a sample of fixed known size, from a population at quasi-stationary equilibrium, characterized by the Wright-Fisher infinite-alleles model or its close relations. The Slatkin (or Ewens) exact test is two-tailed, and a sample can fall outside the chosen critical region (at a given alpha level) either by having too few rare alleles, or far too even a distribution.
Since the Slatkin/Ewens test does not rely upon specific details of a genetic system, it is potentially ideal for testing neutrality in CT modeling. However, its statistical power against alternative models can be limited. Lansing et al. 2008 measured power against an alternative hypothesis of male dominance and non-random mating, for example.
In the present experiment, the goal is to understand the power of the Slatkin/Ewens test against alternative models of conformist (and anti-conformist) transmission, given that a number of archaeological studies have attempted to distinguish between neutral and conformist transmission using artifact class frequency data.
Statistical power is the converse of the probability of Type II error. In other words, it is the probability of not committing a Type II error, or a “false negative”). It measures the ability of a statistical test, in other words, to correctly detect data which derive from a process other than the data generating process designated as the “null hypothesis.”
In general, the “golden rule” is generally that a 5% probability of Type I error should be matched with a 20% probability of making a Type II error, which translates into a test with 80% power. This gives a 4:1 tradeoff between Beta-risk and Alpha-risk. This is pure convention. But since the Slatkin test reports the exact tail probability of its result, and we can similarly calculate the exact fraction of conformist samples that PASS the Slatkin test at a given alpha level, we don’t have to rely on convention and can examine directly how power changes as we vary:
R code for the analysis will be available on github when the analysis is complete (9/27/12).
Without Time Averaging
Without any temporal aggregation, we can see that the Slatkin test can be relatively powerful for detecting conformist violations of the neutral assumptions underlying the Ewens Sampling Distribution, but only under certain conditions:
With Time Averaging
When we aggregate observations over time, the picture is intriguingly different. The x-axis below is scaled duration of observation aggregation, where the duration of aggregation is divided by the mean lifetime of traits at that level of theta (innovation rate). This helps us visualize the time scale of aggregation in something other than arbitrary simulation units.
Immediately, we can see that temporal aggregation of observations from the “Global Most Common” model of conformism increases the power of the test. Apparently, as we pile up observations, our fixed sample of individuals come more and more to reflect a very different distribution of traits than the Ewens Sampling Distribution.
This effect is, of course, sharp and pronounced at higher levels of the conformism probability, such that at almost any level of innovation, if we have reason to believe that conformism is strong in a population, the Slatkin test is nearly 100% powerful in detecting it (even though, in my earlier 2012 paper, time averaging increases the chances of a Type I error which would lead one away from a conclusion of conformism).
At weaker levels of conformism, it takes relatively large amounts of time averaging to increase the power of the test, and only at innovation rates above 3.0 do we reach the usual 80% power which is taken to mark a test capable of distinguishing between hypotheses.
Montgomery Slatkin’s original C code: CODE
The original C code for Slatkin’s monte carlo version required that the input data be coded into the source, and then compiled anew for each data value. In order to automate doing many, many exact tests for this and related studies, I modified Slatkin’s original program to take all input on the command line, and producing abbreviated output (there are several versions) to make scripting and integration into R easy. Code is at:
Slatkin, M. 1994. “An Exact Test for Neutrality Based on the Ewens Sampling Distribution.” Genetical Research 64 (01). Cambridge Univ Press: 71–74.
———. 1996. “A Correction to the Exact Test Based on the Ewens Sampling Distribution.” Genetical Research 68 (03). Cambridge Univ Press: 259–60.
Watkins, J.C. 2010. “Convergence Time to the Ewens Sampling Formula.” Journal of Mathematical Biology 60: 189–206.