Last month, we have published three articles discussing Clean Water Act (CWA) related statistics methods.
When the established nutrient criterion is used in CWA compliance assessment, data from an individual stream will be used and compared to the criterion, only the within stream variance is of concern. What is the implication of this bait and switch?
In the end of the review process, we ran into a very unfortunate mistake on the journal’s side. The journal’s editorial office ran a Plagiarism Checker on the manuscript and found “substantial” amount of text copied from the Internet. The editor rejected the manuscript. The copied text turned out to be several R functions I wrote for my textbook, which I included in the supporting document. When the book was translated into Japanese in 2010, the Japanese published posted all my code online. The editor promptly corrected the mistake. But I had to resubmit the manuscript as a new one, which is why the paper was accepted on the same day when it was submitted.
An interesting by-product of writing this paper is our new interpretation of the 10 percent rule (the raw data approach) discussed in Smith et al (2001). In that paper, the authors interpreted a 1997 EPA rule that stipulates how to declare a water is in compliance with a numeric water quality standard. The rule is know as the 10 percent rule because it declares a water out of compliance when more than 10% of the data exceed the numeric standard. Smith et al (2001) argued that the 10% of data should be interpreted as 10% of the time. Therefore the 10% rule requires the 0.9 quantile of the underlying concentration distribution to be below the numeric standard. They proposed statistical hypothesis testing. As a result of the paper, nearly all states in the US are now using hypothesis testing.
Through reviewing EPA’s interpretation of the CWA, we believe that the legal definition of a numeric standard is intended for the population mean, not the 0.9 quantile. The 10% rule is simply a margin of safety. When comparing the 0.9 quantile of pollutant concentration dsitribution to the standard, instead of the mean, we set a very high standard. But is it necessary?
Stein’s paradox suggests that CWA compliance assessment can benefit when we pool data from similar waters together and apply a shrinkage estimator. In the paper, we documented the benefit of a shrinkage estimator. Writing this paper also allowed me to think more about the idea of prior distribution in Bayesian inference. I often have doubts about the subjective interpretation of the prior, largely because of Daniel Kahneman’s work on how we don’t think in terms of probability. Kahneman’s work suggests that eliciting a prior distribution from a human expert is a risky business. Using the empirical Bayes interpretation of the Jame-Stein estimator, we see an alternative interpretation of a prior distribution. A prior distribution should be the distribution of the parameter of interest at a higer level aggregation. In our example, when using the Bayes estimator for estimating the mean concentration of a water, the prior of the mean should be the distribution of means of similar waters. With this interpretation, we can put a physical meaning to the prior, which will make the process of establishing an informative prior distribution easier.
<!-- dynamically load mathjax for compatibility with self-contained A Continuous Bayesian Networks (cBN) Model
In a paper appeared in Environmental Modelling and Software, we proposed a Bayesian Networks model using continuous variables (cBN). The model is a combination of the Gibbs sampler and empirical models through a graphical model representing the hypothesized causal links among relevant variables. When applied to a data collected for developing nutrient criterioa for Ohio’s small rivers and streams, we found that the concept of a single nutrient criterion for the entire state is impractical. In many cases, nutrient is not the primary factor affecting a stream’s ecological condition, something else (e.g., habitat quality) may be more important. As a result, we revised the mansucript three times to make it clear that we are not comfortable with the idea of a single nutrient criterion. A regional or even water-specific criterion is necessary. This argument should be a no-brainer as there are no two rivers that are exactly the same and the effect of nutrient on stream ecosystem will innevitably be also different, hence different nutrient criteria are needed. In the process of revising the manuscript, we noticed an interesting problem in EPA’s recommendedation on how to establish a nutrient criterion using reference condition. The approach takes the following steps:- Select streams that are largely not affected by human activities (good luck)
- Collect nutrient concentration data from these “reference” streams and calculate a median for each stream
- Pool these median values together to form the reference distribution
- The nutrient criterion should be the 75th percentile of the reference distribution.
When the established nutrient criterion is used in CWA compliance assessment, data from an individual stream will be used and compared to the criterion, only the within stream variance is of concern. What is the implication of this bait and switch?
In the end of the review process, we ran into a very unfortunate mistake on the journal’s side. The journal’s editorial office ran a Plagiarism Checker on the manuscript and found “substantial” amount of text copied from the Internet. The editor rejected the manuscript. The copied text turned out to be several R functions I wrote for my textbook, which I included in the supporting document. When the book was translated into Japanese in 2010, the Japanese published posted all my code online. The editor promptly corrected the mistake. But I had to resubmit the manuscript as a new one, which is why the paper was accepted on the same day when it was submitted.
The frequency component of a water quality standard
In the paper appeared in the journal Environmental Management, we revisited EPA documents on the three components of a water quality standard (or criterion): magnitude, duration, and frequency. Based on our reading of documents as far back as 1972, we found that the interpretation of magnitude and duration is unambiguous. However, we have not found a convincing discussion on the frequency component. Based on a 1985 EPA document, we believe that the frequency component is added to provide a margin of safety. Using this interpretation, we formulated a probabilistic approach to derive the necessary frequency to maintain a consistent level of confidence.An interesting by-product of writing this paper is our new interpretation of the 10 percent rule (the raw data approach) discussed in Smith et al (2001). In that paper, the authors interpreted a 1997 EPA rule that stipulates how to declare a water is in compliance with a numeric water quality standard. The rule is know as the 10 percent rule because it declares a water out of compliance when more than 10% of the data exceed the numeric standard. Smith et al (2001) argued that the 10% of data should be interpreted as 10% of the time. Therefore the 10% rule requires the 0.9 quantile of the underlying concentration distribution to be below the numeric standard. They proposed statistical hypothesis testing. As a result of the paper, nearly all states in the US are now using hypothesis testing.
Through reviewing EPA’s interpretation of the CWA, we believe that the legal definition of a numeric standard is intended for the population mean, not the 0.9 quantile. The 10% rule is simply a margin of safety. When comparing the 0.9 quantile of pollutant concentration dsitribution to the standard, instead of the mean, we set a very high standard. But is it necessary?
Implications of Stein’s paradox
This paper is published in Environmental Science and Technology, a paper we started (technically) in 1991 when we read Efron and Morris (1977) in a study group. Although I did not quite understand the mathematics (and the baseball example, because I had yet seen a baseball game), the conclusion that sample average is not the best estimator when multiple means are estimated simultaneously gave me a strong impression. In the next 20 years, we have talked about the paper from time to time. In 2006 while reading Gelman’s 2005 Bayesian ANOVA paper, I finally made the connection and went back to read the 1977 paper again. (I guess that we should definitely force our graduate students to form reading groups and read regularly.)Stein’s paradox suggests that CWA compliance assessment can benefit when we pool data from similar waters together and apply a shrinkage estimator. In the paper, we documented the benefit of a shrinkage estimator. Writing this paper also allowed me to think more about the idea of prior distribution in Bayesian inference. I often have doubts about the subjective interpretation of the prior, largely because of Daniel Kahneman’s work on how we don’t think in terms of probability. Kahneman’s work suggests that eliciting a prior distribution from a human expert is a risky business. Using the empirical Bayes interpretation of the Jame-Stein estimator, we see an alternative interpretation of a prior distribution. A prior distribution should be the distribution of the parameter of interest at a higer level aggregation. In our example, when using the Bayes estimator for estimating the mean concentration of a water, the prior of the mean should be the distribution of means of similar waters. With this interpretation, we can put a physical meaning to the prior, which will make the process of establishing an informative prior distribution easier.
-->
