The AstroStat Slog » measurement error

Lost in Translation: Measurement Error

vlk — Sat, 03 Jan 2009 03:24:32 +0000

You would think that something like “measurement error” is a well-defined concept, and everyone knows what it means. Not so. I have so far counted at least 3 different interpretations of what it means.

Suppose you have measurements X={X_i, i=1..N} of a quantity whose true value is, say, X₀. One can then compute the mean and standard deviation of the measurements, E(X) and σ_X. One can also infer the value of a parameter θ(X), derive the posterior probability density p(θ|X), and obtain confidence intervals on it.

So here are the different interpretations:

Measurement error is σ_X, or the spread in the measurements. Astronomers tend to use the term in this manner.
Measurement error is X₀-E(X), or the “error made when you make the measurement”, essentially what is left over beyond mere statistical variations. This is how statisticians seem to use it, essentially the bias term. To quote David van Dyk

For us it is just English. If your measurement is different from the real value. So this is not the Poisson variability of the source for effects or ARF, RMF, etc. It would disappear if you had a perfect measuring device (e.g., telescope).
Measurement error is the width of p(θ|X), i.e., the measurement error of the first type propagated through the analysis. Astronomers use this too to refer to measurement error.

Who am I to say which is right? But be aware of who you may be speaking with and be sure to clarify what you mean when you use the term!

[Q] systematic error

hlee — Sat, 21 Jun 2008 03:02:07 +0000

What is systematic error? Can it be modeled statistically? Is it random? Is it fixed? Is it a bias? Is it …?
Astronomers sometimes call it measurement error. Statisticians, therefore, will dig in materials under the topic of measurement error. Don’t be fooled by the name. Statisticians’ measurement error generally corresponds to astronomers’ statistical error, not systematic error. This systematic error could be related to nuisance parameters, profile likelihoods, hierarchical Bayesian modeling, or a fixed bias (empirically known or other source provides its estimate – because of resourcing external information, it may be considered from a meta data analysis viewpoint) from a statistics perspective although astronomers have their sophisticatedly developed ways to treat this error. So far, unfortunately, there is no golden rule to include systematic error in the modeling process. Can you help me to get a better sense about the definition of systematic error with a statistical language? What would be your answer to the question at the top?

my first AAS. V. measurement error and EM

hlee — Fri, 20 Jun 2008 03:46:05 +0000

While discussing different view points on the term, clustering, one of the conversers led me to his colleague’s poster. This poster (I don’t remember its title and abstract) was my favorite from all posters in the meeting.

He rewrote the EM algorithm to include measurement errors in redshifts. Indexed parameters associated with different redshifts and corresponding standard deviations (measurement errors, treated as nuisance parameters) were included in the likelihood function that corrected bias and manifested bimodality in the LFs clearly at the different evolutionary stages.

I encouraged him to talk statisticians to characterize and to generalize his measurement error included likelihoods, and to optimize his EM algorithm. Because of approximations in algebra and the many parameters from measurement errors from redshifts, some assumptions and constraints were imposed intensively and I thought a collaboration with statisticians suits to get around constraints and to generalize his measurement error included likelihood.

[ArXiv] Swift and XMM measurement errors, Sep. 8, 2007

hlee — Tue, 11 Sep 2007 05:12:41 +0000

From arxiv/astro-ph:0708.1208v1:
The measurement errors in the Swift-UVOT and XMM-OM by N.P.M. Kuin and S.R. Rosen

The probability distribution of photon counts from the Optical Monitor on XMM Newton satellite (XMM-OM) and the UVOT on the Swift satellite follows a binomial distribution due to detector characteristics. Incident count rate was derived as a function of the measured count rate, which was shown to follow a binomial distribution.

Discrepancy between the mapping the 1σ Poisson incident count error onto measured count error and the mapping the 1σ binomial measurement count error onto incident counts was illustrated with an example of large counts per frame. Although this discrepancy for small counts per frame was commented to be negligible, the authors urge to use their binomial distribution based formalism to derive the errors in measurements.