Consider a case of fitting a spectral line in addition to continuum with a delta function or a gaussian (normal) density function. Among many regularity conditions, personally the most bothersome one is identifiability. When the scale parameter (σ) goes to zero, we cannot tell which model, delta, or gaussian, is a right one. Furthermore, the likelihood ratio test cannot be applied to the delta function due to its discontinuity. For a classical confidence interval or a hypothesis test which astronomers are familiar with from Numerical Recipes, identifiability and the set property (topology) of model parameters suffer from the lack of attentions from astronomers who performs statistical inference on model parameters. I found a few astronomical papers that ignored this identifiability but used the likelihood ratio tests for an extra component discovery. Clearly, these are statistical malpractices.

Although math.st:0708.0499 did not discuss spectral line fitting, it offers a nice review on identifiability when inferencing for mixtures of symmetric distributions.

Goodness-of-Fit tests have been essential in astronomy to validate the chosen physical model to observed data whereas the limits of these tests have not been taken into consideration carefully when observed data were put into the model for estimating the model parameters. Therefore, I thought this paper would be helpful to have a thought on the different point of views between the astronomers’ practice of goodness-of-fit tests and the statisticians’ constructing tests. (Warning: the paper is abstract and theoretical.)

This paper began with presenting two approaches to constructing test statistics: 1. some measure of distance between the theoretical and empirical distributions like the Cramer-von Mises and the Komogorov-Smirnov statistics and 2. score test statistics, constructed in a way that the tests is asymptotically normal. As the second approach is preferred, the author confined his study to generalize the theory of score tests. The notion of the Neyman type (NT) test was introduced with very minimal assumptions to shape the statistics.

The author discussed the statistical inverse problems or the deconvolution problems of physics, seismology, optics, and imaging where noisy signals and measurements occur. These inverse problems induce the Neyman’s type statistics under appropriate regularity assumptions.

Other type of NT tests in terms of score functions and their consistency was presented in an abstract fashion.

Beyond this historic question, one thing more bothers me is mixing chi-square function with chi-square distribution. The former is not necessarily chi-square distributed but it is practiced that once chi-square function is written, the variable within the function will have a confidence interval automatically according to chi-square distribution with degrees-of-freedom. No checking procedure for regularity conditions.

Statistically and astronomically, answers to my question lead to correcting my knowledge and erasing my prejudice. Vinay wrote about chi-square fitting. This certainly gives a better account for my question. Or Numerical Recipes to follow how chi-square methods are used. I welcome all kind lessons, advice, and references to have extended knowledge and a better perspective about the meaning of chi-square to astronomers.