The AstroStat Slog » introduction http://hea-www.harvard.edu/AstroStat/slog Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 09 Sep 2011 17:05:33 +0000 en-US hourly 1 http://wordpress.org/?v=3.4 arxiv list http://hea-www.harvard.edu/AstroStat/slog/2009/arxiv-list/ http://hea-www.harvard.edu/AstroStat/slog/2009/arxiv-list/#comments Thu, 10 Dec 2009 21:18:36 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=4122 When I begin to subscribe arXiv/astro-ph and arXiv/stat, although only for a year I listed astro-ph papers featuring relatively advanced statistics, I also kept more papers relevant to astrostatistics beyond astro-ph or introducing hot topics in statistics and computer science for astronomical data applications. While creating my own arXiv as follows, I had a hope to write up short introductions of statistics that are unlikely known to most of astronomers (like my MADS) and matching subjects/targets in astronomy. I thought such effort could spawn new collaborations or could expand understanding of statistics among astronomers (see Magic Crystal). Well, I couldn’t catch up the growth rate and it’s about time to terminate the hope. However, I thought some papers can be useful to some slog subscribers. I hope they do.

  • [0704.1743] Fukugita, Nakamura, Okamura, et al (catalogue of morphologically classified galaxies from the SDSS database for trying various machine learning algorithms for automated classification)
  • [0911.1015] Gudendort, Segers ( Extreme-Value Copulas)
  • [0710.2024] Franz (Ratios: A short guide to confidence limits and proper use)
  • [0707.4473] Covey, Ivezic, Schlegel, Finkbeiner, et al. (Outliers in SDSS and 2MASS)
  • [0511503] (astro-ph) MNRAS,Nolan, Harva, Kaban, Raychaudhury, data driven bayesian approach
  • [0505017] (cs) Abellanas, Clavero, Hurtado, Delaunay depth
  • [0706.2704] (astro-ph) Wang, Zhang, Liu, Zhao (SDSS, kernel regression) Quantile regression can be applied
  • [0805.0056] Kong, Mizera, Quantile Tomography: using quantiles with multivariate data
  • [0907.5236] Gosh, Resnick Mean Excess Plots, Pareto
  • [0907.3454] Rinaldo, Wasserman (Low-Noise Density Clustering)
  • [0906.3979] Friendly (Golden Age of Statistical Graphics)
  • [0905.2819] Benjamini, Gavrilov (FDR control)
  • [0903.2651] Ambler, Silverman (Spatial point processes)
  • [0906.0562] Loubes, Rochet, Regularization with Approx. L^2 maximum entropy method
  • [0904.0430] Diederichs, Juditski, et al (Sparse NonGaussian Component Analysis)
  • [0905.0454] McWhirter,Proudler (eds) *Mathematics in Signal Processing V*
    [Tensor Decompositions, by *Peirre Comon*]
  • [0904.3842] Li, Dong (Dimension Reduction)
  • [0903.1283] Wiesel, Eldar, Hero III (Covariance estimation, graphical models)
  • [0904.1148] Beynaud-Bouret, Rivoirard
  • [0903.5147] cai, Zhou (Data driven BLock Thresholding approach to wavelet estimation)
  • [0905.0483] Harmany, Marcia, Willet (Sparse Poisson intensity reconstruction)
  • [0904.2949] Jhort, McKeague, van Keilegom (Empirical Likelihood)
  • [0809.3373] (astro-ph) Bailer-Jones, Smith, et al. (GAIA, SVM)
  • [0904.0156] Berger, Bernardo, Sun (formal definition of reference priors)
  • [0703360] (math.st) Drton *(LRTs and singularities)*
  • [0807.3719] Shi, Belkin, Bin Yu
  • [0903.5480] Andrieu, Roberts
  • [0903.3620] Casella, Consonni (Reconciling Model Selection and Prediction)
  • [0903.0447] Alqallaf, van Aelst et al (propa. outliers in multivariate data)
  • [0903.2654] Ambler, Silverman (Bayesian wvelet thresholding)
  • [0206366] (astro-ph) van de Weygaert, *Comis Foam*
  • [0806.0560] Noble, Nowak, Beyond XSPEC, ISIS
  • [0908.3553] Liang, Stochastic approximation (SAMC), Bayesian model selection
  • [0804.3829] Liu, Li, *Hao,* Jin
  • [0802.2097] Roelofs, Bassa, et al
  • [0805.3983] Carlberg, Sullivan, et al (Clusering of SN IA host galaxies)
  • [0808.0572] *Efron, Microarrays, Empirical Bayes, and Two groups model*
  • [0805.4264] Tempel, Einasto, Einasto, Saar, Anatomy of galaxy functions
  • [0909.0170] Estate, Khmaladze, Koul, (GoF problem for errors in nonparametric regression: dist’n free approach)
  • [0909.0608] *Liu, Lindsay*
  • [0702052] de Wit, Auchere (astro-ph, multispectral analysis of solar EUV images)
  • [0508651] Pires, Juin, Yvon, et al (astro-ph, Sunyaev-Zel’dovich clusters)
  • [0808.0012] Caticha (on slog, lectures on prob., entropy & stat. physics)
  • [0808.3587] Verbeke, Molenberghs, Beunckens, Model selection with incomplete data
  • [0806.1487] Scheider et al. Sim. and cos. inference: a statistical model for power spectra means and covarances.
  • [0807.4209] Adamakis, Morton-Jones, Walsh (solar physics, Bayes Factor)
  • [0808.3852] Diaconis, Khare, Saloff-Coste
  • [0807.3734] Rocha, Zhao, *Bin Yu* (SPLICE)
  • [0807.1005] Erven, Grunwald, Rooij ( … AIC-BIC dilemma)
  • [0805.2838] *E.L. Lehmann* (historical account)
  • [0805.4136] Genovese, Freeman, Wasserman, Nichol, Miller
  • [0806.3301] Tibshirani (not robert, but ryan)
  • [0706.3622] Wittek, Barko (physics,data-an)
  • [0805.4417] Georgakakis, et at (logN-logS, a bit fishy to me)
  • [0805.4141] Genovese, Perone-Pacifico, et al
  • [0806.3286] Chipman, George, McChulloch (BART)
  • [0710.2245] Efron (size, power, and FDR)
  • [0807.2900] Richards, Freeman, Lee, Schafer (PCA)
  • [0609042] (math.ST) Hoff (SVD)
  • [0707.0701] (cs.AI) Luss, d’Aspremont (Sparse PCA)
  • [0901.4252] Benko, Hardle, Kneip (Common Functional PC)
  • [0505017] (cs.CG) Abellanas, Claverol, Hutado (Delaunay depth)
  • [0906.1905] (astro-ph.IM) Guio, Achilleos, VOISE, Voronoi Image Segmentation algorithm
  • [0605610] (astro-ph) Sochting, Huber, Clowes, Howell (FSVS Cluster Catalogue, Voronoi Tessellation)
  • [0611473] (math.ST) Rigollet, Vert, Plug-in, Density Level Sets
  • [0707.0481] Lee, Nadler, Wasserman (Treelets)
  • [0805.4417] Georgakakis, et at (logN-logS, a bit fishy to me)
  • [0805.4141] Genovese, Perone-Pacifico, et al
  • [0806.3286] Chipman, George, McChulloch (BART)
  • [0710.2245] Efron (size, power, and FDR)
  • [0807.2900] Richards, Freeman, Lee, Schafer (PCA)
  • [0609042] (math.ST) Hoff (SVD)
  • [0707.0701] (cs.AI) Luss, d’Aspremont (Sparse PCA)
  • [0901.4252] Benko, Hardle, Kneip (Common Functional PC)
  • [0505017] (cs.CG) Abellanas, Claverol, Hutado (Delaunay depth)
  • [0906.1905] (astro-ph.IM) Guio, Achilleos, VOISE, Voronoi Image Segmentation algorithm
  • [0605610] (astro-ph) Sochting, Huber, Clowes, Howell (FSVS Cluster Catalogue, Voronoi Tessellation)
  • [0611473] (math.ST) Rigollet, Vert, Plug-in, Density Level Sets
  • [0707.0481] Lee, Nadler, Wasserman (Treelets)
  • [0805.2325] (astro-ph) Loh (block boostrap, subsampling)
  • [0901.0751] Chen, Wu, Yi (Copula, Semiparametric Markov Model)
  • [0911.3944] White, Khudanpur, Wolfe (Likelihood based Semi-Supervised Model Selection with applications to Speech Processing)
  • [0911.4650] Varoquaux, Sadaghiani
  • [0803.2344] Vossen
  • [0805.0269] Leach et al (Component Separation methods for the Plank mission: Appendix reviews various component separation/dimension reduction methods)
  • [0907.4728] Arlot, Celisse (survey of CV for model selection)
  • [0908.2503] Biau, Parta (sequential quantile prediction of time series)
  • [0905.4378] Ben-Haim, Eldar, (CRBound for Sparse Estimation)
  • [0906.3082] Cohen, Sackrowitz, Xu (Multiple Testing for dependent case)
  • [0906.3091] Sarkar, Guo (FDR)
  • [0903.5161] Rinner, Dickhaus, Roters (FDR)
  • [0810.4808] Huang, CHen (ANOVA, coefficient, F-test for local poly. regression)
  • [0901.4752] Chretien, (Robust est. of Gaussian mixtures)
  • [0908.2918] James, Wang, Zhu (Functional linear regression)
  • [0908.3961] Clifford, Cosma
  • [0906.3662] Lindquist (stat. anal. fMRI data)
  • [0706.1062] Clauset, Shalizi, Newman (PowerLaw dist’n)
  • [0712.0881] Zuo, Hastie, Tibshirani (DoF, Lasso)
  • [0712.0901] Jiang, Luan, Wang
  • [0705.4020] Chattopadhyay, Misra, et al (GRB, classification, model based)
  • [0707.1891] Holmberg, Nordstrom, Anderson (isochrones, calibration, Geneva-Copenhagen)
  • [0708.1510] Cobb, Bailyn, Connecting GRBs and galaxies:
  • [0705.2774] Kelly
  • [0708.0302] Chamger, James, Lambert, Wiel (incremental quantile, monitoring)
  • [0708.0169] Mikhail, Data-driven goodness of fit tests, attempts to generalize the theory of score tests
  • [0706.1495] Huskova, Kirch, Bootstrapping CI for the change point of time series
  • [0708.4030] Richer, Dotter, et al (NGC6397, GC, CMD, LF)
  • [0708.1071] Shepp, Statistical thinking: From Tukey to Vardi and beyond
  • [0708.0499] *Hunter, Wang, Hettmansperger *
  • [0704.0781] Cabrera, Firmani et al (Swift, long GRBs)
  • [0706.2590] Ramos, &Extreme Value Theory and the solar cycle (pareto dist’n, survival)*
  • [0706.2704] Wang, Zhang, Liu, Zhao (kernel regression, CV, redshift) <- quantile regression?
  • [0707.1611] Budavari, Szalay, (identification, Bayes factor)
  • [0707.1900] Vetere, Soffitta, et al. (GRB, BeppoSAX)
  • [0707.1982] Kim, *Liddle* (random matrix mass spectrum)
  • [0707.2064] Allen, (Star Formation, Bayesian)
  • [0011057] (hep-ex) Cranmer, Kernel Estimation in High Energy Physics
  • [0512484] (astro-ph) Mukherjee, Parkinson, Corasaniti, *Liddle* (model selection, dark energy)
  • [0701113] (astro-ph) Liddle (information criteria for astrophysical model selection)
  • [0810.2821] Cozman, concentration inequalities and LLNs under irrelevance of lower and upper expectations.
  • [0810.5275] Hall, Park, Samworth
  • [0709.1538] Einbeck, Evers, *Bailer-Jones*, localized principal components
  • [0804.4068] *Pires, Stark*, et al, LASTLens (week lensing)
  • [0804.0713] Delaigle, Hall, Meister
  • [0802.0131] (astro-ph) Bobin, Starck, Ottensamer (*Compressed Sensing* in Astronomy)
  • [0803.1708] Taylor, Worsley, (Random Fields of Multivariate Test Statistics, shape analysis)
  • [0803.1736] Salibian-Barrera, Yohai (high breakdown point robust regression, censored data)
  • [0803.4026] Amini, Wainwright, (Sparse Principal Components)
  • [0803.1752] Ren, (weighted empirical liklihood)
  • [0803.3863] Efron (simultaneous inference)
  • [0801.3552] Clifford, Cosma, probabilistic counting algorithms
  • [0802.1406] Blanchard, Roquain (multiple testing)
  • [0707.2877] van de Weygaert
  • [0806.3932] Vavrek, Balazs, Meszaros, etal (testing the randomness in the sky distribution of GRBs), MNRAS, 391(3), 2008
  • [0911.3769] Chan, Spatial clustering, LRT
  • [0911.3749] Hall, Miller
  • [0909.0184] Chan, Hall robust nearest neighbor methods for classifying high dimensional data
  • [0911.3827] Jung, Marron, PCA High Dim
  • [0911.3531] Owen, Karl Pearson’s meta analysis revisited
  • [0911.3501] Wang, Zhu, Zhou, Quantile regression varying coefficient models
  • [0505200] (physica) *Pilla, Loader, Taylor*
  • [0501289] (math.ST) *Meinshausen, Rice* Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses.
  • [0806.1326] Velez, Ariste, Semel (PCA, Sun, magnetic fields)
  • [0906.4582] *Belabbas, Wolfe*, PCA, high-dimensional data
  • [0903.3002] Huang, Zhang, Metaxas Learning with Structured Sparsity
  • [9209010] (gr-qc) Finn, Detection, Measurement, and Gravitational Radiation
  • [0112467] (astro-ph) Petrosian
  • [0103040] (astro-ph) Peebles, N-point correlation functions
  • [9912329] (astro-ph) Kerscher, Stat. analysis of large scale structure in the universe Minkowski functional and J function
  • [0107417] Connolly, Scranton, et al. Early SDSS
  • [0511503] (math.ST) Pilla, Loader, Volume-of-Tube Formula: Perturbation tests, mixture models, and scan statistics
  • [0503033] (astro-ph) Battye, Moss
  • [0504022] (astro-ph) Trotta, Applications of Bayes Model Selection to cosmological parameters
  • [0304301] (astro-ph) Nakamichi, Morikawa, AIC, is galaxy dist’n non-extensive and non-gaussian?
  • [0110230] (astro-ph) Nichol, Chong, Connolly, et al
  • [0806.1506] (astro-ph) Unzicker, Fischer, 2D galaxy dist’n, SDSS
  • [0304005] (astro-ph) Maller, McIntosh, et al. (Angular correlation funtion, Power spectrum)
  • [0108439] (astro-ph) Boschan (angular and 3D correlation functions)
  • [9601103 (astro-ph) Maddox, Efstathiou, Sutherland (sys errors, angular correlation function)
  • [0806.0520] Vio, Andreani
  • [0807.4672] Zhang, Johnson, Little, Cao
  • [0911.4546] Hobert, Roy, Robert
  • [0911.4207] Calsaverini, Vicente (information theory and copula)
  • [0911.4021] Fan, Wu, Feng (Local Quasi-Likelihood with a parametric guide) *
  • [0911.4076] Hall, Jin, Miller
  • [0911.4080] Genovese, Jin, Wasserman
  • [0802.2174] Faure, Kneib, et al. (strong lense, COSMOS)
  • [0802.1213] Bridle et al (Great08 Challenge)
  • [0711.0690] Davies, Kovac, Meise (Nonparametric Regression, Confidence regions and regularization)
  • [0901.3245] Nadler
  • [0908.2901] Hong, Meeker, McCalley
  • [0501221] (math) Cadre (Kernel Estimation of Density Level Sets)
  • [0908.2926] Oreshkin, Coates (Error Propagation in Particle Filters)
  • [0811.1663] *Lyons* (Open Statistical Issues in Particle Physics)
  • [0901.4392] Johnstone, Lu (Sparse Principle Component Analysis)
  • [0803.2095] Hall, Jin (HC)
  • [0709.4078] Giffin (… Life after Shannon)
  • [0802.3364] Leeb (model selection and evalutioin)
  • [0810.4752] Luxburg, Scholkopf (Stat. Learning Theory…)
  • [0708.1441] van de Weygaert, Schaap, The cosmic web: geometric analysis
  • [0804.2752] Buhlmann, Hothorn (Boosting algorithms…)
  • [0810.0944] Aydin, Pataki, Wang, Bullitt, Marron (PCA for trees)
  • [0711.0989] Chen (SDSS, volume limited sample)
  • [0709.1538] Einbeck, Evers, Bailer-Jones (Localized PC)
  • [0610835] (math.ST) Lehmann (On LRTs)
  • [0604410] (math.ST) Buntine, Discrete Component Analysis
  • [0707.4621] Hallin, Paindaveine (semiparametrically efficient rank-based inference I)
  • [0708.0079] Hallin, H. Oja, Paindaveine ( same as above II)
  • [0708.0976] Singh, Xie, Strawderman (confidence distribution)
  • [0706.3014] Gordon, Trotta (Bayesian calibrated significance levels.. the usage of p-values looks awkward)
  • [0709.0711] Quireza, Rocha-Pinto, Maciel
  • [0709.1208] Kuin, Rosen (measurement erros)
  • [0709.1359] Huertas-Company, et al (SVM, morphological classification)
  • [0708.2340] Miller, Kitching, Heymans, et. al. (Bayesian Galaxy Shape Measurement, weak lensing survey)
  • [0709.4316] Farchione, Kabaila (confidence intervals for the normal mean)
  • [0710.4245] Fearnhead, Papaspiliopoulos, Roberts (Particle Filters)
  • [0705.4199] (astro-ph) Leccardi, Molendi , an unbiased temp estimator for stat. poor X-ray specra (can be improved… )
  • [0712.1663] Meinshausen, *Bickel, Rice* (efficient blind search)
  • [0706.4108] *Bickel, Kleijn, Rice* (Detecting Periodicity in Photon Arrival Times)
  • [0704.1584] Leeb, Potscher (estimate the unconditional distribution of post model selection estimator)
  • [0711.2509] Pope, Szapudi (Shrinkage Est. Power Spectrum Covariance matrix)
  • [0703746] (math.ST) Flegal, Maran, Jones (MCMC: can we trush the third significant figure?)
  • [0710.1965] (physics.soc-ph) Volchenkov, Blanchard, Sestieri of Venice
  • [0712.0637] Becker, Silvestri, Owen, Ivezic, Lupton (in pursuit of LSST science requirements)
  • [0703040] Johnston, Teodoro, *Martin Hendry* Completeness I: revisted, reviewed, and revived
  • [0910.5449] Friedenberg, Genovese (multiple testing, remote sensing, LSST)
  • [0903.0474] Nordman, Stationary Bootstrap’s Variance (Check Lahiri99)
  • [0706.1062] (physics.data-an) Clauset, Shalizi, Newman (power law distributions in empirical data)
  • [0805.2946] Kelly, Fan, Vestergaard (LF, Gaussian mixture, MCMC)
  • [0503373] (astro-ph) Starck, Pires, Refregier (weak lensing mass reconstruction using wavelets)
  • [0909.0349] Panaretos
  • [0903.5463] Stadler, Buhlmann
  • [0906.2128] Hall, Lee, Park, Paul
  • [0906.2530] Donoho, Tanner
  • [0905.3217] Hirakawa, Wolfe
  • [0903.0464] Clarke, Hall
  • [0701196] (math) Lee, Meng
  • [0805.4136] Genovese, Freeman, Wasserman, NIchol, Miller
  • [0705.2774] Kelly
  • [0910.1473] Lieshout
  • [0906.1698] Spokoiny
  • [0704.3704] Feroz, Hobson
  • [0711.2349] Muller, Welsh
  • [0711.3236] Kabaila, Giri
  • [0711.1917] Leng
  • [0802.0536] Wang
  • [0801.4627] Potscher, Scheider
  • [0711.0660] Potscher, Leeb
  • [0711.1036] Potscher
  • [0702781] (math.st) Potscher
  • [0711.0993] Kabaila, Giri
  • [0802.0069] Ghosal, Lember, Vaart
  • [0704.1466] Leeb, Potscher
  • [0701781] (math) Grochenig, Potscher, Rauhut
  • [0702703] (math.ST) Leeb, Potscher
  • [astro-ph:0911.1777] Computing the Bayesian Evidence from a Markov Chain Monte Carlo Simulation of the Posterior Distribution (Martin Weinberg)
  • [0812.4933] Wiaux, Jacques (Compressed sensing, interferometry)
  • [0708.2184] Sung, Geyer
  • [0811.1705] Meyer
  • [0811.1700] Witten, Tibshirani
  • [0706.1703] Land, SLosar
  • [0712.1458] Loh, Zhu
  • [0808.4042] Commenges
  • [0806.3978] Vincent Vu, Bin Yu, Robert Kass
  • [0808.4032] Stigler
  • [0805.1944] astro-ph
  • [0807.1815] Cabella, Marinucci
  • [0808.0777] Buja, Kunsch
  • [0809.1024] Xu, Grunwald
  • [0807.4081] Roquain, Wiel
  • [0806.4105] Rofling, Ribshirani
  • [0808.0657 HUbert, Rousseeuw, Aelst
  • [0112467] (astro-ph) Petrosian
  • [0808.2902] Robert, Casella, A History of MCMC
  • [0809.2754] Grunwald, VItanyi, Algorithmic INofmration THeory
  • [0809.4866] Carter, Raich, Hero, An information geometric framework for DImensionality reduction
  • [0809.5032] Allman, Matias, Rhodes
  • [0811.0528] Owen
  • [0811.0757] Chamandy, Tayler, Gosselin
  • [0810.3985] Stute, Wang
  • [0804.2996] Stigler
  • [0807.4086] Commenges, Sayyareh, Letenneur…
  • [0710.5343] Peng, Paul, MLE, functional PC, sparse longitudinal data
  • [0709.1648] Cator, Jongbloed, et al. *Asymptotics: Particles, Processes, and Inverse problems*
  • [0710.3478] *Hall, Qiu, Nonparametric Est. of a PSF in Multivariate Problems*
  • [0804.3034] Catalan, Isern, Carcia-Berro, Ribas (some stellar clusters, LF, Mass F, weighted least square)
  • [0801.1081] Hernandez, Valls-Gabaud, estimation of basic parameters, stellar populations
  • [0410072] (math.ST) Donoho, Jin, HC, detecting sparse heterogeneous mixtures
  • [0803.3863] Efron
  • [0706.4190] Rondonotti, Marron, Park, SiZer for time series
  • [0709.0709] Lian, Bayes and empirical Bayes changepoint problems
  • [0802.3916] Carvalho, Rocha, Hobson, PowellSnakes
  • [0709.0300] Roger, Ferrera, Lahav, et al, Decoding the spectra of SDSS early-type galaxies
  • [0810.4807] Pesquet, et al. SURE, Signal/Image Devonvolution
  • [0906.0346] (cs.DM) Semiparametric estimation of a noise model with quantization errors
  • [0207026] (hep-ex) Barlow, Systematic Errors: Facts and Fictions
  • [0705.4199, Leccardi, Molendi, unbiased temperature estimator for statistically poor x-ray spectra
  • [0709.1208] Kuin, Rosen, measurement error Swift
  • [0708.4316] Farchione, *Kabila* confidence intervals for the normal mean utilizing prior information
  • [0708.0976] Singh, Xia, Strawderman confidence distribution
  • [0901.0721] Albrecht, et al. (dark energy)
  • [0908.3593] Singh, Scott, Nowak, adaptive hausdorff estimation of density level sets
  • [0702052] (astro-ph) de Wit, Auchere, Multipectral analysis, sun, EUV, morphology
  • [0706.1580] Lopes, photometric redshifts, SDSS
  • [0106038] (astro-ph) Richards et al photometric redshifts of quasars
]]>
http://hea-www.harvard.edu/AstroStat/slog/2009/arxiv-list/feed/ 0
Where is ciao X ? http://hea-www.harvard.edu/AstroStat/slog/2009/where-is-ciao-x/ http://hea-www.harvard.edu/AstroStat/slog/2009/where-is-ciao-x/#comments Thu, 30 Jul 2009 06:57:00 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=3260 X={ primer, tutorial, cookbook, Introduction, guidebook, 101, for dummies, …}

I’ve heard many times about the lack of documentation of this extensive data analysis system, ciao. I saw people still using ciao 3.4 although the new version 4 has been available for many months. Although ciao is not the only tool for Chandra data analysis, it was specifically designed for it. Therefore, I expect it being used frequently with popularity. But the reality is against my expectation. Whatever (fierce) discussion I’ve heard, it has been irrelevant to me because ciao is not intended for statistical analysis. Then, out of sudden, after many months, a realization hit me. ciao is different from other data analysis systems and softwares. This difference has been a hampering factor of introducing ciao outside the Chandra scientist community and of gaining popularity. This difference was the reason I often got lost in finding suitable documentations.

http://cxc.harvard.edu/ciao/ is the website to refer when you start using ciao and manuals are listed here, manuals and memos. The aforementioned difference is that I’m used to see Introduction, Primer, Tutorial, Guide for Beginners at the front page or the manual websites but not from the ciao websites. From these introductory documentations, I can stretch out to other specific topics, modules, tool boxes, packages, libraries, plug-ins, add-ons, applications, etc. Tutorials are the inertia of learning and utilizing data analysis systems. However, the layout of ciao manual websites seems not intended for beginners. It was hard to find basics when some specific tasks with ciao and its tools got stuck. It might be useful only for Chandra scientists who have been using ciao for a long time as references but not beyond. It could be handy for experts instructing novices by working side by side so that they can give better hands-on instruction.

I’ll contrast with other popular data analysis systems and software.

  • When I began to use R, I started with R manual page containing this pdf file, Introduction to R. Based on this introductory documentations, I could learn specific task oriented packages easily and could build more my own data analysis tools.
  • When I began to use Matlab, I was told to get the Matlab primer. Although the current edition is commercial, there are free copies of old editions are available via search engines or course websites. There other tutorials are available as well. After crashing basics of Matlab, it was not difficult to getting right tool boxes for topic specific data analysis and scripting for particular needs.
  • When I began to use SAS (Statistical Analysis System), people in the business said get the little SAS book which gives the basis of this gigantic system, from which I was able to expend its usage for particular statistical projects.
  • Recently, I began to learn Python to use many astronomical and statistical data analysis modules developed by various scientists. Python has its tutorials where I can point for basic to fully utilize those task specific modules and my own scripting.
  • Commericial softwares often come with their own beginners’ guide and demos that a user can follow easily. By acquiring basics from these tutorials, expending applications can be well directed. On the other hands, non-commercial softwares may be lack of extensive but centralized tutorials unlike python and R. Nonetheless, acquiring tutorials for teaching is easy and these unlicensed materials are very handy whenever problems are confronted under various but task specific projects.
  • I used to have IDL tutorials on which I relied a lot to use some astronomy user libraries and CHIANTI (atomic database). I guess the resources of tutorials have changed dramatically since then.

Even if I’ve been navigating the ciao website and its threads high in volume so many times, I only come to realize now that there’s no beginner’s guide to be called as ciao cookbook, ciao tutorial, ciao primer, ciao primer, ciao for dummies, or introduction to ciao at the visible location.

This is a cultural difference. Personal thought is that this tradition prevents none Chandra scientists from using data in the Chandra archive. A good news is that there has been ciao workshops and materials from the workshops are still available. I believe compiling these materials in a fashion that other beginners’ guides introducing the data analysis system can be a good starting point for writing up a front-page worthy tutorial. The existence of this introductory material could embrace more people to use and to explore Chandra X-ray data. I hope these tutorials from other softwares and data analysis systems (primer, cookbook, introduction, tutorial, or ciao for dummies) can be good guide lines to fully compose a ciao primer.

]]>
http://hea-www.harvard.edu/AstroStat/slog/2009/where-is-ciao-x/feed/ 1