The AstroStat Slog » computer vision http://hea-www.harvard.edu/AstroStat/slog Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 09 Sep 2011 17:05:33 +0000 en-US hourly 1 http://wordpress.org/?v=3.4 More on Space Weather http://hea-www.harvard.edu/AstroStat/slog/2009/more-on-space-weather/ http://hea-www.harvard.edu/AstroStat/slog/2009/more-on-space-weather/#comments Tue, 22 Sep 2009 17:03:11 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=3319 Thanks to a Korean solar physicist[1] I was able to gather the following websites and some relevant information on Space Weather Forecast in action, not limited to literature nor toy data.


These seem quite informative and I believe more statisticians and data scientists (signal and image processing, machine learning, computer vision, and data mining) easily collaborate with solar physicists. All the complexity, as a matter of fact, comes from data processing to be fed in to (machine, statistical) learning algorithms and defining the objectives of learning. Once settled, one can easily apply numerous methods in the field to these time varying solar images.

I’m writing this short posting because I finally found those interesting articles that I collected for my previous post on Space Weather. After finding them and scanning through, I realized that methodology-wise they only made baby steps. You’ll see a limited number key words are repeated although there is a humongous society of scientists and engineers in the knowledge discovery and data mining.

Note that the objectives of these studies are quite similar. They described machine learning for the purpose of automatizing the procedure of detecting features of interest of the Sun and possible forecasting relevant phenomena that affects our own atmosphere due to associated solar activities.

  1. Automated Prediction of CMEs Using Machine Learning of CME – Flare Associations by Qahwaji et al. (2008) in Solar Phy. vol 248, pp.471-483.
  2. Automatic Short-Term Solar Flare Prediction using Machine Learning and Sunspot Associations by Qahwaji and Colak (2007) in Solar Phy. vol. 241, pp. 195-211

    Space weather is defined by the U.S. National Space Weather Probram (NSWP) as “conditions on the Sun and in the solar wind, magnetosphere, ionosphere, and thermosphere that can influence the performance and reliability of space-borne and ground-based technological systems and can endanger human life or health”

    Personally thinking, the section of “jackknife” needs to be replaced with “cross-validation.”

  3. Automatic Detection and Classification of Coronal Mass Ejections by Qu et al. (2006) in Solar Phy. vol. 237, pp.419-431.
  4. Automatic Solar Filament Detection Using image Processing Techniques by Qu et al. (2005) in Solar Phy., vol. 228, pp. 119-135
  5. Automatic Solar Flare Tracking Using Image-Processing Techniques by Qu, et al. (2004) in Solar Phy. vol. 222, pp. 137-149
  6. Automatic Solar Flare Detection Using MLP, RBF, and SVM by Qu et al. (2003) in Solar Phy. vol. 217, pp.157-172. pp. 157-172

I’d like add a survey paper on another type of learning methods beyond Support Vector Machine (SVM) used in almost all articles above. Luckily, this survey paper happened to address my concern about the “practices of background subtraction” in high energy astrophysics.

A Survey of Manifold-Based Learning methods by Huo, Ni, Smith
[Excerpt] What is Manifold-Based Learning?
It is an emerging and promising approach in nonparametric dimension reduction. The article reviewed principle component analysis, multidimensional scaling (MDS), generative topological mapping (GTM), locally linear embedding (LLE), ISOMAP, Laplacian eigenmaps, Hessian eigenmaps, and local tangent space alignment (LTSA) Apart from these revisits and comparison, this survey paper is useful to understand the danger of background subtraction. Homogeneity does not mean constant background to be subtracted, often cause negative source observation.

More collaborations among multiple disciplines are desired in this relatively new field. For me, it is one of the best data and information scientific fields of the 21st century and any progress will be beneficial to human kind.

  1. I must acknowledge him for his kindness and patience. He was my wikipedia to questions while I was studying the Sun.
]]>
http://hea-www.harvard.edu/AstroStat/slog/2009/more-on-space-weather/feed/ 0
An excerpt from … http://hea-www.harvard.edu/AstroStat/slog/2009/an-excerpt-from/ http://hea-www.harvard.edu/AstroStat/slog/2009/an-excerpt-from/#comments Thu, 26 Feb 2009 20:07:13 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=1701 I’ve been complaining about how one can do machine learning on solar images without a training set? (see my comment at the big picture). On the other hand, I’m also aware of challenges in astronomy that data (images) cannot be transformed freely and be fed into standard machine learning algorithms. Tailoring data pipelining, cleaning, and processing to currently existing vision algorithms may not be achievable. The hope of automatizing the detection/identification procedure of interesting features (e.g. flares and loops) and forecasting events on the surface of the Sun is only a dream. Even though the level of image data stream is that of tsunami, we might have to depend on human eyes to comb out interesting features on the Sun until the new paradigm of automatized feature identification algorithms based on a single image i.e. without a training set. The good news is that human eyes have done a superb job!

From A Survey of the Statistical Theory of Shape by David G. Kendall, Statistical Science, Vol. 4, No. 2 (May, 1989), pp. 87-99.
It is well known that no classical test for two dimensional stochastic point processes can match the performance of the human eye and brain in detecting the presence of improbably large holes in the realized pattern of points. This fact has generated a great deal of research in the last few years, especially in connection with the large “voids” and long “strings” that the eye sees (or declares that it sees) in maps of the Shane and Wirtanen catalogue of positions of galaxies. Astronomers are interested in (i)whether these phenomena are sufficiently extreme to require explanation, and if so (ii) whether any of the various “model” universes now in vsgue can be said to display them to just the same degree.

]]>
http://hea-www.harvard.edu/AstroStat/slog/2009/an-excerpt-from/feed/ 0