The AstroStat Slog

Usama Fayyad (click the image to listen the lecture)



A Data Miner’s Story – Getting to Know the Grand Challenges

This talk was given as an award acceptance speech during KDD 2007 and sounded to aim for general pubic, who heard of Yahoo! My catch point was his overview on data mining, which seemed to be initiated from astronomy (his working at JPL was the opportunity for him to see the extraordinary capability of data mining in the real world to be applied in business modeling on top of extracting informations based on data). The whole story touched the fundamental of data mining.

A few times I attended talks at CfA because abstracts contain the phrase, data mining, but I always felt something is short. My impression from astronomers’ data ming talks is that they tend to focus only on collecting data, which is a small part of data mining although I understand collecting by itself is tremendously difficult. Usama Fayyad’s talk provided the answer to my skepticism as well as other great insights of data mining. Collecting data is not a small part from customer’s view points according to his evolutionary charts describing the real world data mining different from the scientific and technical side of data mining.

I thought astronomers particularly working hard on survey projects might be benefited from his pragmatic perspective of data mining. I hope at some point not just algorithms of collecting data but more embedded data mining tools to be exploited in future surveys and virtual observatory projects.

FYI, the description of the astronomy project that inspired him, called sky cat (I’m not sure) comes around 15 minutes and the talk starts around 6 minutes. The first 5 minutes or so were spent for introduction. And here is the link of another talk, which he pointed in this talk. Please, skip the first 8.5 minutes. You won’t regret it. Usama Fayyad’s talk is good but the opening is …



From Mining the Web to Inventing the New Sciences Underlying the Internet