The two talks will describe how data from the Sloan Digital Sky Survey has led to introducing data-intensive science to the astronomy community. In collaboration with Jim Gray we have built an interactive system serving most of the world’s astronomy community today. The tools were eventually modified to deal with other areas of science, and show how many of the data management and statistical challenges In data intensive science share many commonalities with each other. In order to be able to deal with scalability, we also had to address various complex issues from data layout to fine tuning hardware to novel statistical tools.
Copyright. All Rights Reserved.