Large Databases in Astronomy

The next-generation astronomy digital archives will cover most of the sky at fine resolution in many wavelengths, from X-rays through ultraviolet, op- tical, and infrared. The archives will be stored at diverse geographical locations. The intensive use of advanced data archives will enable astronomers to explore their data interactively. Data access will be aided by multidimensional spatial and attribute indices. The data will be partitioned in many ways. Small tag indices consisting of the most popular attributes will accelerate frequent searches. Split- ting the data among multiple servers will allow parallel, scalable I/O and parallel data analysis. Hashing techniques will allow efficient clustering, and pair-wise com- parison algorithms that should parallelize nicely. Randomly sampled subsets will allow debugging otherwise large queries at the desktop. Central servers will operate a data pump to support sweep searches touching most of the data. The anticipated queries will require special operators related to angular distances and complex sim- ilarity tests of object properties, like shapes, colors, velocity vectors, or temporal behaviors. These issues pose interesting data management challenges.