Implementing Hierarchical Storage on the Basis of File Use

Shared file systems are widely used for storing electronic files, simplifying backup procedures and providing a collaboration environment in both small and large organisations. They are simple to use, cheap to implement and highly flexible but the difficulties in implementing a systematic disposal policy caused by lack of persistent ownership and file use data leads to a steady increase in storage volume. Attempts to reduce shared storage volume through quota charges may be highly effective but degrade data integrity. Although direct storage costs are decreasing, indirect costs such as backup, power and cooling, and risk such as down-time for a full server restoration increase with storage volume. This paper proposes methods of tracking the use of files on shared storage in order to provide a systematic disposal policy based on date since last file use. This policy is implemented as a hierarchical storage management system (HSM) where files not used for a certain time are replaced by shortcuts to a read-only copy on lower cost storage. Such a system can archive a larger proportion of data than HSMs where file status is determined from complex rules based primarily on file date metadata, and the recall rate of data from archive will be much lower. Studies indicate that as few as 1% of files on share drives are accessed over a period of a year within KAZ (an IT service company, although a much larger proportion are moved). Implementation of such a system allows automatic definition of disaster recovery datasets and reduction in disk mirroring requirements. Other methods of storage reduction by archiving on the basis of folder rather than file properties are discussed.