Abstract
The speed of data transmission over high-speed networks such as Gigabit, Myrinet, and Infiniband has come close to that of the system bus in modern computers. Due to the relatively low speed of disk access, reading a cached file in a remote machine connected through a high-speed network is significantly faster than reading the file from a local disk. In this paper, we propose and implement a universal file cache called UFC that fully utilizes and integrates all memory resources in a high-speed distributed system for comprehensive file caching. UFC presents to all applications a single large-scaled read-only file cache. Through a set of publicly-accessed interfaces, every application can share cached files in UFC and avoid multiple caching. To be easily extended and deployed in a heterogeneous distributed system where machines are installed with different operating systems and connected with different communication media, we develop UFC as a middleware consisting of add-on cache drivers and communication daemons built on top of the well-supported BSD socket library. To efficiently locate a file in UFC, we present a novel file lookup algorithm and a set of cache management algorithms dealing with file migrations and replications. We illustrate the performance impact of UFC by a web server that accesses a large file set. The experimental results show that, by caching files in UFC and avoiding expensive disk access, we significantly improve the performance of the web server. Keywords: universal file cache, heterogeneous distributed systems, operating systems, clustered systems