Abstract
Cloud storage service is pervasive nowadays. Many cloud storage services leverage distributed file systems as their base storage. Users usually upload files such as document files, image files and music files that are usually smaller than 10MB. Therefore, most files in cloud storage may be small files smaller than 10MB. There are few large files such as operating system image files, Ubuntu images for example, or some video files such as movies, which are usually bigger than 1GB. Moreover, distribution of file sizes obeys Heavy-Tailed Distribution according to past researches. As a result, we could assume that most files in cloud storage are small files and there are just 10% to 20% large files. Based on the observation above, the goal of this thesis is to design a distributed file system with data allocation mechanisms which perform high write throughputs for large small files and few very large files. This thesis proposes two files allocation mechanisms for hybrid distributed file system based on Ceph and HDFS. The result shows that our hybrid mechanism for hybrid distributed file system outperforms either HDFS or Ceph around 200%. To verify the feasibility and evaluate the performance on cloud storage services, we also implement and integrate the hybrid distributed file system with the files allocation mechanism on cloud storage services, SSBox.