Storage aware data management system for Genomics

In recent years, nucleotide sequencing has become increasingly instrumental in both research and clinical settings. This has led to explosive growth in sequencing data produced worldwide along with an increase in complex analysis algorithms. As the amount of data and analysis increases, so does the need for automated solutions for processing and analysis. The concept of workflows has gained favor in the bioinformatics community, but there is little in the scientific literature describing end-to-end operational automation systems. We provided an automation system that aims at providing a solution to the genomics related operational challenges that face sequencing of both research and clinical facilities. We built on existing open-source technologies, with a modular design allowing for a community-driven effort to create plug and play services. In this research, we describe the system and elaborate on the underlying conceptual framework. Which can be reduced to 3 conceptual levels: Data tagging (using metadata automation), Classifying Storage systems (the steps involved in the classification of storage systems), and execution (using a series of rules to move data around on an operational level).