Viability of S3 Object Storage for the ASC Program at Sandia

The article discusses the viability of using Amazon S3 (S3) object storage for the Advanced Scientific Computing (ASC) program at Sandia National Laboratories. The authors, a team of researchers from various institutions, investigate the feasibility of using S3 as a scalable and cost-effective solution for storing and managing large datasets generated by scientific simulations.
Background and Context
Sand Sandia are collaborating to develop a comprehensive storage system for their high-performance computing (HPC) infrastructure. The goal is to provide efficient and reliable storage for theASC program, which supports various scientific applications running on Sandia’s HPC systems. The team evaluated S3 as a potential solution due to its reputation for scalability, reliability, and ease of use.
S3 Storage System Overview
Amazon S3 is a cloud-based object storage service designed to handle large amounts of data. It stores data in small files called objects, which can be accessed over the internet using standard HTTP protocols. S3 provides a simple storage interface and scales seamlessly to accommodate growing datasets. It also offers features like data encryption, versioning, and access controls, making it an attractive option for storing sensitive data.
Scalability and Performance Evaluation
The authors evaluated the scalability of S3 by simulating a large workload on their testbed. They created a mock dataset consisting of 100TB of data and distributed it across multiple nodes. They observed that S3 handled the workload efficiently, with minimal performance degradation as the dataset size increased. The authors also compared the performance of S3 with other storage solutions and found it to be competitive in terms of IOPS (input/output operations per second) and throughput.
Cost-Effectiveness Analysis
The authors performed a cost analysis to determine if using S3 would be cost-effective for the ASC program. They compared the cost of using S3 with other storage solutions, considering factors like capacity, performance, and data transfer rates. They found that S3 offered competitive pricing and could save costs significantly compared to traditional storage solutions.
Security and Compliance Considerations
The authors examined the security features of S3 to ensure it met Sandia’s security requirements. They found that S3 provides robust security measures, including data encryption and access controls, which can be customized according to Sandia’s needs. They also evaluated compliance issues and found that S3 adheres to various industry standards and regulations, such as HIPAA and PCI-DSS.
Conclusion
In conclusion, the article demonstrates that Amazon S3 is a viable storage solution for the ASC program at Sandia National Laboratories. Its scalability, performance, cost-effectiveness, security features, and compliance with industry standards make it an attractive option for storing large datasets generated by scientific simulations. The authors recommend further investigation into integrating S3 with Sandia’s HPC infrastructure to take full advantage of its capabilities.