ICRAR set up theSkyNet project on AWS in only four days. The team can now quickly and efficiently expand the cloud infrastructure as the public volunteer more CPU cycles to support the initiative.

“The scalability of AWS has been enormously helpful,” says Associate Professor Vinsen. “I can add more capacity as I need it with minimal fuss. Using AWS allows us to process upwards of 150 GB of sky images and store more than 400 GB of imaging data every month.”

By using Amazon S3 as a key-value store, ICRAR is able to seamlessly index and manage input from hundreds of thousands of public CPUs around the world. Amazon ELB helps ICRAR manage the flow of data to and from theSkyNet community.

ICRAR uses Amazon EBS to store upwards of 400 GB worth of imaging data monthly as it is processed by the community. Amazon EC2 provides the compute capacity for ICRAR to analyze data from 400 and 500 galaxies simultaneously.

The project has proved to be enormously popular and shortly after the migration to AWS, online communities in Russia, America, and Australia overloaded ICRAR’s theSkyNet server. However, it only took Associate Professor Vinsen two hours to add additional capacity. “Other community computing projects have taken days to recover from overloads because they have to find more infrastructure resources to bring up new servers,” he says. “With AWS, I can just provision a bigger instance.”

ICRAR plans to use AWS to meet the ongoing computing requirements of future experiments within theSkyNet.