Learn how Goldfinch Bio is able to scale up genome analysis using AWS EMR and Hail, and how they use this solution to analyze vast amounts of genomic data as part of their Kidney Genome Atlas. Hail is an open-source, python-based data analytics library with extra data types and methods for working with genomic data. Hail sees wide use in the life sciences community, but requires many customizations to take full advantage of AWS's EMR platform.
Privo built a new automated AMI pipeline using Hashi Corp's Packer. This new pipeline allowed Goldfinch to automate the installation of Hail along with all its dependencies. This AMI was made available as a public resource so that they can launch a new cluster at any time without having to worry about Hail breaking.
Are you in a similar situation as Goldfinch Bio? Trapped by your outdated systems and inefficient solutions? Let us take a look at what you have and offer suggestions for improvement with a Well-Architected Review.
That will allow our team to learn about your system and put together a digital transformation roadmap to help modernize your infrastructure.
Prior to adopting this new solution, when clusters were torn down they would lose code, work, and most importantly - time. The new solution separates concerns and gives them the ability to now have clusters running independently of the notebooks.
Existing clusters took over 45 minutes to bootstrap. With the new solution, the clusters launch in about 7 to 8 minutes. Goldfinch is now much more willing to change cluster configurations, and launch clusters tailored to the compute job because they can be more agile with this new solution in place.
With Privo owning the responsibility of maintaining the Hail AMI, Goldfinch can focus their time and effort on creating tailored clusters to their data scientists, and not have to worry about building and maintaining the complexities of Hail.
“Privo helped us get where we are with this solution and we couldn't have built this on our own. They were a great benefit to our business and we are excited to keep working with them”
- Adam Tebbe, Senior Director of IT and Informatics, Goldfinch Bio
Privo is an APN Premier Consulting Partner with offices near Boston and San Francisco that helps organizations migrate, optimize, monitor and manage Amazon WorkSpaces, AppStream 2.0, and other AWS services that support end users and their access to back office IT infrastructure.
AWS allows you to have an agile, cost-effective, and scalable infrastructure to enable operational efficiency, and simplify access to applications for end users. With AWS, companies of all sizes can focus more on their core business instead of IT administration.
Privo, A Navisite Company, is an AWS consulting firm with offices near Boston and San Francisco. We have about 45-ish people, all AWS engineers save for a brave few. There are a few things that make us different, but it boils down to this: we hire smart people who love working with others to solve complex problems. We employ the best people you’ll never hire.
Boston Office
400 West Cummings Park, Suite 3250
Woburn MA 01801
San Francisco Office
2120 University Ave
Berkeley, CA 94704