Tech Talk | Build a hybrid data lake and burst processing to Google Cloud Dataproc with Alluxio
As the amount of data analyzed and stored continues to grow exponentially, fixed on-premises infrastructure like Apache Hadoop data lakes becomes costly. Add to that the need to support newer and popular frameworks on an already busy data lake, it is not uncommon to see Hadoop-based data lakes running at beyond 100% utilization and hybrid processing split between physical and cloud infrastructure. As a result, companies are looking to leverage the flexibility and cost savings of the cloud.
Join us for this tech talk where we will show you how Alluxio can help burst your private computing environment to Google Cloud, minimizing costs and I/O overhead. Alluxio coupled with Google’s open source data and analytics processing engine, Dataproc, enables zero-copy burst for faster query performance in the cloud so you can take advantage of resources that are not local to your data, without the need for managing the copying or syncing of that data.
We’ll also show a demo on how to get up and running with Alluxio and Dataproc, including how to:
- Setup your hybrid environment between your private datacenter and Google Cloud Platform
- Burst a Spark based machine learning algorithm to Dataproc while accessing on-prem data
- Scale analytic workloads directly on data on-prem without copying and synchronizing the data into the cloud
Save your spot
Thursday, May 28
Speaker: Matthieu Mayran
Google Cloud Solutions Architect
Speaker: Adit Madan
Software Engineer, Alluxio
Adit Madan is a core maintainer and PMC member of the Alluxio Open Source project. His experience is in distributed systems, storage systems, and large scale data analytics. He has an M.S. from Carnegie Mellon University and a B.S. from IIT.
Matthieu currently works as a cloud solutions architect for Google Cloud where he creates reusable assets to help companies of all sizes build their technical architecture in the Cloud. He specializes in big data and machine learning and is currently focusing his time on content for setting up notebook environments on Google Cloud. When not behind his computer, he enjoys a bit of basketball, [kite]surf and building half-broken furniture for his apartment.