March 25th, 2014

In-memory Big Data: Spark + H2O

Category: Uncategorized
H2O_Spark

Big Data has moved in-memory. Customers using SQL in their Join & Munging efforts via SHARK and Apache Spark need to use Regressions and Deep Learning. To make their experiences great & seamlessly weave SQL workflows with Data Science and Machine Learning, we are architecting a simple RDD data import-export in H2O. This brings continuity to their in-memory interactive experience. And support for Spark MLI using our native Scala API – Shalala.
Big Data users can now use SHARK to extract and fuse datasets and H2O for better predictions.

H2O_Spark-800x559
Here’s a typical pipeline in this (using the unix pipe analogy)

 hdfs | Spark | SHARK/SQL | RDD | h2o.readRDD() | h2o.deepLearning() | h2o.predict() | h2o.persist(RDD or HDFS)

Calling h2o.deepLearning() from within Scala interface alongside Spark (via Shalala) will make the workflow even more seamless for end users.

Leave a Reply

H2O New Year releases

There were two releases shortly after each other. First, on December 21st, there was a

January 18, 2019 - by Jo-fai Chow
Fallback Featured Image
What is Your AI Thinking? Part 1

Explaining AI to the Business Person Explainable AI is in the news, and for good reason.

January 17, 2019 - by Saurabh Kumar
Finally, You Can Plot H2O Decision Trees in R

Creating and plotting decision trees (like one below) for the models created in H2O will

January 15, 2019 - by Jo-fai Chow
What Business Leaders Need to Know About AI

The interest around artificial intelligence (AI) is at an all-time fevered pitch right now, and

January 11, 2019 - by Saurabh Kumar
Celebrating our community and wins!

The last year was an amazing year at H2O.ai. We organized two H2O World’s, gathering

January 11, 2019 - by Bruna Smith
Finding Clarity in the Automated Modeling Space

There is an arms race happening in Data Science and Machine Learning space. It's the

December 12, 2018 - by Jo-Fai Chow

Join the AI Revolution

Subscribe, read the documentation, download or contact us.

Subscribe to the Newsletter

Start Your 21-Day Free Trial Today

Get It Now
Desktop img