Return to page

Welcome to the Community

We look forward to seeing what you make, maker!

Learn


Self-paced Courses

View All

 

Docs Docs


Technical Documentation

View All

 


Blogs

Read All

 


YouTube

Watch All

 

H2O.ai Fights Fire Challenge

Help first responders and the public with new AI applications that can be used to help save lives and property

Learn More

  • Community Meetups

LOADING...
Load More

Slack Community

Discuss, learn and explore with peers and H2O.ai employees the H2O AI Cloud platform, products and services.

Join the Slack Community

 

Already a member? Login

Stack Overflow

Error while using predict function for h2o.deeplearning model on raster stack

I am trying to use `h2o.deeplearning` model to predict on raster data. It returns me the following error > Error: Not compatible with requested type: [type=character; target=double]. Here is a minimal, reproducible, self-contained example ``` library(terra) library(h2o) library(tidyverse) h2o.init() # create a RasterStack or RasterBrick with with a set of predictor layers logo <- rast(system.file("external/rlogo.grd", package="raster")) names(logo) # known presence and absence points p <- matrix(c(48, 48, 48, 53, 50, 46, 54, 70, 84, 85, 74, 84, 95, 85, 66, 42, 26, 4, 19, 17, 7, 14, 26, 29, 39, 45, 51, 56, 46, 38, 31, 22, 34, 60, 70, 73, 63, 46, 43, 28), ncol=2) a <- matrix(c(22, 33, 64, 85, 92, 94, 59, 27, 30, 64, 60, 33, 31, 9, 99, 67, 15, 5, 4, 30, 8, 37, 42, 27, 19, 69, 60, 73, 3, 5, 21, 37, 52, 70, 74, 9, 13, 4, 17, 47), ncol=2) # extract values for points xy <- rbind(cbind(1, p), cbind(0, a)) v <- data.frame(cbind(pa=xy[,1], terra::extract(logo, xy[,2:3]))) %>% mutate(pa = as.factor(pa)) str(v) #### Import data to H2O cluster df <- as.h2o(v) #### Split data into train, validation and test dataset splits <- h2o.splitFrame(df, c(0.70,0.15), seed=1234) train <- h2o.assign(splits[[1]], "train.hex") valid <- h2o.assign(splits[[2]], "valid.hex") test <- h2o.assign(splits[[3]], "test.hex") #### Create response and features data sets y <- "pa" x <- setdiff(names(train), y) ### Deep Learning Model dl_model <- h2o.deeplearning(training_frame=train, validation_frame=valid, x=x, y=y, standardize=TRUE, seed=125) dnn_pred <- function(model, data, ...) { predict(model, newdata=as.h2o(data), ...) } p <- predict(logo, model=dl_model, fun=dnn_pred) plot(p) ```

java.lang.AssertionError when trying to train certain datasets with h2o

I am getting an error for my desired dataset when trying to use an isolation forest method to detect anomalies. However I have another completely different dataset that it works fine for, what could cause this issue? isolationforest Model Build progress: | (failed) | 0% Traceback (most recent call last): File "h2o_test.py", line 149, in <module> isoforest.train(x=iso_forest.col_names[0:65], training_frame=iso_forest) File "/home/ec2-user/.local/lib/python3.7/site- packages/h2o/estimators/estimator_base.py", line 107, in train self._train(parms, verbose=verbose) File "/home/ec2-user/.local/lib/python3.7/site- packages/h2o/estimators/estimator_base.py", line 199, in _train job.poll(poll_updates=self._print_model_scoring_history if verbose else None) File "/home/ec2-user/.local/lib/python3.7/site-packages/h2o/job.py", line 89, in poll "\n{}".format(self.job_key, self.exception, self.job["stacktrace"])) OSError: Job with key $03017f00000132d4ffffffff$_92ee3e892f7bc86460e80153eaec4b70 failed with an exception: java.lang.AssertionError stacktrace: java.lang.AssertionError at hex.tree.DHistogram.init(DHistogram.java:350) at hex.tree.DHistogram.init(DHistogram.java:343) at hex.tree.ScoreBuildHistogram2$ComputeHistoThread.computeChunk(ScoreBuildHistogram2.java:427) at hex.tree.ScoreBuildHistogram2$ComputeHistoThread.map(ScoreBuildHistogram2.java:408) at water.LocalMR.compute2(LocalMR.java:89) at water.LocalMR.compute2(LocalMR.java:81) at water.H2O$H2OCountedCompleter.compute(H2O.java:1704) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.popAndExecAll(ForkJoinPool.java:906) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:979) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) ``` with open('/home/webapp/flask-api/tmp_rows/temp_file2.csv', 'w+') as tmp_file: temp_name = "/tmp_rows/temp_file2.csv" tmp_file.write(text_stream.getvalue()) tmp_file.close() h2o.init() print("TEMP_nAME", temp_name) iso_forest = h2o.import_file('/home/webapp/flask-api/{0}'.format(temp_name)) seed = 12345 ntrees = 100 isoforest = h2o.estimators.H2OIsolationForestEstimator( ntrees=ntrees, seed=seed) isoforest.train(x=iso_forest.col_names[0:65], training_frame=iso_forest) predictions = isoforest.predict(iso_forest) print(predictions) h2o.cluster().shutdown() ``` The CSV is being created fine, so there doesn't seem to be an issue with that, what is causing this Java error? I even increased the size of my ec2 to have more RAM, that didn't solve it either.

RestApiCommunicationException: H2O node http://10.159.20.11:54321 responded with

everyone! I have a list of discounts, and for each discount, I iterate through them. Within each iteration, I generate a few new columns and then predict the sales amount for each product associated with that discount. I aim to compile all the results from these iterations into a single dataframe to have a comprehensive view of every discount on the list along with the predictions. This process has already been successfully implemented in native H2O using Pandas. However, I am now attempting to replicate it in Spark H2O and PySpark. Unfortunately, when attempting to read the dataframe, I encounter the following error: ``` RestApiCommunicationException: H2O node http://10.159.20.11:54321 responded with org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 256.0 failed 4 times, most recent failure: Lost task 0.3 in stage 256.0 (TID 3451) (10.159.20.11 executor 0): ai.h2o.sparkling.backend.exceptions.RestApiCommunicationException: H2O node http://10.159.20.11:54321 responded with Status code: 400 : Bad Request ``` I am seeking a workaround or advice on whether I might be making a mistake. Expected Behavior: I should be able to access and manipulate the dataframe without issues. Observed Behavior: The error "RestApiCommunicationException: H2O node http://10.159.20.11:54321/ responded with" prevents further progress. What I am doing now is saving the predictions to a delta table and then access it, but that takes a little more time than what I expected. I know that loops are not very recommended for PySpark but I am also not seeing a better way of doing this. Thank you so much!

How to prevent h2o cluster shutdown without notice using Python

My code loads the h2o MOJO model to get prediction on a small dataset. However, h2o shutdown abruptly by itself. The same code is working fine on one machine with the same set of inputs but seeing abnormal h2o shutdowns on other machine. ``` self.test = h2o.import_file(dataset_file) preds = imported_model.predict(self.test) ``` I am running this on 1TB machine with 72 cores. I can't believe this is memory issues. The most puzzling the fact is that the same code is working on other machine with the same inputs (configured differently). I don't know the full list of differences. I was previously running with python frozen binary build and couldn't see error messages in detail. I am running python code directly and can see error messages in more detail. ``` File "h2o_model_eval.py", line 160, in ModelEval preds = imported_model.predict(self.test) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/h2o/model/model_base.py", line 334, in predict j.poll() File ".venv/lib/python3.11/site-packages/h2o/job.py", line 71, in poll pb.execute(self._refresh_job_status) File ".venv/lib/python3.11/site-packages/h2o/utils/progressbar.py", line 187, in execute res = progress_fn() # may raise StopIteration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/h2o/job.py", line 136, in _refresh_job_status jobs = self._query_job_status_safe() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/h2o/job.py", line 132, in _query_job_status_safe raise last_err File ".venv/lib/python3.11/site-packages/h2o/job.py", line 114, in _query_job_status_safe result = h2o.api("GET /3/Jobs/%s" % self.job_key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/h2o/h2o.py", line 123, in api return h2oconn.request(endpoint, data=data, json=json, filename=filename, save_to=save_to) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/h2o/backend/connection.py", line 507, in request raise H2OConnectionError("Unexpected HTTP error: %s" % e) h2o.exceptions.H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=54321): Max retries exceeded with url: /3/Jobs/$03017f00000132d4ffffffff$_acab67512114e05db6ec9865ea9849d3 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x2aab3f59ea90>: Failed to establish a new connection: [Errno 111] Connection refused')) ~ ``` How to debug this issue?

Product Resources

Get started with our products

Datatable
 

View on Github
 

H2O-3
 

View on Github
 

H2O AI Feature Store
 

Learn More

H2O Document AI
 

View on Github
Learn More

H2O Driverless AI
 

View on Github
Learn More

H2O Hydrogen Torch
 

Learn More
Product Brief

H2O MLOps
 

Learn More
Product Brief

H2O Sparkling Water
 

View on Github
Learn More

Try the H2O AI Cloud for free for 90 days

Get Started
 

Become part of our community by trying H2O.ai with a free 90-day trial