Mastering Classification & Regression with LLMs: Insights from Kaggle Competitions | Philipp Singer

Overview

In this session, H2O.ai's Philipp Singer explores how large language models can be applied to classical classification and regression problems, not just generative use cases. Using a public financial sentiment dataset from Kaggle, he compares a majority-class baseline, zero-shot LLM prompting, a next-token-probability trick for faster inference, and full fine-tuning with H2O LLM Studio. He also shares results from Kaggle competitions, including a first-place finish in the LLM Science Exam competition, where LLM-based classification outperformed traditional methods.