Detecting Financial Fraud at Scale with Decision Trees and MLflow on  Databricks

Posted Leave a commentPosted in Apache Spark, Company Blog, Decision tree, Education, Engineering Blog, financial, Financial Markets, Financial Services, Fraud, Fraud Detection, Machine Leanring, Machine Learning, Platform

Try this notebook in Databricks Detecting fraudulent patterns at scale is a challenge, no matter the use case. The massive amounts of data to sift through, the complexity of the constantly evolving techniques, and the very small number of actual examples of fraudulent behavior are comparable to finding a needle in a haystack while not […]

Processing Petabytes of Data in Seconds with Databricks Delta

Posted Leave a commentPosted in Apache Spark, Databricks Delta, Engineering Blog, Machine Leanring, Spark SQL, Streaming, Structured Streaming, Unified Analytics Platform

Introduction Databricks Delta is a unified data management system that brings data reliability and fast analytics to cloud data lakes. In this blog post, we take a peek under the hood to examine what makes Databricks Delta capable of sifting through petabytes of data within seconds. In particular, we discuss Data Skipping and ZORDER Clustering. […]