Introducing Databricks Runtime 5.1 for Machine Learning

Posted Leave a commentPosted in Announcements, Apache Spark, Company Blog, Databricks Runtime 5.1 ML, Deep Learning, Engineering Blog, Machine Learning, PyTorch, TensorFlow

Last week, we released Databricks Runtime 5.1 Beta for Machine Learning. As part of our commitment to provide developers the latest deep learning frameworks, this release includes the best of these libraries. In particular, our PyTorch addition makes it simple for a developer to simply import the appropriate Python torch modules and start coding, without […]

Introducing HorovodRunner for Distributed Deep Learning Training

Posted Leave a commentPosted in Apache Spark, Deep Learning, Distributed Learning, Engineering Blog, Keras, Project Hydrogen, TensorFlow

Today, we are excited to introduce HorovodRunner in our Databricks Runtime 5.0 ML! HorovodRunner provides a simple way to scale up your deep learning training workloads from a single machine to large clusters, reducing overall training time. Motivated by the needs of many of our users who want to train deep learning models on datasets […]

Applying your Convolutional Neural Network: On-Demand Webinar and FAQ Now Available!

Posted Leave a commentPosted in Deep Learning, Ecosystem, Engineering Blog, Keras, Machine Learning, Neural Networks, Platform, TensorFlow

Try this notebook in Databricks On October 25th, we hosted a live webinar—Applying your Convolutional Neural Network—with Denny Lee, Technical Product Marketing Manager at Databricks. This is the third webinar of a free deep learning fundamental series from Databricks. In this webinar, we dived deeper into Convolutional Neural Networks (CNNs), a particular type of neural […]

Training your Neural Network: On-Demand Webinar and FAQ Now Available!

Posted Leave a commentPosted in Deep Learning, Ecosystem, Engineering Blog, Keras, Machine Learning, Neural Networks, Platform, TensorFlow

Try this notebook in Databricks On October 9th, we hosted a live webinar—Training your Neural Network—on Data Science Central with Denny Lee, Technical Product Marketing Manager at Databricks. This is the second webinar of a free deep learning fundamental series from Databricks. In this webinar, we covered the principles for training your neural network including […]

Introduction to Neural Networks: On-Demand Webinar and FAQ Now Available!

Posted Leave a commentPosted in Deep Learning, Ecosystem, Keras, Machine Learning, Neural Networks, Platform, TensorFlow

Try this notebook in Databricks On September 27th, we hosted a live webinar—Introduction to Neural Networks—with Denny Lee, Technical Product Marketing Manager at Databricks. This is the first webinar of a free deep learning fundamental series from Databricks. In this webinar, we covered the fundamentals of deep learning to better understand what gives neural networks […]

How to Use MLflow To Reproduce Results and Retrain Saved Keras ML Models

Posted Leave a commentPosted in Apache Spark, Engineering Blog, Keras, Machine Learning, MLflow, Model Management, Platform, TensorFlow, Unified Analytics Platform

In part 2 of our series on MLflow blogs, we demonstrated how to use MLflow to track experiment results for a Keras network model using binary classification. We classified reviews from an IMDB dataset as positive or negative. And we created one baseline model and two experiments. For each model, we tracked its respective training […]

Identify Suspicious Behavior in Video with Databricks Runtime for Machine Learning

Posted Leave a commentPosted in Apache Spark, Company Blog, Deep Learning, Deep Learning Pipelines, Education, Engineering Blog, Machine Learning, OpenCV, Platform, Product, TensorFlow, Video Analytics

Try this notebook series in Databricks With the exponential growth of cameras and visual recordings, it is becoming increasingly important to operationalize and automate the process of video identification and categorization. Applications ranging from identifying the correct cat video to visually categorizing objects are becoming more prevalent.  With millions of users around the world generating […]

Introducing mlflow-apps: A Repository of Sample Applications for MLflow

Posted Leave a commentPosted in Apache Spark, Data Science, Engineering Blog, Machine Learning, MLflow, Platform, python, TensorFlow, Unified Analytics Platform

Introduction This summer, I was a software engineering intern at Databricks on the Machine Learning (ML) Platform team. As part of my intern project, I built a set of MLflow apps that demonstrate MLflow’s capabilities and offer the community examples to learn from. In this blog, I’ll discuss this library of pluggable ML applications, all […]

Bay Area Apache Spark Meetup Summary @ Databricks HQ

Posted Leave a commentPosted in Apache Spark, Company Blog, Deep Learning, Events, Machine Learning, MLflow, Model Management, python, TensorFlow

On July 19, we held our monthly Bay Area Spark Meetup (BASM) at Databricks, HQ in San Francisco. At the Spark + AI Summit in June, we announced two open-source projects: Project Hydrogen and MLflow. Partly to continue sharing the progress of these open-source projects with the community and partly to encourage community contributions, two […]

A Guide to AI, Machine Learning, and Deep Learning Talks at Spark + AI Summit Europe

Posted Leave a commentPosted in Apache Spark, Company Blog, Deep Learning Pipelines, Events, Keras, PyTorch, Spark + AI Summit, TensorFlow

Within a couple of years of its release as an open-source machine learning and deep learning framework, TensorFlow has seen an amazing rate of adoption. Consider the number of stars on its github page: over 105K; look at the number of contributors: 1500+; and observe its growing penetration and pervasiveness in verticals: from medical imaging […]