What’s new with MLflow? On-Demand Webinar and FAQs now available!

Posted Leave a commentPosted in Data Science, Engineering Blog, Machine Learning, Managed MLflow, MLflow, Model Management, Open Source

On June 6th, our team hosted a live webinar—Managing the Complete Machine Learning Lifecycle: What’s new with MLflow—with Clemens Mewald, Director of Product Management at Databricks. Machine learning development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to […]

Detecting Bias with SHAP – The Databricks Blog

Posted Leave a commentPosted in Apache Spark, Bias, Deep Learning, Education, Engineering Blog, Machine Learning, MLflow, SHAP, Stack Overflow

StackOverflow’s annual developer survey concluded earlier this year, and they have graciously published the (anonymized) 2019 results for analysis. They’re a rich view into the experience of software developers around the world — what’s their favorite editor? how many years of experience? tabs or spaces? and crucially, salary. Software engineers’ salaries are good, and sometimes […]

Hyperparameter Tuning with MLflow, Apache Spark MLlib and Hyperopt

Posted Leave a commentPosted in Apache Spark, AutoML, Data Science, Databricks Runtime 5.4 ML, Deep Learning, Ecosystem, Engineering Blog, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, MLlib

Hyperparameter tuning is a common technique to optimize machine learning models based on hyperparameters, or configurations that are not learned during model training.  Tuning these configurations can dramatically improve model performance. However, hyperparameter tuning can be computationally expensive, slow, and unintuitive even for experts. Databricks Runtime 5.4 and 5.4 ML (Azure | AWS) introduce new […]

Announcing the MLflow 1.0 Release

Posted Leave a commentPosted in Announcements, Company Blog, Data Science, Ecosystem, Engineering Blog, Lifecycle, Machine Learning, MLflow, Model Management, Product

MLflow is an open source platform to help manage the complete machine learning lifecycle. With MLflow, data scientists can track and share experiments locally (on a laptop) or remotely (in the cloud), package and share models across frameworks, and deploy models virtually anywhere. Today we are excited to announce the release of MLflow 1.0. Since […]

Enhanced Hyperparameter Tuning and Optimized AWS Storage with Databricks Runtime 5.4 ML

Posted Leave a commentPosted in Announcements, AutoML, Company Blog, Data Science, Databricks Runtime 5.4 ML, Deep Learning, Ecosystem, Engineering Blog, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, MLlib, Platform, Product

We are excited to announce the release of Databricks Runtime 5.4 ML (Azure | AWS). This release includes two Public Preview features to improve data science productivity, optimized storage in AWS for developing distributed applications, and a number of Python library upgrades. To get started, you simply select the Databricks Runtime 5.4 ML from the […]

Spark + AI Summit 2019 Product Announcements and Recap. Watch the keynote recordings today!

Posted Leave a commentPosted in Announcements, Apache Spark, Company Blog, Delta Lake, Events, Koalas, MLflow, Product, Spark + AI Summit

Spark + AI Summit 2019, the world’s largest data and machine learning conference for the Apache Spark™ Community, brought nearly 5000 data scientists, engineers, and business leaders to San Francisco’s Moscone Center to find out what’s coming next. Watch the keynote recordings today and learn more about the latest product announcements for Apache Spark, MLflow, […]

Using Dynamic Time Warping and MLflow to Detect Sales Trends

Posted Leave a commentPosted in Apache Spark, Company Blog, Dynamic Time Warping, Education, Engineering Blog, Machine Learning, MLflow, Platform

Try this notebook series in Databricks This blog is part 2 of our two-part series Using Dynamic Time Warping and MLflow to Detect Sales Trends.  The phrase “dynamic time warping,” at first read, might evoke images of Marty McFly driving his DeLorean at 88 MPH in the Back to the Future series. Alas, dynamic time warping does […]

Introducing MLflow Run Sidebar in Databricks Notebooks

Posted Leave a commentPosted in Announcements, Company Blog, Engineering Blog, Machine Learning, Managed MLflow, MLflow, Platform, Sidebar

At Spark+AI Summit 2019, we announced the GA of Managed MLflow on Databricks in which we take the latest and greatest of open source MLflow and make it easily accessible to all users of Databricks. In that blog post, we promised to build features which bridge Databricks and MLflow concepts to create a seamless integration […]

Announcing General Availability of Managed MLflow on Databricks

Posted Leave a commentPosted in Announcements, Company Blog, Ecosystem, Engineering Blog, Machine Learning, Managed MLflow, MLflow, Platform, Product

Try this tutorial in Databricks MLflow is an open source platform to help manage the complete machine learning lifecycle. With MLflow, data scientists can track and share experiments locally or in the cloud, package and share models across frameworks, and deploy models virtually anywhere. Today at the Spark + AI Summit, we announced the General […]

A Guide to MLflow Talks at Spark + AI Summit 2019

Posted Leave a commentPosted in Company Blog, Events, Machine Learning, MLflow, Open Source, Product, Spark + AI Summit

In less than a year, MLflow has reached almost 500K monthly downloads, and gathered over 80 code contributors and 40 contributing organizations, confirming the need for an open source approach to help standardize the machine learning lifecycle across tools, teams, and processes. We are thrilled to host some of our key contributors and customers next […]