AutoML on Databricks: Augmenting Data Science from Data Prep to Operationalization

Posted Leave a commentPosted in Announcements, AutoML, Company Blog, Data Science, Data Science and Machine Learning, Databricks Labs, Engineering Blog, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, Model Search, Product

Thousands of data science jobs are going unfilled today as global demand for the talent greatly outstrips supply. Every day, businesses pay the price of the data scientist shortage in missed opportunities and slow innovation. For organizations to realize the full potential of machine learning, data teams have to build hundreds of predictive models a […]

Brickster Spotlight: Meet Greg From Intern to Senior Software Engineer

Posted Leave a commentPosted in Company Blog, Culture, Data Science, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, MLlib

At Databricks, we’re committed to learning and development at every level, so it’s important to our teams that we recruit and develop our next generation of Databricks leaders. Our interns are encouraged to live out one of our core values, “be an owner” and they play an integral role in developing our platform during their […]

Automated Hyperparameter Tuning, Scaling and Tracking: On-Demand Webinar and FAQs now available!

Posted Leave a commentPosted in Data Science, Ecosystem, Engineering Blog, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, MLlib

Try this notebook in Databricks On June 20th, our team hosted a live webinar—Automated Hyperparameter Tuning, Scaling and Tracking on Databricks—with Joseph Bradley,  Software Engineer, and Yifan Cao, Senior Product Manager at Databricks. Automated Machine Learning (AutoML) has received significant interest recently because of its ability to shorten time-to-value for data science teams and maximize […]

Hyperparameter Tuning with MLflow, Apache Spark MLlib and Hyperopt

Posted Leave a commentPosted in Apache Spark, AutoML, Data Science, Databricks Runtime 5.4 ML, Deep Learning, Ecosystem, Engineering Blog, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, MLlib

Hyperparameter tuning is a common technique to optimize machine learning models based on hyperparameters, or configurations that are not learned during model training.  Tuning these configurations can dramatically improve model performance. However, hyperparameter tuning can be computationally expensive, slow, and unintuitive even for experts. Databricks Runtime 5.4 and 5.4 ML (Azure | AWS) introduce new […]

Enhanced Hyperparameter Tuning and Optimized AWS Storage with Databricks Runtime 5.4 ML

Posted Leave a commentPosted in Announcements, AutoML, Company Blog, Data Science, Databricks Runtime 5.4 ML, Deep Learning, Ecosystem, Engineering Blog, Hyperopt, Hyperparameter Tuning, Machine Learning, MLflow, MLlib, Platform, Product

We are excited to announce the release of Databricks Runtime 5.4 ML (Azure | AWS). This release includes two Public Preview features to improve data science productivity, optimized storage in AWS for developing distributed applications, and a number of Python library upgrades. To get started, you simply select the Databricks Runtime 5.4 ML from the […]