High-Performance Modern Data Warehousing with Azure Databricks and Azure SQL DW

Posted Leave a commentPosted in Announcements, Azure, Azure Databricks, Modern Data Warehouse, Partners, SQL DW

As organizations move to the cloud, the architecture for a Modern Data Warehouse (MDW) allows a new level of performance and scalability.  A modern data warehouse enables bringing together data at any scale easily, and to get insights through analytical dashboards, operational reports, or advanced analytics.  But what does an MDW look like? The following […]

Announcing Databricks Runtime 5.2 – The Databricks Blog

Posted Leave a commentPosted in Announcements, Engineering Blog, Product

We are excited to announce the release of Databricks Runtime 5.2 which introduces several new features including the following: Delta Time Travel Fast Parquet Import Databricks Advisor Let’s unpack each of these features in more detail: Delta Time Travel Time Travel, released as an Experimental feature, adds the ability to query a snapshot of a table […]

Introducing Delta Time Travel for Large Scale Data Lakes

Posted Leave a commentPosted in Announcements, Company Blog, Databricks Delta, Delta

Data versioning for reproducing experiments, rolling back, and auditing data We are thrilled to introduce time travel capabilities in Databricks Delta, the next-gen unified analytics engine built on top of Apache Spark, for all of our users. With this new feature, Delta automatically versions the big data that you store in your data lake, and […]

Introducing Databricks Library Utilities for Notebooks

Posted Leave a commentPosted in Announcements, Engineering Blog

Databricks has introduced a new feature, Library Utilities for Notebooks, as part of Runtime version 5.1. It allows you to install and manage Python dependencies from within a notebook.   This provides several important benefits: Install libraries when and where they’re needed, from within a notebook.  This eliminates the need to globally install libraries on a […]

Announcing Databricks Runtime 5.1 – The Databricks Blog

Posted Leave a commentPosted in Announcements, Databricks Runtime 5.1

We are excited to announce the release of Databricks Runtime 5.1. This release includes several new features, performance and quality improvements. We highly recommend upgrading to Databricks Runtime 5.1 to benefit from all of these. Release highlights: Azure Active Directory Credential Passthrough to ADLS Gen1 (Preview) 7% performance gain overall for Databricks Azure New in […]

Introducing Databricks Runtime 5.1 for Machine Learning

Posted Leave a commentPosted in Announcements, Apache Spark, Company Blog, Databricks Runtime 5.1 ML, Deep Learning, Engineering Blog, Machine Learning, PyTorch, TensorFlow

Last week, we released Databricks Runtime 5.1 Beta for Machine Learning. As part of our commitment to provide developers the latest deep learning frameworks, this release includes the best of these libraries. In particular, our PyTorch addition makes it simple for a developer to simply import the appropriate Python torch modules and start coding, without […]

CIO Survey: Top 3 Challenges Adopting AI and How to Overcome Them

Posted Leave a commentPosted in AI, Announcements, CIO, Company Blog, data, Events, ML, Product, Survey, Unified analytics

  We recently hosted the webinar — CIO Survey: Enterprise Challenges to AI and How to Overcome Them — featuring Jen Garofalo, Research Director at IDG, the parent company to CIO.com, and Pat McDonough, VP of Customer Success at Databricks. This webinar covered key findings from a recent CIO.com survey of 200 executives on the […]

Introducing Databricks Runtime 5.0 for Machine Learning

Posted Leave a commentPosted in Announcements, Company Blog, Databricks Runtime 5.0 ML, Deep Learning, Ecosystem, Engineering Blog, Machine Learning, Platform

Six months ago we introduced the Databricks Runtime for Machine Learning with the goal of making machine learning performant and easy on the Databricks Unified Analytics Platform. The Databricks Runtime for ML comes pre-packaged with many ML frameworks and enables distributed training and inference. Today we are excited to release the second iteration including Conda […]