Kicking Off 2019 with an MLflow User Survey

Posted Leave a commentPosted in Apache Spark, Ecosystem, Engineering Blog, Machine Learning, MLflow

It’s been six months since we launched MLflow, an open source platform to manage the machine learning (ML) lifecycle, and the project has been moving quickly since then. MLflow fills a role that hasn’t been served well in the open source community so far: managing the development lifecycle for ML, including tracking experiments and metrics, […]

Introducing Databricks Runtime 5.1 for Machine Learning

Posted Leave a commentPosted in Announcements, Apache Spark, Company Blog, Databricks Runtime 5.1 ML, Deep Learning, Engineering Blog, Machine Learning, PyTorch, TensorFlow

Last week, we released Databricks Runtime 5.1 Beta for Machine Learning. As part of our commitment to provide developers the latest deep learning frameworks, this release includes the best of these libraries. In particular, our PyTorch addition makes it simple for a developer to simply import the appropriate Python torch modules and start coding, without […]

MLflow v0.8.1 Features Faster Experiment UI and Enhanced Python Model

Posted Leave a commentPosted in Apache Spark, Data Science, Ecosystem, Engineering Blog, Machine Learning, Machine Learning Life Cycle, MLflow, Model Management, Platform, Spark UDF

Try this notebook in Databricks MLflow v0.8.1 was released this week. It introduces several UI enhancements, including faster load times for thousands of runs and improved responsiveness when navigating runs with many metrics and parameters. Additionally, it expands support for evaluating Python models as Apache Spark UDFs and automatically captures model dependencies as Conda environments. […]

A Guide to AI, Machine Learning, and Deep Learning Talks at Spark + AI 2019

Posted Leave a commentPosted in AI, Apache Spark, Company Blog, Deep Learning, Events, Machine Learning, Spark + AI Summit

To a good degree, this back-of-the-envelope flowchart, by Karen Hao of MIT Technology Review, charts to elucidate what constitutes the use of AI in the grand scheme of things. source: While many conferences may not have a flowchart to select sessions to identify what’s AI, the sessions, though, do speak to technical aspects that […]

How AI Is Transforming Lending And Loan Management

Posted Leave a commentPosted in AI, Artificial Intelligence, Big Data, finance, Fintech, lending, loan management, Machine Learning, SmartData Collective Exclusive

The old days of banking may be long gone in just a few years and this, thanks to all the recent technological developments that we are currently witnessing. You are no longer willing to enter a bank, discuss all the things that you need to discuss with a banker and hope that you get your […]

How Will Industrial Internet Of Things Shape CNC Machining?

Posted Leave a commentPosted in cnc machining, industrial internet of things, Internet of Things, IoT, Machine Learning, SmartData Collective Exclusive

Machinists everywhere are already enjoying some of the benefits of the Industrial Internet of Things for CNC equipment and other tools. Like any other major breakthrough, adoption might be slow for the moment, but the excitement is anything but underwhelming. Below is a rundown of some of the biggest ways we expect the IIoT to […]

Why Tech Pros Are Turning To Machine Learning For Career Growth

Posted Leave a commentPosted in Jobs, Machine Learning, SmartData Collective Exclusive, tech business, tech field

While the topic of machine learning has been floating around for quite a while, modern technology has transformed it from a couple of buzzwords that most people didn’t understand into the force that’s behind many of the modern technological comforts that we now take for granted. As an industry, machine learning is one that’s experiencing […]

Introducing Built-in Image Data Source in Apache Spark 2.4

Posted Leave a commentPosted in Apache Spark, Data Source, Databricks Runtime, DataFrames, Deep Learning Pipelines, Ecosystem, Engineering Blog, Machine Learning

Introduction With recent advances in deep learning frameworks for image classification and object detection, the demand for standard image processing in Apache Spark has never been greater. Image handling and preprocessing have their specific challenges – for example, images come in different formats (eg., jpeg, png, etc.), sizes, and color schemes, and there is no […]

Machine Learning Is Moving Corporate VPN Security Into The 21st Century

Posted Leave a commentPosted in Business Intelligence, corporate data security, corporate vpn security, cybersecurity, data security, Machine Learning, News, Security, VPN, vpn security

Machine learning has played a crucial role in digital security. According to experts on digital security, machine learning is playing a vital role in the creation of new VPN solutions and corporate VPN security. The Journal of Cyber Security Technology published a study in 2017 titled “Comparison of machine-learning algorithms for classification of VPN network […]

Discover The Power of Big Data And Learning Analytics For Education

Posted Leave a commentPosted in Analytics, Big Data, leaning analytics, Machine Learning, Predictive Analytics, SmartData Collective Exclusive

Data-driven decision making is all about making decisions that are backed up by solid, verifiable data and not observations. Strategic decision making, which was popularized in the 1980s and 1990s, has transformed into a sophisticated concept. It bears the name of big data. With the help of advanced analytic techniques, it is possible to take […]