How Big Data Can be Used for Social Good

Posted Leave a commentPosted in Big Data

In almost every industry these days, big data is one of the most critical components of progress and growth. This is true of big business, like Amazon, but it is also true in the world of an industry that works towards social good. Today we will examine a few social service applications that prove big […]

Processing Petabytes of Data in Seconds with Databricks Delta

Posted Leave a commentPosted in Apache Spark, Databricks Delta, Engineering Blog, Machine Leanring, Spark SQL, Streaming, Structured Streaming, Unified Analytics Platform

Introduction Databricks Delta is a unified data management system that brings data reliability and fast analytics to cloud data lakes. In this blog post, we take a peek under the hood to examine what makes Databricks Delta capable of sifting through petabytes of data within seconds. In particular, we discuss Data Skipping and ZORDER Clustering. […]

How Big Data And Architecture Are Teaming Up For Success

Posted Leave a commentPosted in Architecture, Big Data, virtual reality

Most people know about how big data and virtual reality (VR) have collectively changed sectors such as marketing, but they may not realize that these two technologies have made significant impacts on other industries, too. Big data and architecture is one combination worth exploring in depth. The Two Technologies Can Complement Each Other Some forward-thinking […]

7 Ways Airlines Use Artificial Intelligence and Data Science to Improve Operations

Posted Leave a commentPosted in Artificial Intelligence

Technology changes the way businesses interact with their customers, make business decisions, and build workflows. For instance, such actions as booking a flight via phone or conducting solely offline surveys seem unusual these days. Real-time access to data — the 21st-century oil — allows organizations to take informed steps towards operational efficiency. We discussed with […]

How Are Predictive Analytics Shaping the Future of Fintech?

Posted Leave a commentPosted in Analytics, Big Data, finance, financial technology, Fintech, Predictive Analytics

The financial industry has undergone tremendous changes over the past decade. A lot of this can be attributed to big data. As advances in data become more impressive, new fintech solutions are reaching the market. Predictive analytics is one of the most important big data trends affecting FinTech. Established financial companies like Payoneer and PayPal […]

rquery: Practical Big Data Transforms for R-Spark Users

Posted Leave a commentPosted in Apache Spark, Big Data, Data Science, Engineering Blog, Machine Learning, R, Spark SQL, SparkR

This is a guest community blog from Nina Zumel and John Mount, data scientists and consultants at Win-Vector. They share how to use rquery with Apache Spark on Databricks Try this notebook in Databricks Introduction In this blog, we will introduce rquery, a powerful query tool that allows R users to implement powerful data transformations […]

Why We Should Ban Lethal Autonomous Weapons

Posted Leave a commentPosted in Artificial Intelligence

A few days ago, some of the world’s most renowned entrepreneurs, engineers and scientists in the technology industry warned for the dangers of artificial intelligence and that of lethal autonomous weapons. Over 150 companies and 2400 individuals from 90 countries signed the lethal autonomous weapon pledge at the 2018 International Joint Conference on Artificial Intelligence. […]