Migrating Transactional Data to a Delta Lake using AWS DMS

Posted Leave a commentPosted in Company Blog, Delta Lake, Partners

Try this notebook in Databricks Note: We also recommend you read Efficient Upserts into Data Lakes with Databricks Delta which explains the use of MERGE command to do efficient upserts and deletes. Challenges with moving data from databases to data lakes Large enterprises are moving transactional data from scattered data marts in heterogeneous locations to a […]

Announcing Databricks Runtime 5.5 and Runtime 5.5 for Machine Learning

Posted Leave a commentPosted in Announcements, Apache Spark, Machine Learning

Databricks is pleased to announce the release of Databricks Runtime 5.5.  This release includes Apache Spark 2.4.3 along with several important improvements and bug fixes as noted in the latest release notes [Azure|AWS].  We recommend all users upgrade to take advantage of this new runtime release.  This blog post gives a brief overview of some […]

Making the Move to Amsterdam: Bilal Aslam

Posted Leave a commentPosted in Amsterdam, Company Blog, Culture, Databricks Culture, Employee Experience, europe, Moving

While we are proud of our Berkeley roots, Databricks now calls many cities around the world our home. In addition to offices in London, Singapore, New York and our headquarters in San Francisco, we have one of our major engineering hubs in the fast-growing European R&D center, Amsterdam. Databricks offers the exciting opportunity to relocate […]

Technology Is Transforming the Real Estate Market

Posted Leave a commentPosted in IT

Do you remember what it was like trying to buy a house before the days of online listings? If you were born after about 1980, you probably never experienced that. The internet has catapulted the real estate industry into a world where people can buy houses site-unseen and sign the paperwork without ever meeting in […]

How Big Data Can Help With Crop Yield

Posted Leave a commentPosted in Big Data

Agriculture has long been the backbone of a healthy civilization and with the capabilities technology currently offers, it would be irresponsible to not embrace an opportunity to unite both and maximize our efforts towards the improvement of sustainable farming. One way to achieve innovative solutions is through the collection and evaluation of big data. By […]

How Big Data Can Help Forex Traders And Brokers Build A Better Future

Posted Leave a commentPosted in Big Data, brokers, financial tech, Fintech, forex, SmartData Collective Exclusive

It is no longer a secret that Big Data has left a major footprint on the financial market and has changed the way we interact with financial institutions. However, its influence goes much farther. Trading, which is an field that welcomes change and adopts disruptive innovations early, now also benefits from the ramifications of Big […]

Grasping The Cutting Edge Technology Behind Data Recovery Tools

Posted Leave a commentPosted in Big Data, data recovery, data recovery tools, SmartData Collective Exclusive

Data loss is a common problem that many people face. The average data loss in incident at the corporate level costs $5.4 million. These costs might be lower for smaller businesses, but the consequences can be even higher, considering that they have more limited budgets. The good news is that big data technology has made […]

Global Hospitals Embark On A Worldwide Medical Data Initiative

Posted Leave a commentPosted in Big Data, data, Data Management, hospitals and big data, medical data, Policy and Governance, SmartData Collective Exclusive, Transparency

Big data is changing the nature of healthcare. One of the biggest developments was the implementation of the Medical Information Mart for Intensive Care, which took data from 50,000 patients dating back to 2001. Big data will have an even more profound impact in the near future. More than 15,000 hospitals around the world collect […]