Data Scalability Raises Considerable Risk Management Concerns

Posted Leave a commentPosted in Analytics, Best Practices, Big Data, Data Management, Data Quality, data scalability, Risk Management, SmartData Collective Exclusive

Are you frustrated by an increase in the quantity of the data that your organization handles? Well, you aren’t alone! Many businesses globally are dealing with big data which brings along a mix of benefits and challenges. A report by China’s International Data Corporation showed that global data would rise to 175 Zettabyte by 2025. […]

Accurately Building Genomic Cohorts at Scale with Delta Lake and Spark SQL

Posted Leave a commentPosted in Apache Spark, Delta, Delta Lake, Ecosystem, Engineering Blog, Genomics, HLS, Joint Genotyping, SparkSQL

This is the second post in our “Genomic Analysis at Scale”  series.  In our first post, we explored a simple problem: how to provide real-time aggregates when sequencing large volumes of genomes. We solved this problem by using Delta Lake and a streaming pipeline built using Spark SQL. In this blog, we focus on the more advanced […]

Big Data Advances Lead to More Optimal SEO-Predicated Hosting

Posted Leave a commentPosted in Big Data, big data helping hosting companies, Hadoop, SmartData Collective Exclusive

Big data is a very important part of any digital marketing strategy. There are a number of reasons that machine learning, data analytics and Hadoop technology are changing SEO: Machine learning is becoming more widely used in search engine algorithms. SEOs that use machine learning can partially reverse engineer these algorithms. Big data helps SEO […]

Simplifying Streaming Stock Analysis using Delta Lake and Apache Spark: On-Demand Webinar and FAQ Now Available!

Posted Leave a commentPosted in ACID Transactions, Apache Spark, Company Blog, Delta Lake, Education, Engineering Blog, Financial Services, Product, Streaming, Structured Streaming, Time Travel, Unified Batch and Streaming Sync

On June 13th, we hosted a live webinar — Simplifying Streaming Stock Analysis using Delta Lake and Apache Spark — with Junta Nakai, Industry Leader – Financial Services at Databricks, John O’Dwyer, Solution Architect at Databricks, and Denny Lee, Technical Product Marketing Manager at Databricks. This is the first webinar in a series of financial […]

Detecting Bias with SHAP – The Databricks Blog

Posted Leave a commentPosted in Apache Spark, Bias, Deep Learning, Education, Engineering Blog, Machine Learning, MLflow, SHAP, Stack Overflow

StackOverflow’s annual developer survey concluded earlier this year, and they have graciously published the (anonymized) 2019 results for analysis. They’re a rich view into the experience of software developers around the world — what’s their favorite editor? how many years of experience? tabs or spaces? and crucially, salary. Software engineers’ salaries are good, and sometimes […]

Data Scalability Leads To New Evolutions In Smart Technology

Posted Leave a commentPosted in Big Data, data scalability, SmartData Collective Exclusive, technology

Big data is changing the nature of the world we live in. It has created a number of new forms of smart technology, which are simplifying our lives in wonderful ways. The classic example is with smart homes. You can probably remember watching movies about smart houses on the Disney Channel and other networks over […]

Machine Learning Delivers Cutting-Edge POS Software For Online Stores

Posted Leave a commentPosted in Machine Learning, machine learning software, POS, SmartData Collective Exclusive, Software

Online retailers are using machine learning solutions to deliver the highest level of service to their customers. They have found that big data makes it easier to personalize services and offer the highest value for the lowest cost. One of the best ways machine learning is helping online retailers is with new POS software applications. […]

Cutting-Edge Mobile App Development Protocols In The Big Data Era

Posted Leave a commentPosted in app development, app development protocols, Big Data, mobile apps, SmartData Collective Exclusive

Big data is changing the nature of app development in several major ways. Some of the new practices include: Since big data has increased the complexity of app development, you need a clearly outlined strategy. PRDs Meet Requirements for Mobile App Development in the Age of Big Data Do you need help making a mobile […]

Do The Benefits Outweigh The Risks?

Posted Leave a commentPosted in AI, ai for video, ai for video editing, Artificial Intelligence, SmartData Collective Exclusive, video editing

Video is one of the many technologies that is being shaped by new developments in artificial intelligence. Some of the changes artificial intelligence has brought are spooky, to say the least. However, the vast majority of the changes are going to have wonderful implications for video technologists and society as a whole. Overall, we are […]