Databricks delivers security and scalability enhancements

Databricks has introduced new features within its platform that provide deeper security controls, proactive administration and automation across the data and ML lifecycle.

Friday, 20th March 2020 Posted 4 years ago in by Phil Alsop

As data teams enable analytics and machine learning (ML) applications across their organisations, they require the ability to securely leverage data at massive scale. Doing this can be complex and risky, especially when operating in a multi-cloud environment. Security is fragmented, which makes corporate access policies difficult to extend, administration is reactive and inefficient, and devops processes like user management or cluster provisioning are manual and time consuming. Databricks’ Unified Data Analytics Platform addresses these challenges by helping organisations bring all their users and data together in a simple, scalable and secure service that can leverage the native capabilities of multiple clouds.

“The biggest challenge for organisations today is selecting an enterprise platform that can handle all of your data and all of the people that interact with it - today and in the future,” said David Meyer, senior vice president of Product Management at Databricks. “Databricks is the only platform that has successfully achieved the massive scale and simplicity that enables enterprises to make data, business analytics and machine learning pervasive enterprise-wide. We’re committed to preserving this for our customers, regardless of if and how their cloud strategies evolve over time. These new features are a great example of how we’re doing that.”

New features within the Databricks platform further enhance:

Cloud-native Security - Enterprises can already leverage a fully-managed SaaS service without losing control over their data by running Databricks clusters inside their cloud account. The addition of customer-owned revocable data encryption keys and customised private networks to run these clusters, allows customers to further tailor the service to their unique enterprise and compliance requirements.
Simple and Proactive Administration - To support hundreds of teams with thousands of users that create hundreds of thousands of compute instances, visibility and control are critical. For full transparency organisations can now audit and analyse all the activity in their account, and set policies to administer users, control budget and manage infrastructure.
Automation at Scale - With an API-driven approach, Databricks now enables customers to productionise analytics and ML rapidly with CI/CD (Continuous integration and continuous delivery). With the addition of git support, APIs for everything from user management, workspace provisioning, cluster policies to application and infrastructure monitoring, DevOps teams can automate the whole data and ML lifecycle.

“With simplified administration and governance, Databricks’ Unified Data Analytics Platform has allowed us to bring data-based decision making to teams across our organisation. The ease of adding users, native security integrations with cloud providers and APIs-for-everything have enabled us to bring the data and tools they need, to every employee in Wehkamp,” said Tom Mulder, Lead Data Scientist at Wehkamp. “By enabling us to marry our corporate identity and access policies with the cloud-native security services offered by our cloud providers, Databricks enables our data teams to stay secure and do their best work, regardless of how our cloud strategy evolves.”