Sharing data securely across teams, regions, or even organizations has always been a challenge. With Databricks Delta Sharing, this process becomes seamless, scalable, and highly secure.
In this guide, we’ll break down:
- What Delta Sharing is
- How Databricks-to-Databricks sharing works
- Step-by-step implementation
- Security and permission best practices
If you’re working with Power BI and Databricks, mastering this concept can significantly improve your data architecture and collaboration workflows.
What is Delta Sharing?
Delta Sharing is an open-source protocol developed by Databricks that enables secure data sharing across platforms.
Key benefits include:
- Share live data without copying it
- Cross-platform compatibility (Azure, AWS, GCP)
- No need for recipients to use Databricks (in some cases)
- Built-in governance and auditing
Unlike traditional methods, Delta Sharing eliminates data duplication while maintaining full control.
What is Databricks-to-Databricks Delta Sharing?
Databricks-to-Databricks Delta Sharing is a specialized implementation designed specifically for environments using Unity Catalog.
Requirements
- Both workspaces must have Unity Catalog enabled
- Sharing occurs between metastores
What Makes It Powerful?
This method unlocks advanced capabilities such as:
- Table and schema sharing
- Notebook sharing
- Volume and AI model sharing
- Full governance and auditing
- Cross-cloud compatibility
Why Use Databricks-to-Databricks Sharing?
Here are the biggest advantages:
1. Simplified Setup
No need for complex configurations or manual data transfers.
2. Enhanced Security
- No token management required
- Authentication handled automatically
- Built-in auditing
3. High Performance
Data is accessed directly without duplication, improving efficiency.
4. Cross-Environment Flexibility
Works across:
- Azure
- AWS
- Google Cloud
Real-World Use Case: Metastore-to-Metastore Sharing
In Unity Catalog:
- Metastores are isolated by default
- They cannot communicate directly
Delta Sharing solves this by enabling:
- Secure data exchange between metastores
- Collaboration across teams or regions
- External organization data sharing
This is especially useful for enterprises with multi-region or multi-team architectures.
Step-by-Step: How to Implement Delta Sharing
Step 1: Create a Share
A share is a collection of data assets you want to distribute.
You can include:
- Tables
- Schemas
- Historical data
- Partitions
Example:
- Tables:
diamonds,taxi
Step 2: Add Additional Assets
Databricks-to-Databricks sharing allows more than just tables.
You can also include:
- Notebooks
- Views
- Materialized views
- Volumes
This is a major advantage over open sharing.
Step 3: Create a Recipient
A recipient represents the destination metastore (not an individual user).
Important notes:
- It points to a metastore, not a person
- Requires a sharing identifier
Step 4: Retrieve the Sharing Identifier
You can get this using:
SQL Method:
SELECT current_metastore();
UI Method:
- Go to settings
- Navigate to Delta Sharing
- Copy the sharing identifier
Step 5: Grant Access to the Share
Once the recipient is created:
- Assign the share to the recipient
- This completes the provider’s setup
Step 6: Access the Shared Data (Recipient Side)
On the recipient side:
- Navigate to Delta Sharing
- View available shares
- Mount the share to a catalog
Step 7: Mount the Share to a Catalog
This step makes the data usable.
After mounting:
- Tables become queryable
- Data is available for analytics tools like Power BI
Understanding Permissions
Permissions are critical in Delta Sharing.
Required Privileges:
USE PROVIDERCREATE CATALOG
Important Consideration:
Granting CREATE CATALOG:
- Allows users to create catalogs across the metastore
- Can introduce governance risks
Best Practice:
- Let admins create catalogs
- Then assign access to users
Sharing Across Organizations
When sharing externally:
- Enable Delta Sharing for external organizations
- Configure:
- Organization name
- Token settings (if required)
Note: This step is not required for internal sharing within the same tenant.
Key Security Features
- No token management for DB-to-DB sharing
- Centralized identity verification
- Built-in auditing and tracking
- Read-only access for recipients
Bonus: Sharing Notebooks
You can also share notebooks as part of the dataset.
Important points:
- Notebooks are view-only
- Users can clone them to edit
Best Practices for Power BI Integration
When using Delta Sharing with Power BI:
- Use shared catalogs as a single source of truth
- Avoid duplicating datasets
- Apply row-level security where needed
- Monitor usage via audit logs
For deeper learning and hands-on guidance, check out this training resource
