Stop the clock! Microsoft Fabric just unleashed a wave of updates for Dataflows Gen2 that fundamentally change how you approach ETL (Extract, Transform, Load). If you’ve been hesitant about using Dataflows Gen2 due to performance drag or capacity unit (CU) costs, the new features rolled out at FabCon Vienna will make you rethink your entire data engineering strategy.
The days of slow previews and high capacity consumption are over. We’re diving into the game-changers from accelerated authoring to a massive 90% cost reduction for long-running processes, highlighting these critical Fabric Dataflows Gen2 Updates.
What Is Dataflows Gen2 and Why Use It?
Dataflows Gen2 is the dedicated, UI-driven ETL tool inside Microsoft Fabric. It provides a familiar, low-code interface Power Query for connecting to countless sources, transforming data, and landing it in multiple destinations (like Lakehouses, Warehouses, or even external SQL/Snowflake).
It is the most direct bridge for Power BI users transitioning to enterprise-scale data engineering in Fabric. This latest round of Dataflows Gen2 updates significantly enhances its utility.
Performance: Write Code Faster, Run Queries Quicker
The biggest pain points in dataflow development and execution have been directly addressed with three major acceleration features included in these Dataflows Gen2 updates:
1. Preview-Only Steps for Lightning-Fast Authoring
When developing complex transformations, waiting for the data preview to render every time you apply a step is tedious.
-
The Fix: You can now right-click any transformation step and select “Enable only in previews.”
-
The Impact: This feature, part of the new Fabric Dataflows Gen2 Updates, speeds up your development by using sample data to render results almost instantly. Crucially, the transformation is skipped during the final scheduled execution, improving refresh time without compromising data integrity.
2. Modern Query Evaluation Engine
This is a backend upgrade that boosts the speed of your dataflow execution.
-
The Fix: Simply navigate to Options > Scale and check the box for “Query evaluation.”
-
The Impact: It improves overall query performance using a new, more efficient engine. This is a vital new enhancement for Dataflows Gen2.
3. Partitioned Compute for Parallel Execution
This feature is a game-changer for working with large datasets and files structured in partitions (like Parquet files in ADLS Gen2).
-
The Fix: Under Options > Scale, enable “Partitioned compute.”
-
The Impact: Dataflows Gen2 can now process data in parallel. It automatically detects and leverages pre-partitioned sources, evaluating them concurrently. This is a massive time-saver for large file sets, further showcasing the power of these new features for Dataflows Gen2.
Cost Optimization: Save Your Capacity Units (CUs)
For many organizations, the capacity unit consumption of Dataflows Gen2 was a point of concern. Microsoft has responded with a massive pricing adjustment a core component of the latest Fabric Dataflows Gen2 Updates.
The New Two-Tier Pricing Model
Capacity consumption for dataflows has been drastically reduced, making them a much more viable and efficient tool.
| Duration | Previous CU Rate | NEW CU Rate | Savings |
| First 10 Minutes | Higher Flat Rate | Flat 12 CU | 25% Reduction |
| Beyond 10 Minutes (Long-Running) | Steady Rate | Drops to 1.5 CU | Up to 90% Reduction |
The Takeaway: Whether you’re running short or long dataflows, you are spending fewer CUs. For extended, enterprise-scale operations, this new structure makes Dataflows Gen2 incredibly budget-friendly
New Features & Destinations
Dataflows Gen2 is also expanding its capabilities and reach within and outside Fabric.
-
Expanded Destinations: You can now write your transformed data to Azure Data Lake Storage Gen2 (ADLS Gen2), external SQL Databases, and a highly anticipated new Preview destination for Snowflake. These new destinations are a crucial part of the Fabric Dataflows Gen2 Updates.
-
Lakehouse Incremental Refresh: The classic, efficient Power BI incremental refresh model is now available when writing data to a Lakehouse table, ensuring faster processing for large, time-stamped datasets.
-
Enhanced Schema Navigation: When connecting your dataflow to a Fabric Warehouse, an advanced option now allows you to “Navigate using full hierarchy.” This finally lets you select and browse schemas (like
dboorsales) for cleaner data organization. -
Copilot Integration: AI is everywhere! Copilot is now integrated to help with:
-
Calculated Columns: Use natural language to generate M code for custom columns.
-
Explain Step: Right-click any transformation to get a natural language explanation of what that M step is doing
-
a great addition to Dataflows Gen2.
-
Modern Get Data: A new interface for connecting to data that includes a Copilot tab for conversational connection setup.
-
The New ETL Standard
The combination of massive CU cost reduction and the ability to run queries in parallel using Partitioned Compute firmly establishes Dataflows Gen2 as the intended enterprise-scale ETL tool in Fabric. If you shelved Dataflows Gen2 previously, it’s time to take advantage of these significant Fabric Dataflows Gen2 Updates and bring it back into your data ecosystem.
Want to build robust, modern data skills?
Check out this fantastic resource to build robust analytics expertise: Power BI Training from DataBear
