Success Story: DCCEEW

DCCEEW modernises Biodiversity Offset Scheme data with a secure Microsoft Fabric platform

Executive Summary

The Department of Climate Change, Energy, the Environment and Water (DCCEEW) oversees Australia’s Biodiversity Offset Scheme (BOS), ensuring development impacts on native habitats are offset by conservation actions. Previously, BOS data was scattered across SharePoint spreadsheets, legacy SSIS pipelines, and multiple Power BI reports – causing up to 10 days of delay before consolidated insights could be delivered. Data-Driven AI partnered with DCCEEW to implement a unified Microsoft Fabric platform to ingest, cleanse, and govern BOS data via a Medallion Architecture (bronze, silver, gold layers). As a result, DCCEEW reduced end-to-end data-prep time from days to under two hours and now delivers near-real-time Power BI dashboards – empowering faster, evidence-based environmental decisions across the agency .

Data-Driven’s implementation based on Microsoft Fabric has now automated our data integration from various sources and set a foundation for a consolidated data estate long-term. We can now analyse our data more efficiently and make informed decisions faster. Further, our new Microsoft Fabric Data Platform has given us an excellent source for our key Power BI Reports and removed our dependencies on direct-to-source reporting. Data-Driven helped enable our transition to Microsoft Fabric and to discover, ingest and consolidate source data.

Durvesh Chattopadhyay

Data and Information Management, BOS Digital Transformation Team, DCCEEW

The Challenge

Identify the cause of data systems delays, data silos, and quality issues

Prior to this project, BOS data resided in disparate systems, SharePoint-hosted CSV/Excel files, legacy SSIS pipelines, multiple Power BI workspaces, and a small on-premises SQL Server data warehouse. These disconnected sources caused frequent data silos, inconsistent quality, and processing bottlenecks. Generating consolidated BOS reports could take up to 10 days, severely limiting DCCEEW’s ability to respond to emerging conservation issues or regulatory changes. Stakeholders repeatedly flagged that historical data was lost during ETL jobs, and manual corrections were commonplace. With the 2024 federal mandate for more transparent, digital-first environmental reporting, DCCEEW sought a partner experienced in Microsoft Fabric to design a modern data ecosystem that would eliminate silos, enforce governance, and accelerate time to insight

The Solution

Shift to a unified Microsoft Fabric solution to meet 2024 federal transparency mandates.

Data-Driven AI implemented a multi-stage Microsoft Fabric solution.

1. Medallion Architecture Implementation (Bronze, Silver, Gold):

Bronze Layer: Raw BOS datasets were ingested nightly from SharePoint (CSV/Excel), BOAMS, and the existing BOS Data Warehouse into OneLake
Silver Layer: Fabric pipelines and Fabric notebooks cleansed and harmonised Bronze layer Lakehouse tables, standardising naming conventions, normalising date/time formats, and reconciling lookups.
Gold Layer: Curated, analytics-ready tables were materialised in Fabric, providing a single source of truth for all BOS transactions, offsets, and compliance records.

2. End-to-End ELT and Orchestration:

Fabric pipelines now orchestrate data ingestion each night and process records. These replaced manual SSIS jobs, eliminating frequent schedule failures and data-loss incidents.

3. Interactive Reporting with Power BI:

Existing Power BI dashboards were re-linked to the Fabric “Gold” dataset.
Key BOS metrics (e.g., total hectares offset, compliance gaps, offset project timelines) now refresh automatically, eliminating manual data uploads.

4. Security and Compliance:

Role-Based Access Control (RBAC) ensures that only those with explicit clearance can view or modify records.

Key Outcomes for the Business

DCCEEW has realised these outcomes by using Data-Driven’s solution

Reduction in Data-Prep Time

Prior to Fabric, consolidating BOS data took up to 10 days; the new Fabric pipelines now complete ingestion and cleansing in under 2 hours.

Scalable, Cost-Effective Platform

Fabric’s serverless architecture handles data processing quickly and reliably.

Faster, Self-Service Reporting

BOS stakeholders now view near-real-time Power BI dashboards, tracking offset project statuses, compliance gaps, and environmental KPIs without waiting for monthly manual refreshes.

About The Department of Climate Change, Energy, the Environment and Water (DCCEEW)

The NSW Department of Climate Change, Energy, the Environment and Water (DCCEEW) is responsible for shaping and implementing environmental policy, including the Biodiversity Offset Scheme (BOS), which ensures that development impacts on native habitats are offset by conservation actions. Its core responsibilities include monitoring greenhouse-gas emissions, enforcing ecological regulations, and providing transparent environmental data to policymakers, researchers, and the public. DCCEEW’s strategic vision is to foster data-driven sustainability and resilience – ensuring environmental decisions are informed by timely, accurate analytics.

Would you like to get insights from your data?

We can help you design a strategic roadmap to get you from your current data estate to your future target state with advanced analytics.

We do end-to-end consulting from integrating and consolidating your data, Al enablement and visualisations/apps to surface the insights.