Book a call

Case Study: Department of Climate Change, Energy, the Environment and Water

DCCEEW modernizes Biodiversity Offset Scheme data with a secure Microsoft Fabric platform and Open AI
nsw government

Executive Summary

The Department of Climate Change, Energy, the Environment and Water (DCCEEW) oversee Australia’s Biodiversity Offset Scheme (BOS), ensuring development impacts on native habitats are offset by conservation actions. Previously, BOS data was scattered across SharePoint spreadsheets, legacy SSIS pipelines, and multiple Power BI reports – causing up to 10 days of delay before consolidated insights could be delivered. Data-Driven AI partnered with DCCEEW to implement a unified Microsoft Fabric platform to ingest, cleanse, and govern BOS data via a Medallion Architecture (bronze, silver, gold layers). As a result, DCCEEW reduced end-to-end data-prep time from days to under two hours and now delivers near-real-time Power BI dashboards – empowering faster, evidence-based environmental decisions across the agency.
Read More

Jump to:

“Data-Driven’s implementation based on Microsoft Fabric has now automated our data integration from various sources and set a foundation for a consolidated data estate long-term. We can now analyse our data more efficiently and make informed decisions faster. Further, our new Microsoft Fabric Data Platform has given us an excellent source for our key Power BI Reports and removed our dependencies on direct-to-source reporting. Data-Driven helped enable our transition to Microsoft Fabric and to discover, ingest and consolidate source data.”

Durvesh Chattopadhyay
Durvesh Chattopadhyay
Data Management, BOS Digital Transformation Team

Challenges

Identify the cause of data systems delays, data silos, and quality issues

Prior to this project, BOS data resided in disparate systems, SharePoint-hosted CSV/Excel files, legacy SSIS pipelines, multiple Power BI workspaces, and a small on-premises SQL Server data warehouse. These disconnected sources caused frequent data silos, inconsistent quality, and processing bottlenecks.Generating consolidated BOS reports could take up to 10 days, severely limiting DCCEEW’s ability to respond to emerging conservation issues or regulatory changes. Stakeholders repeatedly flagged that historical data was lost during ETL jobs, and manual corrections were commonplace.With the 2024 federal mandate for more transparent, digital-first environmental reporting, DCCEEW sought a partner experienced in Microsoft Fabric to design a modern data ecosystem that would eliminate silos, enforce governance, and accelerate time to insight.

The Solution

Shift to a unified Microsoft Fabric solution to meet 2024 federal transparency mandates.

Data-Driven AI implemented a multi-stage Microsoft Fabric solution. 1 Medallion Architecture Implementation (Bronze, Silver, Gold):
  • Bronze Layer: Raw BOS datasets were ingested nightly from SharePoint (CSV/Excel), BOAMS, and the existing BOS Data Warehouse into OneLake.
  • Silver Layer: Fabric pipelines and Fabric notebooks cleansed and harmonised Bronze layer Lakehouse tables, standardising naming conventions, normalising date/time formats, and reconciling lookups.
  • Gold Layer: Curated, analytics-ready tables were materialised in Fabric, providing a single source of truth for all BOS transactions, offsets, and compliance records.
2 End-to-End ELT and Orchestration:
  • Fabric pipelines now orchestrate data ingestion each night and process records.
  • These replaced manual SSIS jobs, eliminating frequent schedule failures and data-loss incidents.
3 Interactive Reporting with Power BI:
  • Existing Power BI dashboards were re-linked to the Fabric “Gold” dataset.
  • Key BOS metrics (e.g., total hectares offset, compliance gaps, offset project timelines) now refresh automatically, eliminating manual data uploads.
4 Security and Compliance:
  • Role-Based Access Control (RBAC) ensures that only those with explicit clearance can view or modify records.
Azure DevOpsAzure OpenAIMicrosoft FabricPower AppsPower BI

Key Business Outcomes

Faster, Unified, and Secure BOS Insights
Improved Data Accessibility

DCCEEW gained a unified and modern data platform on Azure, making critical environmental and energy data more accessible to stakeholders.

Streamlined Reporting & Analytics

The integration of Power BI and Azure Synapse enabled real-time dashboards and automated reporting, reducing reliance on manual processes.

Future-Ready Data Infrastructure

Leveraging Azure Data Factory, Synapse, and scalable cloud architecture, the department established a secure and flexible foundation to support ongoing sustainability initiatives.

DCCEEW has transformed its BOS data management, reducing end-to-end data preparation from days to under two hours. Stakeholders now have near-real-time access to validated, consolidated datasets, enabling faster, evidence-based environmental decisions. The solution also enforces data governance, ensures historical data integrity, and eliminates manual corrections.

“Our long-standing partnership with DCCEEW reflects a shared commitment to innovation and environmental stewardship. By modernising the Biodiversity Offset Scheme with Microsoft Fabric, we’ve helped build a future-ready data platform that delivers trusted insights in near real-time—empowering faster, evidence-based decisions across the agency.”

Rodney Joyce
Rodney Joyce Managing Director
nsw government

The Technical Solution

Data-Driven deployed a secure, scalable Microsoft Fabric platform with a Medallion Architecture: the Bronze Layer ingested raw BOS datasets nightly from SharePoint, BOAMS, and the legacy Data Warehouse into OneLake; the Silver Layer cleansed and harmonised data using Fabric pipelines and notebooks, standardising naming conventions, normalising date/time formats, and reconciling lookups; and the Gold Layer materialised analytics-ready tables providing a single source of truth for BOS transactions, offsets, and compliance records. ELT Orchestration automated nightly ingestion and transformation, replacing manual SSIS jobs and eliminating failures and data loss; Power BI Integration ensures interactive dashboards refresh automatically with curated data; and Security and Compliance with RBAC ensures only authorised users can view or modify sensitive records. This solution enables DCCEEW to respond quickly to environmental and regulatory changes while providing reliable, governed, and actionable data for decision-making.

DCCEEW

The NSW Department of Climate Change, Energy, the Environment and Water (DCCEEW) is responsible for shaping and implementing environmental policy, including the Biodiversity Offset Scheme (BOS), which ensures that development impacts on native habitats are offset by conservation actions. Its core responsibilities include monitoring greenhouse-gas emissions, enforcing ecological regulations, and providing transparent environmental data to policymakers, researchers, and the public. DCCEEW’s strategic vision is to foster data-driven sustainability and resilience – ensuring environmental decisions are informed by timely, accurate analytics.

Government

Energy and resources

NSW, Australia

Subscribed! We'll let you know when we have new blogs and events...

Ok