Connect Jupyter to Data Lake with Azure AD B2B

This 2018 tutorial joined a hosted Azure Notebook to Azure Data Lake Storage Gen1. A guest user signed in through Azure AD B2B, then the Data Lake ACL limited which folders the notebook could reach.

The design still illustrates a useful rule: authenticate the person, authorise the data path, and test the effective permission. The products and Python packages in the original example are now historical.

Why the original design used B2B and ACLs

The project needed people from different organisations to work with shared data without exchanging database passwords or storage keys.

The access model had these parts:

1. Azure AD B2B represented each external collaborator.
2. Azure role-based access control governed access at the Azure resource level.
3. Data Lake file-system ACLs limited access to the relevant folders and files.

Azure AD is now named Microsoft Entra ID, and Azure AD B2B is now Microsoft Entra B2B. Microsoft explains the terminology change in its current naming guide.

The archived 2018 notebook steps

The notebook installed the Gen1 management and client packages:

pip install azure-mgmt-resource
pip install azure-mgmt-datalake-store
pip install azure-datalake-store

It then requested a user token through the device sign-in flow:

import pandas as pd
from azure.datalake.store import core, lib, multithread

token = lib.auth(
    tenant_id="<AzureADTenantId>",
    resource="https://datalake.azure.net/",
)

The token was passed to the Gen1 file-system client:

adls_file_system_client = core.AzureDLFileSystem(
    token,
    store_name="<DataLakeName>",
)

print(adls_file_system_client.ls())
adls_file_system_client.mkdir("/addedfromjupyter")

The visible directory list and write operation depended on the signed-in user’s ACLs. That made the notebook a consumer of the existing access model, not a bypass around it.

Why this code should not be reused

Azure Data Lake Storage Gen1 retired on February 29, 2024. Microsoft records that date in the Azure product lifecycle.

The sample also depends on legacy Python packages and a 2018 hosted-notebook experience. Do not copy it into a new environment or use its package versions as a security baseline.

The current pattern for new work

For Data Lake Storage Gen2, Microsoft documents the azure-storage-file-datalake client with token credentials from azure-identity.

A minimal client shape is:

from azure.identity import DefaultAzureCredential
from azure.storage.filedatalake import DataLakeServiceClient

credential = DefaultAzureCredential()
service = DataLakeServiceClient(
    account_url="https://<storage-account>.dfs.core.windows.net",
    credential=credential,
)

See Microsoft’s maintained Python ACL guidance for Data Lake Storage Gen2 before implementing it.

The credential does not grant access by itself. An administrator still needs to configure the identity, Azure roles, file-system ACLs, network controls, and guest lifecycle.

Validate the boundary, not only the sign-in

Test with the same identity type your users will have. Confirm that the user can read and write only the intended paths, then repeat the test after access is removed.

A successful sign-in proves identity. It does not prove that authorisation, data isolation, or offboarding is correct.