02-Unity Catalog Configuration - Prerequisites

Prerequisites for Unity Catalog Configuration

Logging into Azure

• The video begins with an introduction to the prerequisites for configuring Unity Catalog, using a personal account for demonstration.

• Users are instructed to log into Azure via portal.tasher.com using their credentials.

Creating a User in Azure Active Directory

• The first step involves creating a new user in Azure Active Directory. Users should navigate to the user section and select "Create New User."

• When creating the user, it's important to assign the "Global Administrator" role from Active Directory, which is essential for managing resources effectively.

Assigning Roles at Subscription Level

• After creating the user, note down the User ID as it will be used later in the demo.

• The next step requires assigning roles at the subscription level by selecting the appropriate subscription and adding role assignments.

Logging In with New Account

• Once roles are assigned, users need to sign out and log back in using the newly created account.

• Upon logging in, users must change their password to meet security requirements (including uppercase letters, lowercase letters, and special characters).

Setting Up Multi-Factor Authentication

• Users may be prompted to set up multi-factor authentication; however, this can be deferred if desired during this demo.

Creating Resources for Unity Catalog

Creating a Resource Group

• The first resource needed is a Resource Group. Users should create one named appropriately (e.g., "Unity catalog_demo") and select their preferred location (e.g., South India).

Creating Databricks Workspace

• Next, users are guided to create an Azure Databricks workspace. They should ensure that all resources share the same location and choose a premium tier since Unity Catalog requires it.

Networking Options for Databricks Workspace

• Users can opt for either custom networking or default settings where Databricks manages network configurations automatically.

Additional Resources Creation

Creating a Storage Account and Access Connector in Azure

Setting Up the Storage Account

• The storage account is placed in the same Resource Group as other resources. A unique name must be chosen for the storage account to avoid errors.

• The location selected for the storage account is South India, with a standard performance tier and locally redundant redundancy settings chosen for this demo.

• Hierarchical namespace must be enabled; default settings are used for networking, data protection, and encryption options.

• An access connector for Databricks is required to connect Databricks with the storage account or ADLs (Azure Data Lake Storage).

Creating an Access Connector

• A new access connector is created within the Unity catalog demo Resource Group, using a name that can be customized. The same location (South India) is selected.

• Default tag settings are left unchanged, and managed identity remains on by default during creation of the access connector.

Configuring Permissions

• After resource deployment, multiple resource groups are visible including those created automatically by Azure.

• To add permissions, navigate to the storage account's Identity and Access Control (IAM), selecting "Add role assignment."

Assigning Roles

• Search for "Storage Blob Data Contributor" role within IAM settings of the storage account to assign necessary permissions.

• Choose "Managed Identity" option when assigning roles; select the previously created access connector from available options.

Launching Databricks Workspace

• Open Databricks workspace using launch option; multi-factor authentication setup can be deferred if desired.

• Navigate to manage accounts option in Databricks workspace to configure Unity Catalog.

Accessing Unity Catalog Configuration