Databricks Platform Administration
•
Administer Databricks workspaces across development, test, and production environments.
•
Manage clusters, SQL Warehouses, serverless resources, jobs, workflows, and workspace configurations.
•
Configure and maintain Unity Catalog, external locations, storage credentials, and data access controls.
•
Monitor platform health, performance, utilization, and costs.
•
Implement security controls, audit logging, and compliance requirements.
•
Partner with Engineering teams to establish platform standards and best practices.
•
Administer Snowflake accounts, warehouses, databases, schemas, roles, resource monitors, and security policies.
•
Manage workload performance, query optimization, and warehouse sizing.
•
Implement Snowflake governance, access controls, and secure data-sharing processes.
•
Manage integrations between Snowflake, Databricks, cloud storage, and downstream applications.
•
Perform capacity planning and cost optimization activities.
Infrastructure as Code (Terraform)
In cooperation with DevOps team, own and maintain Terraform-based platform administration.
Responsibilities include:
•
Create and maintain Terraform modules for:
•
Snowflake databases and schemas
•
Data platform infrastructure
•
Enforce platform ownership boundaries:
•
Platform team owns catalogs, schemas, databases, storage credentials, permissions, and governance controls.
•
Engineering teams own only:
•
Establish GitOps-based deployment pipelines for all administrative changes.
•
Ensure all platform configuration changes are version-controlled and auditable.
Identity & Access Management (Terraform Driven)
Own enterprise access governance for Databricks and Snowflake.
Responsibilities include:
•
Implement role-based access control (RBAC) using Terraform.
•
Manage user provisioning, group assignments, and entitlement policies.
•
Maintain Azure AD / Entra ID group integration.
•
Snowflake roles and grants
•
Eliminate manual access assignment wherever possible.
•
Conduct periodic access reviews and remediation.
•
Maintain least-privilege access standards.
Unity Catalog Ownership & Governance
Serve as the primary owner of Unity Catalog governance.
Responsibilities include:
•
Own catalog lifecycle management.
•
Define and enforce naming standards.
•
Manage catalog and schema ownership models.
•
Maintain metadata governance standards.
•
Implement data classification and tagging frameworks.
•
Enforce governance policies across environments.
•
Cross-workspace data access
•
Drive cleanup and rationalization of unused assets.
•
Establish processes for catalog onboarding and retirement.
Hive Metastore Cleanup & Rationalization
Lead migration and cleanup initiatives to modernize Databricks governance.
Responsibilities include:
•
Inventory legacy Hive Metastore assets.
•
Lead migration of assets into Unity Catalog.
•
Implement archival and retention policies.
•
Remove obsolete datasets and metadata.
•
Reduce cloud storage costs through lifecycle management and cleanup activities.
•
Improve metadata quality and catalog discoverability.
•
Publish governance and cleanup metrics.
Data Governance & Compliance
•
Establish and enforce enterprise data governance standards.
•
Support audit and compliance requirements.
•
Implement data lineage and metadata management practices.
•
Maintain documentation for governance policies and platform standards.
•
Partner with Security and Compliance teams on platform reviews.
Platform Reliability & Operations
•
Manage incident response and root-cause analysis.
•
Develop operational runbooks and support procedures.
•
Maintain SLAs for platform availability and support.
•
Implement automated monitoring and alerting.
•
Plan platform upgrades and migrations.