Job description

Databricks Platform Administrator Cincinnati, OH locals preferable , but Remote is okay Databricks Platform Administration • Deploy, configure, and maintain Azure Databricks workspaces • Manage clusters, pools, and jobs for optimal performance. • Handle Databricks Runtime upgrades and compatibility testing. • Configure Databricks notebooks, Repos, and Delta Lake for efficient workflows. Security & Access Control: • Implement Identity and Access Management (IAM) using Azure AD / Azure IAM / SCIM. • Set up and enforce Role-Based Access Control (RBAC) and network security. • Ensure compliance with data governance policies (Unity Catalog, creation of Catalogs, schema etc.). • Applying grants of catalogs and schemas • Manage personal access tokens (PATs) and OAuth-based authentication. Performance Monitoring & Optimization: • Monitor cluster performance, auto-scaling, and cost optimization. • Analyze and fine-tune Spark jobs to improve execution efficiency. • Use Databricks SQL for querying and performance diagnostics. Integration & Automation: • Integrate Databricks with Azure Data Factory, Synapse, Snowflake, Kafka, and other data sources. • Automate cluster management using Terraform. • Implement CI/CD pipelines for Databricks notebooks using GitHub or Azure DevOps. Troubleshooting & Support: • Investigate and resolve job failures, cluster crashes, and query performance issues. • Provide L2/L3 support for Databricks infrastructure issues. • Collaborate with Databricks support for bug fixes and escalations. Please share to manasa@calabitek.com

Requirements