We are seeking an experienced Databricks Platform Engineer / Administrator with strong expertise in managing and optimizing Databricks environments on AWS. The ideal candidate will have hands-on experience in platform administration, data pipeline management, and cloud infrastructure automation.
Must-Have Skills
- Strong experience in Databricks administration (workspace, cluster, job management)
- Hands-on experience with AWS services (S3, IAM, VPC, CloudWatch, EKS/ECS)
- Expertise in Terraform (Infrastructure as Code)
- Strong experience with dbt and Fivetran
- Proficiency in Python / PySpark for automation and data processing
- Experience in CI/CD using GitHub Actions
- Strong understanding of data pipelines and cloud data architecture
- Experience with Unity Catalog and data governance in Databricks
- Knowledge of monitoring and observability tools (e.g., Datadog)