Role: Data Engineer (GCP)
Job Location:- Bentonville, AR (Local Preferred)
Job Type:- Long Term
Must Have
11+ years’ experience
-
Strong programming skills in Python, SQL
• Hands-on experience with cloud platforms
• Expertise in GCP data tools: BigQuery, Dataproc, Vertex AI, Pub/Sub, Cloud Functions.
• Strong hands-on experience with Apache Airflow (incl. Astronomer), PySpark, and Python.
• Familiarity with SQL, Spark SQL, Hive, PL/SQL, and data modeling.
• Comfortable supporting distributed data systems and large-scale batch/stream data processing.
• Optimize and support Spark jobs and ETL pipelines running on Dataproc.
Job Description:
Key Responsibilities:
1. Design, develop, and automate data processing workflows using Airflow, PySpark, and Dataproc on GCP.
2. Develop ETL (Extract, Transform, Load) processes that handle diverse data sources and formats.
3. Manage and provision GCP resources including Dataproc clusters, serverless batches, Vertex AI instances, GCS buckets, and custom images.
4. Provide platform and pipeline support to analytics and product teams, troubleshooting issues related to Spark, BigQuery, Airflow DAGs, and serverless workflows.
5. Collaborate with data scientists and analysts to understand data needs and deliver robust solutions.
6. Provide timely and effective technical support to internal users (e.g., data analysts, data scientists) addressing their data-related queries and problems
7. Optimize and fine-tune data systems for high performance, reliability, and cost efficiency.
8. Perform root cause analysis for recurring issues and collaborate with data analysts and scientists to implement preventative measures to minimize future occurrences.
Vaibhav | Senior Talent Acquisition Specialist
Amaze Systems Inc
USA: 8951 Cypress Waters Blvd, Suite 160, Dallas, TX 75019
Canada: 55 York Street, Suite 401, Toronto, ON M5J 1R7