senior data engineer

Data Software Engineering, Python, Amazon Web Services, Databricks, Apache Spark, SQL

Poland

Facebook LinkedIn Twitter Send via email

Senior Data Engineer

Facebook LinkedIn Twitter Send via email

We are searching for a skilled Senior Data Engineer to join our team. If you are proficient in Python and AWS, and you have experience with migrating GCP BigQuery and DataFlow pipelines into Databricks hosted on AWS, we invite you to apply for this exciting opportunity.

responsibilities

Collaborate with the team on the migration of GCP BigQuery and DataFlow pipelines to Databricks hosted on AWS, ensuring a seamless transition
Design, develop, and optimize data pipelines using Python, Spark, and Databricks to meet business requirements and enhance data processing efficiency
Work closely with cross-functional teams to understand data requirements and contribute to the architecture and implementation of data solutions
Perform data modeling and transformation tasks to prepare data for analysis and reporting
Collaborate on code reviews, ensuring the quality and reliability of data pipelines
Troubleshoot and resolve issues related to data pipelines, ensuring smooth and consistent data flow

requirements

A minimum of 3 years of relevant experience as a Data Software Engineer, demonstrating your proficiency in developing and optimizing data solutions
Strong familiarity with AWS, leveraging its resources for hosting and managing data solutions
Proficiency in Python, utilizing its capabilities to build robust and efficient data pipelines
Experience with Apache Spark, enabling you to work with large-scale data processing and analysis
Expertise in Databricks, particularly in the context of migrating pipelines and working on AWS-hosted platforms
Solid command of SQL, allowing you to perform data transformations and analysis effectively
Fluent English communication skills at a B2+ level, facilitating effective collaboration with a diverse team

nice to have

Familiarity with GCP, particularly experience with BigQuery and DataFlow
Knowledge of GCP BigQuery and DataFlow, aiding in the migration process and facilitating collaboration across platforms