Senior Data Engineer

Project information:

• Industry: Insurance
• Rate: up to 185 pln/h net + vat, B2B
• Location: Warsaw – first 2-3 months hybrid, then remote
• Project language: Polish, English

About the Role

As a Senior Data Engineer, you will design and build scalable Data Hubs that support analytical, reporting, operational, and Generative AI use cases. This role involves defining standards, best practices, and architecture patterns, mentoring engineers, and ensuring high-quality data solutions.

Responsibilities:

• Data Hub Architecture – Design and implement efficient, scalable Data Hubs integrating multiple data sources.
• Technical Leadership – Define best practices, patterns, and standards, ensuring consistency across implementations.
• Mentoring & Strategy – Guide engineers, review code, and provide technical direction.
• Data Pipelines – Develop batch and real-time data processing workflows.
• Quality & Monitoring – Implement data validation, anomaly detection, and automated monitoring, taking action when needed.
• Automation & CI/CD – Enable seamless deployment and infrastructure automation.
• Security & Compliance – Ensure data governance, security, and regulatory compliance.
• Documentation – Maintain clear, structured technical documentation.

Requirements:

• Python, SQL – Strong data engineering and automation expertise.
• Databricks & Spark – Deep knowledge of Databricks (primary tool) and Apache Spark.
• Azure Data Stack – Experience with Data Factory, ADLS, Synapse, Azure SQL, Event Hubs.
• Real-Time Processing – Expertise in streaming data and real-time analytics.
• Automation & DevOps – Proficiency in CI/CD, Terraform, Kubernetes/AKS, Docker.
• Leadership & Mentoring – Ability to set direction, mentor engineers, and drive best practices.
• Documentation & Communication – Clear, structured technical writing and knowledge sharing.
• Language Skills – English proficiency, minimum B2 level.

Technology Stack

• Primary Tools – Databricks, Apache Spark, Delta Lake.
• Azure Services – Data Factory, ADLS, Azure SQL, Synapse, Azure DevOps, Event Hubs.
• Development & Automation – Python, SQL, GitHub, Terraform, Docker, Kubernetes/AKS.