Data Engineer

Team Connect is Poland’s leading nearshore and offshore IT provider. Since 2008 we successfully create and develop software for our clients.

We are also a proud certified Salesforce Partner. 

We specialize in Agile and DevOps-based software development. From the analysis stage through implementation. We develop backend, frontend, and mobile applications.

Currently, we are looking for: Senior Big Data Engineer

The ideal candidate will be responsible for designing, implementing and maintaining scalable data pipelines and solutions on on-prem / migration / cloud projects for large scale data processing and analytics.

BIG DATA ENGINEER @ CAPCO - WHAT TO EXPECT

• You will work on engaging projects with some of the largest banks in the world, on projects that will transform the financial services industry.
• You’ll be part of digital engineering team that develop new and enhance existing financial and data solutions, having the opportunity to work on exciting greenfield projects as well as on established Tier1 bank applications adopted by millions of users.
• You’ll be involved in digital and data transformation processes through a continuous delivery model.
• You will work on automating and optimising data engineering processes, develop robust and fault tolerant data solutions and enhancing security standards both on cloud and on-premise deployments.
• You’ll be able to work across different data, cloud and messaging technology stacks.
• You’ll have an opportunity to learn and work with specialised data and cloud technologies to widen the skill set. 

THINGS YOU WILL DOWork alongside clients to interpret requirements and define industry-leading solutions

• Design and develop robust, well tested data pipelines
• Demonstrate and help clients adhere to best practices in engineering and SDLC
• Excellent knowledge of building event-driven, loosely coupled distributed applications
• Experience in developing both on-premise and cloud-based solutions
• Build and improve strong relationships with peers, senior stakeholders and client
• Leading and mentoring the team of junior and mid-level engineers
• Contribute to security designs and have advanced knowledge of key security technologies e.g. TLS, OAuth, Encryption
• Support internal Capco capabilities by sharing insight, experience and credentials

TECH STACK: Python, OOP, Spark, SQL, Hadoop, GCP

Nice to have: Pub/Sub, Big Query, Kafka, Juniper, Apache NiFi, Hive, Impala, Cloudera, CI/CD

SKILLS & EXPERIENCES YOU NEED TO GET THE JOB DONE

• Strong cloud provider’s experience on GCP
• Hands on experience using Python. Scala and Java are also nice to have but not fully required.
• Experience in most or all of data and cloud technologies such as Hadoop, HIVE, Spark, Flume, PySpark, DataProc, Cloudera, Airflow, Oozie, S3, Terraform etc.
• Hands on experience with schema design using semi-structured and structured data structures
• Experience using messaging technologies – Kafka, Kafka Connect, Spark Streaming, Amazon Kinesis
• Strong experience in SQL
• Good understanding of the differences and tradeoff between SQL and NoSQL, ETL and ELT
• Understanding of containerisation (Docker, Kubernetes) and orchestration techniques
• Experience with data lake formation and data warehousing principles and technologies – BigQuery, Redshift, Snowflake
• Experience using version control tool such as Git.
• Familiar with development good practices and optimisation techniques.
• Experience in design, build and maintain CI/CD Pipelines on Jenkins, CircleCI
• Enthusiasm and ability to pick up new technologies as needed to solve your problems

We offer:

• Long-term cooperation
• Benefit package - Multisport, private medical care, life insurance
• Training budget
• Free English lessons
• Individual support from a dedicated company mentor