Big Data Retraining program (Java or Scala)

We are re-establishing Big Data retraining program for middle data engineers. Big Data is one of our core-expertise. We constantly need new team members, eager developers to grow our data engineering. This team requires quite a niche skillset (Scala/Spark) and we decided to look for candidates to take a part in educational program which contains 95% practice approach.

Responsibilities:

• The program/project includes (but not limited):
• Uploader (Go written binary client with its backend API to transfer data from third-party into platform)
• Scheme registry (It is a metadata store with dataset schemas including schema management capabilities)
• Pipeline manager (Manage ETL pipelines: CRUD and execute. Triggers ETL pipeline on upload event Pipelines metadata storage)
• (Airflow DAG) data ingestion pipeline including data decryption, validation (optionally) and ingestion.
• Monitor data metrics. uploads and integrity. At least data size, num objects, uploader entity, path.
• Hone the soft skills necessary for effective collaboration with customer team members and stakeholders, including engineering managers.

Min requirements:

Candidate for this position should have prior experience as Backend Engineer with focus and expertise of databases, data processing and ETL pipelines.

• Java and/or Scala
• SQL
• ETL pipelines
• AWS basic knowledge preferably (or knowledge of any other cloud service)

Would be a plus:

• Python
• Airflow
• MySQL, PostgreSQL, MongoDB, NoSQL, Cassandra, Hadoop
• Snowflake
• Knowledge of Apache Iceberg, Flink, Druid
• Kafka
• Data Lakes, Data Warehouse
• AWS(EKS, IAM, S3, SNS, SQS, MSK etc), Docker, Kubernetes
• Theoretical knowledge of Big Data concepts
• Commercial experience, working with real clients (not only pet-projects or RnD or Internships)
• Eager to invest his/her time to education and learn new technologies

We offer:

• Opportunity to work on bleeding-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Benefits package - medical insurance, sports
• Corporate social events
• Professional development opportunities
• Well-equipped office

About us:

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.