Middle/Senior Data Engineer Lithuania or Remote
Middle/Senior Data Engineer Description
Job #: 74138
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
We are looking for an open-minded Middle/Senior Data Engineer with a healthy level of critical thinking to develop and maintain Data-Hub architecture within Google Cloud Platform. On this project, we are developing pipelines to build business-related data-products and design cloud-based platform solutions to support Data-Hub functionality.
Our customer is a multinational delivery services company, known for its overnight shipping service and pioneering a system that could track packages and provide real-time updates on package location.
We have several teams, which are performing the following activities:
• Building Dataflow pipelines to create data products on top of existing resources within Google Cloud infrastructure
• Performing data ingestion processes to include existing on-premise data sources into existing Data-Hub cloud-based platform
• Platform engineering, including cloud-based solutions to support Data-Hub architecture, such as Ingestion Frameworks, Platform Orchestrators, Exploratory environments etc.
• MLOps activities to build production-ready cloud-based solutions to support existing ML pipelines
• Migration of existing on-premise infrastructure to Google Cloud
#LI-DNI
#LI-DNP
#cloudtill30Apr
DESCRIPTION
We are looking for an open-minded Middle/Senior Data Engineer with a healthy level of critical thinking to develop and maintain Data-Hub architecture within Google Cloud Platform. On this project, we are developing pipelines to build business-related data-products and design cloud-based platform solutions to support Data-Hub functionality.
Our customer is a multinational delivery services company, known for its overnight shipping service and pioneering a system that could track packages and provide real-time updates on package location.
We have several teams, which are performing the following activities:
• Building Dataflow pipelines to create data products on top of existing resources within Google Cloud infrastructure
• Performing data ingestion processes to include existing on-premise data sources into existing Data-Hub cloud-based platform
• Platform engineering, including cloud-based solutions to support Data-Hub architecture, such as Ingestion Frameworks, Platform Orchestrators, Exploratory environments etc.
• MLOps activities to build production-ready cloud-based solutions to support existing ML pipelines
• Migration of existing on-premise infrastructure to Google Cloud
#LI-DNI
#LI-DNP
#cloudtill30Apr
What You’ll Do
- Depending on the team you join, activities might differ but not limited to:
- Designing and implementing data pipelines
- Contributing to platform development
- Taking part in the design of on-prem to cloud migration activities
What You Have
- Solid experience in Java
- Understanding of common Big-Data problems (splitability, massive parallel processing, load-balancing, data locality etc.) and related technologies (Apache Spark, Map-Reduce)
- Familiarity with common data-related architectures such as Data-Hub/Data Lake is a big plus
- Focusing on the result: educational activities, proof of concepts development, research tasks are mapped to a tangible result
- Ability and desire to work in a team
- Sharing the ideas of Brutal Intellectual Honesty
- Acquaintance with with data-modeling: relational/dimensional data, SCD, snowflaking etc
- Experience in cloud-based solutions, GCP, Terraform, Kubernetes is a big plus
Technologies
- Apache Beam (Java api)
- Dataflow
- BigQuery
- Cloud Storage
- Cloud Functions
- Airflow
- PubSub
- Dataproc
- Apache Kafk
- Jenkins
- Kubernetes with Google Kubernetes Engine
- Terraform
- Avro
- Parquet
We offer
- Outstanding career development opportunities
- Knowledge-sharing with colleagues all around the world
- Regular assessments and salary reviews
- Competitive compensation
- Social package – medical & family care
- Friendly team and enjoyable working environment
- Unlimited access to learning courses (LinkedIn learning, EPAM training courses, English regular classes, Internal Library)
- Community of 36,700+ industry’s top professionals
- Flexible working schedule
- Corporate and social events
