Lead Kafka Engineer Chennai, India
Lead Kafka Engineer Description
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
We are looking to hire an experienced Lead Kafka Engineer who will be responsible for installing, monitoring, troubleshooting, and maintaining Kafka platform, ensuring optimal performance, security, developing new features/automation/integration.
#LI-DNI#EasyApply
Responsibilities
- Install and provision new Kafka clusters and supporting infrastructure
- Continuously monitor the health and performance of Kafka platforms and data pipelines
- Troubleshoot and resolve issues related to data pipelines, network disruptions, and other infrastructure failures
- Perform regular performance tuning and cost optimization for Kafka components
- Manage the upgrade process for Kafka platforms, including planning, testing, and implementation
- Enforce and manage security protocols including access control, encryption, and conduct regular security audits
- Implement disaster recovery procedures and conduct regular platform backups
- Oversee capacity management and scaling projections
- Document technical procedures, configurations, issue resolutions and share knowledge across teams
- Collaborate with internal and vendor support teams for escalated issue resolutions
- Maintain and enhance Infrastructure as Code (IaC) and Configuration Management (CM) automations
- Develop and refine onboarding and automation scripts for streamlined operations
- Facilitate Kafka setup for application teams including consumers, producers, and connectors
- Respond to team requests efficiently, converting complex issues into CLOUD Tickets when required
- Integrate new vendor features and capabilities in collaboration with relevant stakeholders
Requirements
- Proven experience in the implementation and maintenance of Confluent Platform
- Minimum 8 years of experience in Kafka administration
- Knowledge of Helm and Kubernetes
- Proficiency in deploying Kafka in Kubernetes
- Skills in Python or Shell Scripting
- Background in cloud technologies including AWS, GCP (compute, networking, storage, IAM)
We offer
- Opportunity to work on technical challenges that may impact across geographies
- Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications
- Opportunity to share your ideas on international platforms
- Sponsored Tech Talks & Hackathons
- Unlimited access to LinkedIn learning solutions
- Possibility to relocate to any EPAM office for short and long-term projects
- Focused individual development
- Benefit package:
- Health benefits
- Retirement benefits
- Paid time off
- Flexible benefits
- Forums to explore beyond work passion (CSR, photography, painting, sports, etc.)