Lead Data Engineer (AWS Data Platform)

Position: Lead Data Engineer (AWS Data Platform)
Experience: 5 to 8 Years
Location: Hybrid (Pakistan)

Note: (Candidates must be available during UAE business hours and follow UAE public holidays)

Job Summary

We are seeking a highly skilled Lead Data Engineer with 5 to 8 years of experience in designing developing and optimizing large-scale data platforms and ETL/ELT pipelines. The ideal candidate will have strong hands-on expertise in PySpark AWS Glue Amazon EMR Amazon Redshift and SQL-based data warehousing along with proven experience in performance tuning and data optimization.

The candidate will work closely with a UAE-based customer and must be comfortable collaborating with distributed teams while adhering to UAE working hours and holiday schedules.

Key Responsibilities

• Design develop and maintain scalable data pipelines using PySpark AWS Glue and Amazon EMR.
• Build and optimize data ingestion transformation and processing frameworks for structured and semi-structured data.
• Develop and maintain enterprise data warehouse solutions using Amazon Redshift.
• Write complex SQL queries stored procedures and data transformations to support analytics and reporting requirements.
• Implement ETL/ELT processes to move data efficiently across multiple systems and platforms.
• Perform performance tuning and optimization of Spark jobs ETL pipelines SQL queries and Redshift workloads.
• Ensure data quality integrity security and governance across data platforms.
• Troubleshoot production issues and perform root cause analysis for data-related incidents.
• Collaborate with business stakeholders analysts architects and engineering teams to understand data requirements.
• Participate in code reviews technical design discussions and best-practice implementation.
• Monitor data pipelines and proactively identify opportunities for performance improvements and automation.
• Create and maintain technical documentation data models and operational procedures.

Required Skills & Experience

Must-Have Skills

• 5 to 8 years of experience in Data Engineering and Data Warehousing.
• Strong hands-on experience with PySpark.
• Extensive experience with AWS Glue.
• Experience building and managing workloads on Amazon EMR.
• Strong expertise in Amazon Redshift.
• Excellent SQL development and query optimization skills.
• Strong understanding of Data Warehousing concepts dimensional modeling and ETL/ELT processes.
• Experience in performance tuning of Spark jobs SQL queries ETL pipelines and data warehouse workloads.
• Experience handling large-scale datasets and distributed data processing.
• Strong debugging troubleshooting and analytical skills.

Preferred Skills

• Experience with additional AWS services such as S3 IAM CloudWatch Lambda and Step Functions.
• Knowledge of CI/CD pipelines and DevOps practices for data platforms.
• Experience with workflow orchestration tools.
• Familiarity with data governance security and compliance practices.
• Exposure to Agile/Scrum development methodologies.

Qualifications

• Bachelors degree in Computer Science Software Engineering Information Technology or a related field.
• Relevant AWS certifications will be considered an advantage.

Soft Skills

• Strong communication and stakeholder management skills.
• Ability to work independently in a remote environment.
• Excellent problem-solving and analytical thinking abilities.
• Ability to collaborate effectively with cross-functional and geographically distributed teams.
• Strong ownership mindset and commitment to delivering high-quality solutions.

Required Experience:

Senior IC

Employment Type : Full-Time
Department / Functional Area: Data Engineering
Experience: years
Vacancy: 1