Data Engineer in Test

We are undergoing a significant data transformation to ensure that accurate and consistent data is available precisely when and where it's needed.

Responsibilities

• Write PySpark and SQL scripts to validate data pipelines, transformations, and integrations.
• Design and run tests for data validation, storage, and retrieval using Azure services like Data Lake, Synapse, and Data Factory, adhering to industry standards.
• Continuously enhance automated tests as new features are developed, ensuring participation in data reconciliation and verifying data quality frameworks to maintain data accuracy, completeness, and consistency across the platform.
• Formulate and maintain test strategies—including smoke, performance, functional, and regression testing—to ensure data processing and ETL jobs meet requirements.
• Collaborate with development teams to assess changes in data workflows and update test cases to preserve data integrity.
• Share knowledge and best practices by collaborating with business analysts and technology teams to document testing processes and findings.
• Communicate testing progress effectively with stakeholders, highlighting issues or blockers, and ensuring alignment with business objectives.
• Maintain a comprehensive understanding of the Azure Data Lake platform's data landscape to ensure thorough testing coverage.

Requirements

#J-18808-Ljbffr