Principal Data Engineer – Data Quality & Governance

We are seeking an experienced Azure Data Engineer(5+ years) with strong expertise in data warehousing on lakehouse architecture, data engineering, data quality, and governance. The ideal candidate will design and build scalable data pipelines on Microsoft Fabric, implement data quality frameworks, and operationalize governance processes using Microsoft Purview. This role combines technical delivery with data compliance, ensuring trusted and well-governed data across the organization.

Key Responsibilities

Design and develop

batch data pipelines

using

Fabric Pipelines and PySpark .

Implement

data quality rules , validation checks, and monitoring dashboards.

Develop

data catalog, lineage, classification, and business glossary

in

Microsoft Purview .

Enforce

data governance policies, standards, and best practices

across Microsoft Fabric

Collaborate with DevOps to automate deployments using

Azure DevOps .

Integrate data quality in

CI / CD pipelines

through

Azure DevOps .

Ensure compliance with

data privacy regulations

(e.g., PDPL).

Partner with business and data stewards to improve overall data integrity and usability.

Required Skills & Experience

5+ years

of experience in data engineering, quality, and governance roles.

Strong hands-on experience with

Azure Data Factory ,

Apache Spark (PySpark) , and

Azure Synapse / Fabric .

Proven expertise in

Microsoft Purview

for governance and catalog management.

Proficient in

Python

and

SQL

for data processing and validation.

Understanding of

data quality dimensions , profiling, and cleansing techniques.

Strong analytical, problem-solving, and communication skills.

Experience with

Great Expectations

or similar data validation tools is a plus.

Preferred Qualifications

Microsoft Certified : Fabric Data Engineer Associate

or equivalent certification.

Experience with

Microsoft Fabric

and

AI-driven metadata or quality tools .

We have an amazing team of 700+ individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001 : 2015, 27001 : 2013 & 20000-1 : 2018 certified. We have a vibrant culture of learning via collaboration and making the workplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.

To know more about Confiz Limited, visit :