Big Data Specialist

Senior Big Data SME (Subject Matter Expert)

Location : Remote (or specify your office location)

About Us : We are a dynamic team focused on building scalable, high-performance data platforms across the cloud ecosystem (Azure, AWS, GCP). As part of an exciting growth phase, we are looking for a Senior Big Data SME to join our team. This role will allow you to leverage your expertise in cloud technologies, big data, and data engineering to drive innovation and excellence in data architecture and solutions.

Job Description

We are seeking a Senior Big Data Subject Matter Expert (SME) to help us architect, build, and optimize large-scale data platforms using leading cloud technologies and Big Data tools. As an SME, you will bring deep technical expertise and strategic vision to our cloud data initiatives, guiding the team on best practices, ensuring the successful delivery of data projects, and improving operational efficiency. You’ll work closely with engineers, architects, and leadership to ensure the implementation of cutting-edge solutions.

Key Responsibilities

• Architect and Design Scalable Data Platforms : Leverage your expertise in Azure, AWS, and GCP to design and build scalable, high-performance data platforms for analytics and real-time processing.
• Cloud Technologies Expertise : Drive the adoption of cloud-native tools and services like Azure Synapse, AWS Glue, Redshift, and Google BigQuery, and ensure alignment with industry best practices.
• Big Data & Streaming Solutions : Provide hands‑on support and guidance on big data technologies including Apache Spark (PySpark, Scala, Java), Hadoop (HDFS, YARN, Hive, Impala), and streaming frameworks like Kafka, Kinesis, and Flink.
• Data Pipelines & ETL Frameworks : Lead efforts in creating and optimizing ETL pipelines using tools like DBT, Apache Airflow, and cloud‑native data orchestration tools (e.g., AWS Glue, Azure Data Factory).
• Containerization & Orchestration : Lead efforts in containerizing applications using Docker and orchestrating them with Kubernetes (AKS, EKS, GKE) for scalable and reliable deployment.
• Data Security & Governance : Ensure data security, governance, and compliance using IAM, RBAC, and data lineage tracking tools (Azure Purview, AWS Glue Data Catalog).
• CI / CD & DevOps : Contribute to the development of CI / CD pipelines with Terraform, Jenkins, Azure DevOps, and GitHub Actions to automate the deployment of cloud infrastructure and data workflows.
• Cost Optimization : Implement strategies for optimizing cloud resource utilization and reducing operational costs, particularly in storage (AWS S3, Azure Blob, GCS) and compute resources.
• Team Collaboration & Mentorship : Provide mentorship and guidance to junior engineers, and lead initiatives to build and maintain high‑performing teams through knowledge‑sharing sessions and best practice adoption.

Requirements

#J-18808-Ljbffr