Principal Azure Data Engineer

Key Responsibilities

• Strategic Data Architecture:

  • Lead the design and implementation of enterprise-level data architectures using Azure Data services, including Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, and Azure Databricks.
  • Define long-term strategies for data integration, transformation, storage, and analytics to support the organization’s business goals.

• Technical Leadership:

  • Act as the subject matter expert for Azure data solutions, providing guidance and mentorship to technical teams.
  • Collaborate with senior stakeholders, data scientists, and business leaders to understand requirements and translate them into scalable data solutions.
  • Establish best practices and architectural standards for data engineering projects.

• Data Integration & Pipeline Development:

  • Oversee the design and optimization of ETL/ELT processes to seamlessly integrate data from diverse sources.
  • Ensure data quality, reliability, and performance across all platforms while maintaining robust security standards.

• Innovation & Continuous Improvement:

  • Stay current with emerging trends and technologies in data engineering and Azure cloud services, and proactively introduce innovations to improve data workflows.
  • Drive automation initiatives through CI/CD pipelines, advanced monitoring, and performance tuning.

• Project Oversight & Stakeholder Engagement:

  • Manage multiple complex projects from conception through deployment, ensuring timely delivery and adherence to budget constraints.
  • Prepare detailed technical documentation and reports for executive leadership and external stakeholders.
  • Act as a key liaison between business units and IT, ensuring alignment of technology initiatives with strategic objectives.

Required Qualifications

• Experience:

  • Minimum of 10+ years of experience in data engineering, architecture, or a related field, with at least 5 years focused on Azure data services.

• Technical Skills:

  • Expert-level proficiency in Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, and Azure Databricks.
  • Strong programming skills in SQL, Python, or other relevant languages used for data processing and automation.
  • Extensive experience with ETL/ELT, data warehousing, and data modeling.
  • Solid understanding of DevOps practices, CI/CD pipelines, and containerization (Docker, Kubernetes) in a data environment.
  • Experience with data governance, security best practices, and compliance requirements.

• Educational Background:

  • Bachelor’s degree in Computer Science, Information Technology, or a related field; Master’s degree is a plus.
  • Relevant certifications, such as Microsoft Certified: Azure Data Engineer Associate or Microsoft Certified: Azure Solutions Architect Expert, are highly desirable.

• Soft Skills:

  • Exceptional leadership, communication, and stakeholder management abilities.
  • Proven track record in managing and mentoring technical teams.
  • Strong analytical and problem-solving skills with a focus on innovative solutions.
  • Ability to work effectively in a fast-paced, on-site environment and manage multiple priorities.

Preferred Qualifications

• Experience in strategic roles within highly regulated industries (e.g., finance, healthcare, telecommunications).
• Familiarity with additional cloud platforms (AWS, Google Cloud) is an advantage.
• Proven success in large-scale digital transformation projects.