Job Title: Microsoft Fabric Data Engineer
Location: Remote (Milestone-based travel required)
Duration: 12 Months 

Position Overview

We are seeking a Senior Data Engineer to join the Americas IT Software and Architecture team. This business-focused IT organization enables the development of innovative technology solutions and integrates global and regional IT initiatives into business operations.

The ideal candidate will bring strong hands-on expertise in Apache Spark, Microsoft Fabric, and cloud-scale data architecture. In this role, you will design and implement scalable data pipelines, optimize large-scale data architectures, and enable data-driven decision-making across the organization.

This position requires the ability to work independently, collaborate with business and IT leaders, mentor junior engineers, and architect data solutions based on complex business requirements.

Key Responsibilities

Enterprise Data Pipeline Development

• Design, build, and maintain scalable and distributed data pipelines using Apache Spark and Microsoft Fabric to process large structured and unstructured datasets.

Data Integration

• Integrate data from various internal and external systems while ensuring reliability, consistency, and proper data lineage across the enterprise.

Performance Optimization

• Optimize ETL/ELT workloads to improve cost efficiency, throughput, and reliability for large-scale analytics environments.

Data Quality & Governance

• Implement and enforce standards for data quality, metadata management, data cataloging, lineage tracking, and governance compliance.

Cross-Functional Collaboration

• Work closely with data scientists, analysts, architects, and IT teams to gather requirements, deliver insights, and integrate analytical models.

Documentation

• Create and maintain comprehensive documentation for pipeline architecture, workflows, schemas, and operational processes.

Technology Innovation

• Evaluate emerging technologies and introduce modern data engineering practices including Lakehouse architecture, Delta formats, automation, and real-time data processing.

Troubleshooting & Reliability

• Diagnose and resolve complex data pipeline issues to ensure platform stability and enterprise-grade reliability.

Security & Compliance

• Enforce enterprise data security, privacy, and access governance policies.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related field

• 7+ years of experience in data engineering, data architecture, or large-scale data platform development

• Strong expertise in Apache Spark for batch and streaming data processing

• Hands-on experience with Microsoft Fabric, including:

  • Data Engineering

  • Data Factory

  • Data Pipelines

  • Lakehouse implementations

• Advanced proficiency in SQL, Python, and/or Scala

• Experience with cloud platforms such as Microsoft Azure (preferred), AWS, or GCP

• Strong understanding of distributed systems, lakehouse architecture, and data modeling

• Proven experience designing and optimizing complex ETL/ELT pipelines

• Strong communication, leadership, and mentoring skills

Preferred Qualifications

• Certifications in Azure Data Engineering, Apache Spark, or Microsoft Fabric

• Experience with real-time streaming technologies such as Kafka or Azure Event Hubs

• Knowledge of DevOps practices including CI/CD, Infrastructure as Code, and automated testing

• Experience with data visualization tools such as Power BI or Tableau