Manager – Data Engineering | Inspire Brands | Oct’23 – Present

As Manager – Data Engineering, I lead the operations and evolution of Inspire Brands’ Enterprise Data Platform—an ecosystem of 600+ high-stakes pipelines powering transactions, customer intelligence, loyalty, campaigns, and operational insights across the business.

I guide a high-performing team of engineers and own the stability, scalability, and performance of a deeply complex modern data stack spanning ADF, Databricks, Snowflake, Airflow, ADLS, and multi-cloud sources.

My work blends decisive leadership with hands-on technical oversight: critical incident management, platform hardening, architectural modernization, automation, and data quality governance.

This role is where I cement the engineering and operational rigor required to run a large-scale, cloud-native data platform that the business depends on every single day.

Responsibilities

• Operations Management & Incident Resolution

  • Own end-to-end reliability, availability, and performance of pipelines across ADF, Databricks, ADLS, Snowflake, and Airflow.

  • Lead incident triage, RCA, and recovery for high-impact issues; implement resilient response strategies to minimize downtime.

  • Enforce SLA compliance, platform stability, and operational rigor through proactive monitoring and structured escalation paths.

• Technical Support & Troubleshooting

  • Provide L2/L3 support for failures, schema drifts, data discrepancies, and pipeline performance issues.

  • Develop troubleshooting playbooks and collaborate with Infra, DBAs, Cloud, and Security teams to resolve platform-level issues.

  • Optimize resource utilization, job retries, and error-handling patterns to strengthen system resilience.

• Development, Enhancements & Production Fixes

  • Oversee design, build, and deployment of new ADF, Snowflake, Databricks, and Airflow pipelines.

  • Drive production fixes, preventive engineering, code refactoring, and continuous improvement of ETL workflows.

  • Lead RCA-driven enhancements aligned with business priorities and architectural best practices.

• Data Quality & Observability

  • Implement enterprise-grade DQ rules, schema validations, and anomaly detection across ingestion and transformation layers.

  • Build centralized observability dashboards (Power BI) and end-to-end lineage tracking using Airflow and Snowflake.

  • Establish real-time monitoring for job health, SLAs, table loads, and data integrity.

• Collaboration & Stakeholder Management

  • Partner with data scientists, business teams, and engineering groups to translate requirements into scalable solutions.

  • Drive governance, prioritization, and cross-functional alignment on deliverables and platform improvements.

  • Provide transparent communication, operational reporting, and stakeholder updates.

• Process Improvement & Automation

  • Identify and automate high-effort operational tasks to reduce manual load and human error.

  • Implement proactive monitoring, alerting, retry logic, and self-healing patterns for critical workflows.

  • Standardize reusable frameworks for validation, logging, and error handling.

• Team Leadership & People Management

  • Lead and mentor a team of data engineers, ensuring growth, accountability, and high performance.

  • Drive capability development through training, code reviews, and structured coaching.

  • Support hiring, workforce planning, and performance management.

• Technology Adoption & Continuous Learning

  • Stay current with trends in cloud engineering, orchestration, automation, and modern data stack tools.

  • Promote adoption of scalable, future-ready technologies to strengthen platform resilience and team productivity.

Highlights

• Reduced job failure rates through robust retry logic, standardized error-handling patterns, and conditional handling across Airflow & ADF.

• Led migration of ETL workflows from Databricks (PySpark) to Snowflake (SnowSQL), integrating orchestration into Airflow for unified monitoring and retries.

• Designed and deployed centralized Power BI observability dashboards for Airflow DAGs, ADF pipelines, REST APIs, SLAs, and data validations.

• Delivered real-time data lineage and job status visualizations, accelerating issue identification and preventing SLA breaches.

• Standardized DQ checks and validation alerts across pipelines, improving trust and consistency in enterprise data.

• Redesigned alerting logic to eliminate false positives and significantly reduce incident volume.

Takeaways & Learnings

This role made me what I am, I evolved into a modern data platform leader—driving reliability, quality, and innovation across a complex, cloud-scale engineering ecosystem.

• Built deep expertise in operating and scaling a modern, cloud-native data platform across ADF, Snowflake, Databricks, ADLS, and Airflow.

• Strengthened leadership capability by managing high-stakes operations, critical incidents, and cross-team engineering efforts.

• Developed a product-mindset toward pipelines—focusing on reliability, observability, automation, and continuous improvement.

• Improved architectural thinking through large-scale migrations, workflow redesigns, and platform-wide reliability patterns.

• Sharpened decision-making under pressure, balancing business impact with technical constraints.

• Learned to build a high-performing team culture centered on accountability, ownership, and technical excellence.