Christian Yamepi

• • End-to-End Pipeline Enhancement: Diagnosed and resolved critical data ingestion issues directly within the SQL Server source system. This involved modifying complex stored procedures and functions to correct synchronization logic across different countries, ensuring data consistency before the migration process began.

• • Developed Complex PostMapper Scripts: Authored and optimized several advanced SQL PostMapper scripts for Vertica. These scripts were crucial for handling transformations where direct mapping was insufficient, such as reconstructing missing fields, consolidating data from multiple sources, and pivoting records after the main ETL process.

• • Built a Comprehensive Data Validation Framework: Architected and implemented a suite of automated PySpark and SQL scripts to perform exhaustive, column-by-column validation for over 70 complex tables (SINGLE_VALUE and MULTI_VALUE), ensuring data integrity between production and test environments.

• • Deep Pipeline Analysis & Data Quality Assurance: Analyzed the end-to end data flow, including SQL Server CDC mechanisms and Databricks transformation logic. Identified, diagnosed, and documented critical migration issues such as data fragmentation, scope discrepancies (missing vs. extra records), NULL handling inconsistencies, and data type mismatches.

• • Process Optimization & Automation: Proactively developed and proposed an automated maintenance job to purge stale data from Databricks staging tables, aiming to improve pipeline performance, reduce costs, and simplify long-term maintenance.

As Data Engineer, I architected and built a new big data processing solution from the ground up for Mastercard. The platform, developed in Python and PySpark , was designed to provide Small and Medium-sized Enterprises (SMEs) with insights into their financial transaction flows. I was responsible for designing scalable ETL pipelines, optimizing for performance and reliability. The project was managed in a full Agile/DevOps environment, leveraging Jira for progress tracking and Azure DevOps for CI/CD and code repository management. Performance & Reliability Engineering: Focused on optimizing Spark jobs for big data volumes and near real-time processing requirements, significantly enhancing pipeline performance and stability. Established Best Practices: Implemented a robust development lifecycle by incorporating unit testing for PySpark code and managing the entire codebase through Azure DevOps for version control and CI/CD. Agile Project Management: Utilized Jira to track development progress and manage tasks within an Agile framework, ensuring project milestones were met on schedule.

As data engineer I design and ensure the maintenance of pipellines mainly based on airflow to meet the needs of CRM teams