About Lucas
Senior Data Engineer with 10+ years of experience designing and scaling data platforms/architectures in high demand, regulated, international environments (finance, retail, public sector).
I specialize in building robust, production-grade data systems using Databricks and AWS, with a strong focus on performance optimization, developer experience, and platform reliability at scale.
Currently working at Decathlon, I support a data platform used by 3,500+ users (Data Engineers, Analysts, Data Scientists), where I focus on:
- Platform administration, development & FinOps.
- Integrating latest Databricks features at scale.
- Developers tooling & support & standardization.
- Large-scale data pipeline architecture.
I enjoy bridging the gap between complex data engineering challenges and significant business impacts.
Core skills: Databricks, Spark, AWS, Data Architecture, Performance Optimization
French
Native or bilingual
English
Fluent
Remote only
Primarily works remotely
Experience
- DecathlonLead Data EngineerRETAIL (LARGE RETAILERS)January 2023 - Today (3 years and 6 months)Paris, FranceOperating as a Lead Data Engineer of a team of 6 Data Engineers within the central Data Platform team, supporting and scaling a Databricks-based ecosystem on AWS used by 3,500+ data professionals.
- Provide advanced support to Data Engineers, BI Engineers, and Data Scientists (debugging, performance optimization, architecture guidance).
- Contributed to the design, governance, and reliability of the data platform.
- Designed and rolled out reusable project templates (Spark, dbt, Airflow, AWS Lambda, EKS) using Cookiecutter, Cruft, Poetry, GitHub Actions, and SonarCloud, enabling standardized and industrialized data development across teams.
- Improved Databricks infrastructure standardization using Terraform at scale through Hashicorp Entreprise, improving reproducibility, governance and evolution of platform resources.
- Led the technical migration from AWS Glue to Databricks Unity Catalog, improving data governance, security, and cross-team data accessibility.
- Implemented platform monitoring, alerting, and FinOps practices to perfect usage and control costs.
Technical Stack: Databricks, AWS, Apache Spark, dbt, Apache Airflow, AWS Glue, Unity Catalog, Spark Declarative Pipelines, Lakeflow Jobs, AWS EKS, AWS Lambda, Github Actions, Data Contracts. - Autorité des marchés financiers (AMF) – FranceSenior Data EngineerBANKING AND INSURANCEJuly 2021 - January 2023 (1 year and 6 months)Paris, FranceJoined the cross-functional DataFab team within the Data & Market Surveillance Department, offering technical ability to support data analysts and data preparation workflows.
- Improved the computation time of a cumulative hypergeometric distribution using Apache Spark (Scala) from 7h to 45min by forking the class HypergeometricDistribution from Apache Commons Math, significantly improving performance of the pipeline.
- Reduced daily data ingestion time into HBase via Phoenix from 6 hours to 1 hour through performance tuning and process optimization.
- Integrated analyst-developed market surveillance and alerting tools into a unified software framework to enhance maintainability and scalability, including advanced analytics functions (machine learning models), enabling easier adoption by analyst teams.
- Designed and implemented data pipelines for publishing open datasets to Data.gouv, ensuring compliance with open data standards.
- Defined data architecture for integrating annual financial reports of French companies into the AMF data platform.
Technical Stack: Apache Spark, Apache Hive, Hadoop ecosystem, Python, Scala, Java - European Security Market Authority (ESMA)Big Data EngineerBANKING AND INSURANCEApril 2020 - June 2021 (1 year and 2 months)Paris, FranceProvided consulting and optimization ability for data processing workflows related to European banking regulations within the Supervision & Data Analytics Systems team. Operated in an international environment with English as the primary working language.
- Designed and implemented data ingestion and transformation pipelines to convert XML regulatory files into structured CSV formats for EMIR, SFTR, and SECR (European Union financial regulations).
- Defined deployment methodologies and built automated CI/CD pipelines for data processing workflows.
- Developed, supported and monitored automated data pipelines using Talend, MySQL, and TIBCO Spotfire.
- Built a scalable XML-to-CSV mapping framework in Java using Altova MapForce, standardizing transformation logic across regulatory datasets.
- Optimized processing of a 19TB Oracle Database table via partitioning, indexing, and stored procedures, and reduced query latency by implementing caching strategies in TIBCO Data Virtualization.
- Improved performance and reliability of recurring analytical scripts used by ESMA analysts, implementing monitoring and optimization best practices.
Technical Stack: Oracle Database, Talend, Altova MapForce, TIBCO Spotfire, TIBCO Data Virtualization, Python, Java, SQL.
Recommendations
Be the first to recommend Lucas
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- Diplôme d'ingénieur, Technologies de l'informationIMT NORD EUROPE2016Diplôme d'ingénieur, Technologies de l'information
- Diplôme d'ingénieur, Technologies de l'informationСанкт-Петербургский государственный университет2015Diplôme d'ingénieur, Technologies de l'information