Hamid A.

Project:

Working on a centralized data management platform to integrate, normalize,

and deliver financial data (FX, Fixed Income) from multiple internal and external

sources (CSE, CSU, Bonds, etc.)

Key Responsibilities & Technical Tasks:

• Developed and maintained data ingestion pipelines in batch and streaming

modes using Apache Spark on Azure HDInsight (HDI).

• Integrated complex financial datasets (FX, FI) from various sources ensuring

high data quality and consistency.

• Designed and implemented data normalization and enrichment processes to

provide standardized data models to business users.

• Migrated existing Spark jobs from HDInsight (HDI) to SOK8s (Spark on

Kubernetes) running on Azure Kubernetes Service (AKS) to enhance scalability

and reduce costs.

• Automated the entire data processing pipeline, including deployment

(CI/CD), monitoring, and error handling.

• Developing Airflow DAGs to schedule and manage data workflows.

• Worked closely with data consumers to understand requirements and ensure

the delivery of trusted, high-performance data services.

• Participated in architecture design and performance tuning of Spark jobs and

Kubernetes clusters.

Technologies

AKS (Kubernetes on Azure), Spark on AKS (SOK8s), HDInsight, Azure Storage

Account, Azure Synapse Analytics, Azure Key Vault, Azure Cosmos DB, Apache

Spark (Batch & Streaming mode), Apache Kafka, Parquet, Java, Python, Apache

Airflow, Git, Harbor, ArgoCD, Jenkins, JFrog Artifactory, SonarQube

Project:

- Implementation of market reserves computation

- Developed a tool enabling business teams to simulate reserve calculations with

customized data (shocks, sensitivities).

Achievements :

• Designed and set up the project's architecture.

• Developed reserve calculation jobs using Apache Spark.

• Implemented of BDD/TDD using Cucmber

• Developed Backend Services and Controllers (SpringBoot)

• Implemented CI/CD pipeline.

Technologies

Apache Spark, Scala, Java, Hadoop, Kafka, Hive, Angular, SpringBoot, CI/CD,

Maven, Bitbucket, Jenkins, XLdeploy, XLRelease, Unix, SQL, TDD/BDD.

Project:

- Implementation of an accounting and tax audit platform for Financial

Accounting Files under Azure.

https://www.pwcavocats.com/fr/nos-expertises/gestion-et-strategie-fiscale-de-lentreprise/fec-quatre-point-zero.html

Achievements :

• Analyzed client requirements and proposed technical solutions.

• Designed and implemented the application architecture.

• Configured infrastructure environments in collaboration with PwC Cloud team.

• Implemented over 150 business rules using Spark Java API (Spark Core, Spark

SQL).

• Developed an API for communication with the Azure Data Lake Store.

• Implemented backend controllers and services.

• Generated advanced PDF reports.

• Performed unit testing to ensure coverage.

Technologies

Big Data, Hadoop, Spark Java API (Spark core, Spark SQL), AzureHDInsight, Azure Data Lake Store, App service, Apache Livy, Yarn, Java 8, SpringBoot, Spring MVC,SpringIoC, SpringData, SQL Server, Azure Data Lake Store SDK, Xml Apache Graphics (Xml, xsl-fo), Angular 6,Git, Maven, Postman.