Description

I am a seasoned data engineering leader with deep expertise in designing and scaling modern data platforms. He has successfully led initiatives to solve complex data challenges across industries, focusing on data integration, architecture, and quality. With a strong foundation in cloud-native technologies and distributed data systems, I excel at translating business needs into robust technical solutions.

As Co-Founder of Datafarer, I bring strategic hands-on approach to helping clients harness the full potential of their data.

Languages

English
Native or bilingual
Telugu
Native or bilingual

Workplace preferences

Remote only

Primarily works remotely

The Apache Software Foundation
Committer
June 2014 - Today (12 years)
Worked on the initial proposal and release of Pig-on-Spark project, which aims to support Apache Spark as a compute engine for Pig. More details on the project can be found on this umbrella jira - https://issues.apache.org/jira/ browse/PIG-4059
Inflection AI
Senior Data Engineer
May 2022 - October 2022 (5 months)
Hyderabad, Telangana, India
Developed scalable data pipelines using Apache Spark and PySpark to process and deduplicate over 12TB of web archive data stored in AWS S3. Implemented a fuzzy matching algorithm based on Jaccard similarity to identify and remove redundant records, enhancing data quality and reducing bias in large language model training. Deployed on AWS EMR clusters, achieving 80% accuracy in duplicate detection and significantly improving model generalization and computational efficiency.
Amazon
Data Engineer
E-COMMERCE
June 2017 - May 2020 (2 years and 11 months)
Hyderabad, Telangana, India
Worked as a Data Engineer II, as part of the Supply Chain and Optimization Technology(SCOT) org of Amazon.