Hi there, I’m Prajwal Suresh. I’m currently exploring Data Engineering, Deep learning, machine learning technologies. This is where I write about my learning journey and explorations
Azure Data Engineer
Networth Data Products Private Limited, Bengaluru, India
January 2023 - Present
- Served as an Data Engineer, implementing modern data solutions for data extraction, transformation and loading data from source systems into the Azure Data Lake using Azure cloud services.
- Orchestrated end-to-end data pipelines using Azure Data Factory, integrating Azure Data Lake for storage, Azure Databricks for data transformation, and Azure Synapse for serving data to dashboards and downstream teams, reducing manual intervention by 60%
- Migrated legacy data pipelines to Medallion architecture, improving data quality, scalability, processing time and consistency, reducing data ingestion time by 30% and operational costs by 50%
- Implemented incremental data processing using SQL and PySpark on Azure Databricks, reducing storage and compute costs by 20%
- Utilized Delta Lake’s schema evolution capabilities to adapt to changing data, ensuring data pipelines remained scalable and flexible and reducing schema management overhead
- Developed generic pipeline templates to get data from various types of source (API, SQL, Excel, ADLS, CSV, Parquet)
- Collaborated with business stakeholders regularly to gather and refine data requirements, ensuring 100% alignment between data outputs and business objectives.
- Partnered with Power BI developers to translate business requirement into actionable Gold layer datasets, enabling the creation of 20+ interactive dashboards.
Assistant System Engineer
Tata Consultancy Services Private Limited, Bengaluru, India
August 2021 - January 2023
- Developed ETL pipelines in Azure Data Factory, improving data availability by 40%
- Involved in migration of on-premises databases to Azure SQL Database, ensuring data integrity and achieving 20% improvement in query performance.
- Automated data pipeline monitoring and error handling, achieving 99% pipeline success rate
- Developed PySpark and SQL scripts in Azure Databricks to transform data
Programming & Scripting
Python, C++, SQL
Cloud
Microsoft Azure Cloud Platform, Azure Data Factory, Azure Databricks, ADLS Gen 2, Azure Synapse, Azure Logic app, Azure Key Vault
Machine Learning
PyTorch, fastai, Scikit-Learn, Pandas, Numpy, MatplotLib, OpenCV, HuggingFace
Web Development
Django, Flask, HTML, CSS
Sapthagiri College of Engineering, Bengaluru, India
Bachelor of Engineering in Computer Science & Engineering | GPA: 8.55/10
2017-2021
MES PU College of Arts, Commerce & Science, Bengaluru, India
Pre-University Education | Percentage: 87.16%
2017
St Mary’s High School, Bengaluru, India
Tenth Standard | Percentage: 97.6%
2015
