Maithil Deore

Data Engineer with 3+ years of production experience designing, building, and optimising large-scale batch and streaming data pipelines using Python, PySpark, and SQL across BFSI and Telecom domains. Led a Databricks/Delta Lake POC migrating two legacy Hive pipelines - validated a 20% query speedup and zero-data-loss ACID upserts, adopted into the team's lakehouse migration roadmap. Cut SLA-breaching Spark job runtimes by 25% on a 100 TB+ Hadoop/HDFS platform, contributing $75K+/yr in infrastructure savings.

Built a Python data-quality framework (schema validation, null-rate thresholds, row-count reconciliation, alerting) that consistently catches upstream failures before they reach BI/reporting layers. Working exposure to Microsoft Azure alongside primary AWS expertise; version control and code review discipline via Git. Comfortable across the full SDLC in Agile teams, from requirements gathering with stakeholders through development, testing, and deployment.

Complementary hands-on exploration of GenAI/RAG pipeline design and applied AI engineering. AWS Cloud Practitioner certified.

Data Engineer - Kyndryl - Pune, India

(2023-08 - 2024-09)

Client: Vodafone Idea

• Led a Databricks/Delta Lake POC migrating two legacy Hive pipelines: validated a further 20% query speedup and zero-data-loss ACID upserts - a proof of value adopted into the team's lakehouse migration roadmap.
• Diagnosed three SLA-breaching PySpark jobs and rewrote them with partition-pruned reads, broadcast joins, and predicate pushdown - a 25% runtime reduction and lower CPU-hours, contributing $75K+/yr in infrastructure savings.
• Delivered 10+ end-to-end PySpark pipelines (Kafka ingestion -> transform -> Hive, Airflow-scheduled) serving 500K+ subscribers at 5M+ records/month; pipeline design docs were adopted as team-wide standards.
• Built a Python data-quality framework (schema validation, null-rate thresholds, row-count reconciliation, Slack alerting) that caught three upstream feed breaks before BI impact and cut mean time to detect from days to minutes.
• Restructured Hive tables to ORC format with date partitioning and bucketing, reducing worst-case BI query time from 8+ minutes to under 10 seconds.

Data Engineer / Backend Developer - Businessnext - Mumbai, India

(2024-10)

Clients: Bank of India & HDFC Sales

• Engaged directly with business stakeholders to elicit requirements and translate them into ETL components reconciling 2M+ financial records/month; added schema-validation checkpoints and row-count reconciliation that surfaced silent upstream data corruption before it reached BI dashboards, eliminating a recurring class of P1 incidents.
• Introduced Great Expectations checks on inbound feeds, reducing downstream data-quality incidents by 60% within 90 days of rollout.
• Developed and tested RESTful data-access APIs (FastAPI) integrated with the company's proprietary CRM platform and backed by SQL Server, PostgreSQL, and Oracle - handling 1,000+ API calls/day at sub-200ms p99 latency, tested and documented via Postman; diagnosed and resolved connection-pool exhaustion under peak load, improving service reliability by 40%.
• Mentored a junior engineer on ETL design patterns and SQL optimisation, reducing their onboarding ramp by roughly three weeks.

Data Science Intern - TCR Innovation - Mumbai, India

(2021-10 - 2022-03)

• Built ML classification models for HR attrition and heart-disease prediction (Scikit-learn, Python) on 3K-5K-record datasets, reaching 87% accuracy via feature engineering and cross-validation.
• Co-authored NLP research on disease prediction from lab reports, published in the International Journal of Innovative Research in Engineering.

B.E. - Information Technology - Savitribai Phule Pune University (2019 - 2023)