Siddharth N.

Principal Consultant

New Delhi, India

Experience

Jul 2021 - Present
4 years 6 months

Principal Consultant

StatusNeo Consulting

  • Consulted with CXO-level stakeholders on data strategy, architecture reviews, and modernization roadmaps across Hospitality, Finance, Retail, Sports, and Telecom.
  • Led an 8-member Data, Analytics & AI team delivering large-scale data engineering and warehousing solutions, mentoring juniors, and collaborating with cross-functional teams for seamless delivery.
  • Designed and delivered an end-to-end Medallion Architecture on Microsoft Fabric integrating Opera APIs, ReviewPro, SFTP, and on-prem sources.
  • Implemented Direct Lake models in Power BI, reducing dashboard load times from 1+ min to <10 sec.
  • Launched an AI-powered Data Agent enabling conversational queries for CXOs.
  • Enforced RBAC, governance, and lineage to ensure compliance and security.
  • Migrated pipelines from Azure Data Factory to Databricks, orchestrating transformations with dbt and Delta Lake.
  • Integrated Salesforce and RDBMS sources using reusable ETL frameworks with SCD1 and SCD2 handling.
  • Applied DataOps practices including CI/CD, monitoring, and quality checks for reliability.
  • Successfully decommissioned the legacy data warehouse, cutting daily costs from 250 to 50 USD through cost optimization.
  • Automated P&L reporting with Alteryx workflows and Power BI dashboards, cutting manual effort by 80%.
  • Designed advanced transformations with Crew Macros, improving reporting accuracy.
  • Established DataOps pipelines for automated validation and CI/CD.
  • Built a real-time Cricket Data Analytics Platform ingesting high-frequency JS events into Cassandra.
  • Developed Flask/OpenAPI microservices deployed on AKS for web and mobile APIs.
  • Delivered real-time streaming and batch processing pipelines balancing low latency and scalability.
  • Enabled instant match and event updates via FCM push notifications with SSL pinning and parameterized queries for security.
  • Built preprocessing pipelines for catalog data, establishing Elasticsearch indexes and a Redis serving layer.
  • Developed FastAPI suggest APIs for high-performance search and deployed them as microservices on AKS with CI/CD.
Nov 2020 - Jul 2021
9 months

Consultant

Deloitte USI

  • Designed and deployed scalable ETL pipelines using Azure Data Factory and Databricks, reducing ETL runtime by 50%.
  • Built reusable dimension and fact frameworks with SCD1 and SCD2 handling on Delta Lake, improving maintainability and historical accuracy.
  • Implemented dynamic data masking in Azure Synapse to protect PII and ensure compliance with regulatory requirements.
  • Partnered with cross-functional business teams to translate requirements into scalable technical solutions, improving reporting efficiency and data trust.
Jan 2020 - Nov 2020
11 months

Big Data Engineer

Reddoorz

  • Built AWS ETL pipelines with Apache NiFi, Step Functions, and Redshift, optimizing SQL queries and reducing load times by 40%.
  • Developed Python-based validation and SCD frameworks for Redshift, with monitoring and orchestration automated via CloudWatch and Lambda.
  • Replaced legacy reporting tool “Holistic” by implementing automated pipelines, reducing CXO report generation from hours to minutes.
  • Improved data quality and consistency across pipeline outputs, ensuring reliable executive-level reporting.
Jul 2017 - Dec 2019
2 years 6 months

Technical Associate

Genpact India Pvt. Ltd.

  • Developed reusable ETL and validation frameworks in Python and Spark with SCD1 and SCD2 logic, improving data consistency across ingestion jobs.
  • Automated large-scale batch processing pipelines (>100 TB) using AWS Data Pipeline, Step Functions, Lambda, CloudWatch, and EMR, reducing manual oversight and improving reliability.
  • Optimized Amazon Redshift schema design and queries, improving performance and reducing storage and compute costs.
  • Implemented monitoring and alerting using CloudWatch and integrated workflows with CloudFormation for better orchestration.
  • Partnered with onshore and offshore teams to deliver data migration and modernization projects under strict SLAs.

Summary

Accomplished Data Architect with 8+ years of experience helping organizations transform their data into a strategic asset. Proven ability to bridge business and technology by consulting with stakeholders, shaping enterprise architectures, and delivering modern cloud-native platforms. Skilled at leading high-performing teams and guiding clients across industries including hospitality, finance, telecom, sports, power & energy and retail.

Hands-on with AWS, Azure, Microsoft Fabric, and GCP, bringing a holistic approach to data engineering, governance, and analytics that drives innovation and competitive advantage.

Languages

Hindi
Native
English
Advanced
German
Elementary

Education

Oct 2013 - Jun 2017

AKTU

B.Tech, Computer Science Engineering · Computer Science Engineering · Lucknow, India

Need a freelancer? Find your match in seconds.
Try FRATCH GPT
More actions