Amandeep Singh
Data Engineer/Analyst
Experience
Data Engineer/Analyst
Advance Delivery Consulting
- Phase 1 of the migration project involved archiving and purging data from the HAMBS database (SQL Server) based on membership and claims cutoff dates to comply with Australian GDPR requirements. Created SQL scripts for archiving and purging data.
- Phase 2 was migrating data from the HAMBS database to the CIVICA database (both SQL Server).
- Analyzed the HAMBS SQL Server tables for data discovery and to identify potential data quality issues before migration.
- Mapped data lineage between CIVICA and HAMBS tables and worked with business analysts to prepare mapping sheets.
- Created and executed SSIS ETL packages to migrate data from HAMBS to CIVICA tables.
- Optimized SQL query performance.
- Supported and fixed Jira tickets logged by business users after data migration.
Onshore ETL Developer
Infosys Technologies
- Worked with business analysts and designers to understand requirements and build ETL pipelines in Azure Data Factory and Python.
- Created mapping data flows and triggers in Azure Data Factory to schedule pipelines.
- Integrated and transformed data from various sources and loaded it into Azure SQL Database.
- Optimized pipeline performance.
- Built tools for promoting code to higher environments and supported production incidents and defects.
- Deployed code to higher environments.
- Ran a proof of concept to migrate existing applications to Azure Cloud by building data pipelines in Azure Data Factory.
- Set up self-hosted integration runtime services to move data between Azure cloud and on-premises databases.
- Implemented CI/CD pipelines in Azure DevOps.
Offshore – Informatica BDM Lead Developer
Infosys Technologies
- Designed data models in SAP PowerDesigner based on business requirements.
- Developed ETL mappings, workflows, and applications in Informatica BDM.
- Deployed data model DDLs in Hive.
- Built data audit framework scripts for deployment to higher environments and for scheduling batches.
- Performed unit testing, created test documentation for ETLs, and worked with testers on ELT FAT fixes.
- Ran a proof of concept to test various table file and compression formats in Hive for performance optimization.
- Converted legacy ETL mappings from the native Informatica engine to run on the Hadoop Spark engine to improve performance.
- Tuned and optimized performance.
Offshore – SQL and SSIS Developer, Data Analyst
Infosys Technologies
- Phase 1 of the migration project involved archiving and purging data from the HAMBS database (SQL Server) based on membership and claims cutoff dates to meet Australian GDPR requirements. Created SQL scripts for archiving and purging.
- Phase 2 was migrating data from the HAMBS database to the CIVICA database (both SQL Server).
- Analyzed HAMBS SQL Server tables to discover data and spot potential quality issues before migration.
- Mapped data lineage between CIVICA and HAMBS tables and worked with business analysts to create mapping sheets.
- Built and executed SSIS ETL packages to move data from HAMBS to CIVICA tables.
- Optimized SQL query performance.
- Supported and fixed issues logged by application users after migration.
Informatica Offshore Lead
Infosys Technologies
- Developed ETL mappings, sessions, and workflows in Informatica PowerCenter for CIDA data loads to feed the data warehouse on Oracle.
- Worked closely with data architects on design.
- Tuned and optimized performance.
- Ran unit testing and prepared test cases and test data.
- Added new data sources and business logic to existing Informatica mappings.
- Handled and resolved defects tracked in Quality Center.
- Responded to business queries and performed data analysis.
- Led a team of five offshore Informatica developers.
- Ran a proof of concept for Informatica BDM as part of CBL's migration strategy to Informatica 10.1/BDM.
Informatica and SAP HANA Developer
Infosys Technologies
Worked on change requests related to Informatica ETL for populating sales, order, and inventory reports.
Handled business queries and incidents opened by end users.
Migrated existing reports from Informatica to SAP HANA by rewriting code and implementing existing logic in SAP HANA.
Created ETL mappings and workflows, and developed equivalent information models in SAP HANA.
Scheduled Informatica jobs for batch loads.
Performed performance tuning and code push-down to source and target databases (including Teradata).
Created and maintained aggregated tables in Teradata, which were populated through ETL and formed the basis for final reports.
SQL Developer
Infosys Technologies
Worked on production incidents and change requests related to Nationwide’s SQL Server–based CAW reporting platform.
Monitored and supported daily batch loads.
Analyzed data and responded to business queries.
Created SQL stored procedures to populate work and final tables.
Supported deployment activities.
Analyzed logged data defects in Quality Centre.
Scheduled daily batch loads and created DDL for tables in SQL Server.
Summary
Over 13 years of experience in data engineering and ETL, with strong hands-on experience in Informatica BDM, Azure Data Factory, and SSIS.
Involved in various parts of data warehousing and data lake development, including gathering requirements, design, development, testing, deployment, and maintenance.
Solid understanding and experience in the full lifecycle of data warehousing and data lake projects. End-to-end data engineering and data architecture management.
Building data-driven ETL pipelines and data flows on Azure Cloud. Good understanding of Azure DevOps and CI/CD pipelines.
Experience working in Agile projects and big data environments.
Strong and extensive experience working with SQL and writing SQL queries.
Skills
- Etl - Informatica Bdm, Informatica Powercenter, Ssis, Azure Data Factory
- Sql Databases - Sql Server, Oracle, Teradata, Sap Hana, Informix Db
- Nosql Databases - Mongodb
- Cloud Technologies - Microsoft Azure Cloud (Azure Sql Database, Azure Data Lake, Azure Data Factory, Azure Databricks, Azure Devops, Azure Synapse)
- Strong Sql And Advanced Sql Knowledge
- Data Visualization - Power Bi, Sap Business Objects
- Programming Languages - Python (With Pyspark)
- Others - Hive, Jira, Confluence, Unix
Languages
Education
Panjab University
Bachelor of Engineering, Computer Science and Engineering · Computer Science and Engineering · Chandigarh, India
Certifications & licenses
Infosys Certified Agile Developer
Microsoft Certified: Azure Data Engineer Associate (DP-203)
Microsoft Certified: Power BI Data Analyst Associate (PL-300)
Similar Freelancers
Discover other experts with similar qualifications and experience
Senior AWS Data Engineer
Enhancement and Enrichment Database Manager - IT
Principal Consultant
Global Data Analytics & Business Intelligence Consultant
Full-Stack AI and Product Consultant
Senior Data Scientist
SAC Architect
Microsoft BI Specialist
Data Engineer and Founder
Owner / Founder, IT Consultant
Senior Security Technologist
SSIS Development
Senior Software Engineer
AI Agent Evaluation Specialist
Senior Data Engineer
Microsoft Fabric & Power BI Developer
MLOps (machine learning operations)
Workshop Leader 'Introduction to AI Development Tools'
Senior Data Architect & Data Engineer
Data-Scientist/AI Engineer
ServiceNow Consultant
Quality Assurance Manager (QSV)
BI consultant
Senior Cloud Consultant (AWS Services and Consulting)
Founder and CEO
Senior Manager, Strategy & Analytics
Business Analyst, Data Warehouse Developer
Solution Architect
Microsoft Azure Senior Data Engineer / Senior Data Scientist
