Microsoft Azure Senior Data Engineer / Senior Data Scientist

• Advising on the use of analytics and BI tools and services in the Microsoft Azure stack (e.g., MS Fabric, Synapse Workspaces and dedicated SQL pools, SQL Database, PostgreSQL, Snowflake, Databricks, Data Factory, SSIS, Analysis Services, Function Apps, Power BI, ML)
• Independently designing analytics solutions using Python, SQL, etc.
• Designing and implementing ETLs and data pipelines
• Creating and maintaining APIs
• Independently applying CI/CD, testing, and version control
• Data modeling
• Model development and optimization
• Anomaly detection with AI
• Predictive analytics

Technologies used:

• Snowflake
• Fabric
• Azure Synapse Analytics
• Azure DataFactory
• Azure Data Lake
• Azure DevOps
• Databricks
• Spark
• CI/CD
• SQL Database
• Python
• Power Platform

• Data governance in the CDP
• Data classification and cataloging (e.g., PII detection)
• Metadata management and data inventory
• Access controls and permissions management (Access Management)
• Ensuring GDPR and data privacy compliance
• Monitoring data quality and integrity
• Conducting audits and lifecycle management (archiving/deletion)
• Planning, organizing, and executing the project
• Integrating data from HTTP clients, Azure Blob Storage, and Azure Data Lake Gen2 with Azure Data Factory
• Branching and chaining activities in Azure Data Factory pipelines using control flow activities
• Implementing data pipelines to extract data from various sources
• Transforming data with ADF
• Designing and implementing ETLs and data pipelines

Technologies used:

• Cloudera
• Hadoop
• Spark
• Airflow
• Hive
• Kafka
• NiFi
• Python
• Azure
• CI/CD
• Power BI
• AWS

• Building an architecture using Databricks, PySpark, and Delta Lake for SAP data
• Integrating data quality rules into Delta Lake via Blob Storage using PySpark
• Developing PySpark scripts in Databricks notebooks to validate SAP data
• Implementing data quality rules with PySpark and Delta Lake
• Troubleshooting data flows with Databricks tools and Spark logs
• Creating Databricks workflows for automated data quality checks
• Monitoring data quality pipelines with alerts in Databricks
• Creating data quality reports with Databricks SQL and Power BI
• Ensuring SAP data quality with Delta Lake and governance reports

Technologies used:

• Snowflake
• Azure
• Databricks
• PySpark
• Delta Lake
• Blob Storage
• Power BI
• CI/CD
• DevOps

• Introduction to machine learning
• Computer-based signal processing
• Mechanics and thermodynamics
• Nuclear astrophysics
• Research and lab internship