Guilherme M.
Data Engineer
Experience
Jun 2024 - Present
1 year 7 monthsData Engineer
Globo
- Design and implement data solutions primarily using Google Cloud Platform (GCP), focusing on Dataform for data transformation.
- Develop and maintain cloud functions for various automation tasks, utilizing Python for backend processes.
- Create and manage data storage solutions including sinks and buckets for efficient data retrieval and processing.
- Utilize Terraform for infrastructure as code, ensuring scalable and reliable deployment of resources.
- Implement CI/CD pipelines using GitLab CI, incorporating automated testing in both CI processes and Dataform.
- Monitor data flows from multiple sources, configuring alarms to ensure system reliability and performance.
- Automate permission management to streamline access controls across various data products.
- Collaborate with cross-functional teams in a data products creation environment, leveraging data mesh principles to enhance data accessibility and usability.
Jan 2023 - Dec 2023
1 yearData Engineer
Maxxidata - Bayer
- Developed new pipelines using AWS Lambda and Docker image, ensuring efficient data processing.
- Designed and implemented data warehousing solutions in AWS S3, enabling scalable data storage and retrieval.
- Utilized Terraform for infrastructure maintenance, optimizing cloud resources and cost management.
- Leveraged BigQuery and Apache Airflow for ETL and data orchestration tasks in AWS EMR (Spark).
- Utilized Datadog for AWS infrastructure monitoring.
Jun 2022 - Dec 2022
7 monthsData Engineer
EVT - BRK
- Created ETL workflows in Azure Databricks, facilitating data manipulation and transformation.
- Developed custom macros to import Oracle data into a new data warehouse in Databricks.
- Implemented best practices and data patterns in every data operation for enhanced efficiency.
- Collaborated with the team to ensure seamless integration with Apache Airflow for workflow management.
Summary
Results-driven Data Engineer with extensive experience in cloud technologies, including AWS, Google Cloud Platform (GCP), and Azure. Proficient in designing and implementing ETL workflows using tools like Apache Airflow and Dataform, alongside strong skills in big data processing with Apache Spark. Adept at utilizing Infrastructure as Code (IaC) with Terraform for efficient resource management. Demonstrated ability to deliver impactful data solutions in collaborative team settings, enhancing data accessibility and usability across organizations.
Languages
Portuguese
NativeEnglish
AdvancedSpanish
Intermediate Need a freelancer? Find your match in seconds.
Try FRATCH GPT More actions
Similar Freelancers
Discover other experts with similar qualifications and experience
- English
- Deutsch
2025 © FRATCH.IO GmbH. All rights reserved.