Cloud Architect
Headhuntr.io / TalentScout Inc.
Taguig-BGC • JAN 2020 - DEC 2022 (3 YEARS)
Overview
Architected and built a scalable cloud infrastructure from the ground up. Designed data processing pipelines and automation systems that significantly reduced costs while improving reliability.
Key Achievements
- Setup and automated the provisioning of infrastructure for multiple environments on AWS using Terraform to have a consistent setup across the board
- Automated the data pipeline with AWS Step Functions, reducing manual intervention by 90%
- Configured a local development workflow to run the entire application offline, improving developer productivity
- Established a highly scalable and cost-effective data processing pipeline with Apache Spark deployed on AWS EMR with EC2 Spot Instances, reducing costs by 70%
- Automated the build and deployment pipeline with BitBucket and integrated the entire process with JIRA and MS Teams
- Established best practices for a speedy but sustainable development cadence through the thoughtful application of automation on all aspects of the development pipeline
- Improved cost-effectiveness of the data warehouse solution by introducing a Serverless scheme, reducing monthly costs by 60%
- Mentored peers with a practical approach on test-driven development
Technologies & Tools
Backend/Middleware: Spring Boot, Java, Python, Spring Data JPA, Spring Data Elasticsearch, Liquibase, RabbitMQ, Redis, Netty, Reactor, Webflux, R2DBC, Lombok
AWS Technologies: AWS Glue, Amazon EMR, API Gateway, EKS, Elasticsearch Service, Amazon Aurora(RDS), Redshift, Lambda, S3, Elasticache, AMQ, Step Functions, Redshift Serverless
Testing: Spock, JUnit, Reactor Test, Postman, Sonarqube
Backend/Data Processing: Apache Spark
Automation: BitBucket Pipelines, Terraform, Helm
Runtime/Platform: Kubernetes, Docker
Project Management: JIRA, BitBucket, MS Teams
Concepts: REST, Functional (Reactive) Programming, Test-Driven Development, Domain-Driven Development, Microservices, Object-Oriented Programming, Agile
What I Liked
- Significant creative freedom
- High-trust work environment
- Direct communication channels with upper management
- High performers receive recognition and rewards
- Early schedule (6AM-3PM) provides ample time for activities after work
What Needs to Improve
- Compensation could be more competitive, though acceptable