Vadym S., Data Engineer
B1 (Intermediate) English
Middle (3-5 years)
Summary
- 4+ years of experience as a Data Engineer, focused on ETL automation, data pipeline development, and optimization;
- Strong skills in SQL, DBT, Airflow (Python), and experience with SAS, PostgreSQL, and BigQuery for building and optimizing ETL processes;
- Experience working with Google Cloud (GCP) and AWS: utilizing GCP Storage, Pub/Sub, BigQuery, AWS S3, Glue, and Lambda for data processing and storage;
- Built and automated ETL processes using DBT Cloud, integrated external APIs, and managed microservice deployments;
- Optimized SDKs for data collection and transmission through Google Cloud Pub/Sub, used MongoDB for storing unstructured data;
- Designed data pipelines for e-commerce: orchestrated complex processes with Druid, MinIO, Superset, and AWS for data analytics and processing;
- Worked with big data and stream processing: using Apache Spark, Kafka, and Databricks for efficient transformation and analysis;
- Amazon sales forecasting using ClickHouse, Vertex AI, integrated analytical models into business processes;
- Experience in Data Lake migration and optimization of data storage, deploying cloud infrastructure and serverless solutions on AWS Lambda, Glue, and S3.
Main Skills
AI & Machine Learning
Programming Languages
C++ Libraries and Tools
Mobile Frameworks and Libraries
Python Libraries and Tools
Data Analysis and Visualization Technologies
Databases & Management Systems / ORM
Amazon Web Services
Azure Cloud Services
SDK / API and Integrations
Virtualization, Containers and Orchestration
Other Technical Skills
Work Experience
Data Engineer, NDA
(July 2021 - Present)
Data Engineer, ETL Automation
Summary: Building and automating ETL data pipelines with a focus on optimizing PostgreSQL models in the DBT cloud, integrating with third-party APIs using Python, and refactoring Zeppelin notebooks.
Responsibilities: ETL Automation, designing and implementing storage systems, managing API integrations, developing PostgreSQL models in DBT Cloud, establishing microservice deployment jobs, and refactoring notebooks.
Technologies: Python, PySpark, Zeppelin, Docker, AirFlow, Kubernetes, MiniKube, S3, Athena, ECR, PubSub, DBT Cloud, Airbyte, API, BigQuery, PostgreSQL, HiveDB, GitHub, GitLab, Miro, Jira, Teams
Data Engineer, Analytic Platform
Summary: Advanced SDK development to optimize data reception from API endpoints, transformation into special format events, and efficient transmission to Google Cloud Pub/Sub, incorporating MongoDB and Google Cloud storage.
Responsibilities: Optimizing SDK data reception, transforming data into event formats, data transmission with Pub/Sub, integrating MongoDB, and using Google Cloud storage.
Technologies: Python, GCP storage, PubSub, API, MongoDB, GitHub, BitWarden, Jira, Confluence
Data Engineer, E-commerce platform
Summary: Orchestration of complex data pipeline for an e-commerce platform, focusing on data transfer, ingest, processing, and optimization using Airflow, Druid, Minio, and Superset for visualization, and building architectures with AWS.
Responsibilities: Data pipeline orchestration, architectural planning and visualization, workflow optimization, data processing with Spark and Kafka, and implementation with AWS services.
Technologies: Python, Airflow, Druid, Minio, MongoDB, Spark, Kafka, AppFlow, Glue, Athena, Quicksight, PostgreSQL, GitLab, Superset, InSight, Draw.io, Jira, and Confluence
Data Engineer, Retail Platform
Summary: Developed a technical pipeline for a retail platform, emphasizing economic efficiency, integrating key technologies like AWS, Firebase, and Stripe, and utilizing no-code solutions with Xano.
Responsibilities: Technical pipeline development, data transfer optimization, authenticating users with Firebase, payment integration with Stripe, enhancing data processing with AWS IoT, and utilizing Xano's no-code solution.
Technologies: Python, AWS, Xano, Firebase, API, Stripe
Data Scientist, E-commerce Analytic Platform
Summary: Big data analysis and sales forecasting for Amazon product sales, utilizing advanced statistical and programming skills.
Responsibilities: Collecting historical data, preparing sales forecasts, big data analysis, and predictive modeling.
Technologies: Sales Prediction, ClickHouse, Vertex AI, AirFlow, Jenkins, Kibana logs, Keepa
Data Engineer, Simulation, and Automation Worker traffic
Summary: Created a simulation and automation framework for worker traffic, automated EC2 instance management, and deployed solutions using containerization and AWS cloud services.
Responsibilities: Developing simulation and automation framework, managing EC2 instances, and deploying containerized solutions.
Technologies: Python, EC2, ECR, Docker, Windows
Data Engineer, Hotel & Restaurant
Summary: Led the optimization of cloud infrastructure for the hotel industry using AWS services, improving performance, scalability, and cost-effectiveness.
Responsibilities: Cloud infrastructure review and optimization, code refactoring, enhancement of AWS services, and pipeline setup for room price prediction.
Technologies: AWS Lambda, Glue, ECR, DMS, EventBridge, SNS, API Gateway, S3, Python
Data Engineer / Big Data Engineer, Scalable ETL Pipeline with Databricks and AWS
Summary: Designed and implemented an ETL pipeline with Databricks and AWS, processing large-scale data with Apache Spark and integrating with AWS services for schema management, data governance, and real-time processing.
Responsibilities: Designing and implementing end-to-end ETL pipeline, data transformation, and cleaning, metadata and schema management, querying and dashboard integration, and maintaining cost efficiency.
Technologies: Databricks, AWS (S3, Glue, Lambda, EMR), Apache Spark, Delta Lake, Python, SQL
Education
Bachelor in Software Engineer
West Ukrainian National University (WUNU) is a classical university of Ternopil, a leading modern education institution.
2020 - Present
Certification
- Programming for Everybody (Getting Started with Python)
Coursera certificate
- What is Data Science?
Coursera certificate
- Introduction to Data Science in Python
Coursera certificate
- Applied Machine Learning in Python
Coursera certificate
- Amazinum Data Science Camp
Amazinum certificate
- Machine learning with Python
Coursera certificate
- Google Cloud Big Data and Machine Learning Fundamentals
Coursera certificate