Dmitriy K. — Python Software Engineer

Experience

Data Engineer, Back-end developer, Thrasio

February 2020 – June 2023

Description:  Scraping system of products (products, reviews, comments, user's profiles) data from Amazon marketplaces

Responsibilities:

• Scraping data from Amazon marketplaces, Walmart,
• Development of bots for downloading data from seller dashboards (Amazon Seller Central, Walmart Seller Central, Shopify Dashboards)
• API crawlers to collect data from Rainforest API, Amazon Seller Partner API
• Building microservice platform for pipelines management
• Building pipelines for data transformation/ normalization/ de-duplication in terms of ETL/ELT processes
• filling a data warehouse (Redshift, Snowflake) for use by other services
• Conducting technical interviews for new hires.

Technologies:Python, Apache Airflow, boto3, aws wrangler, Selenium, SqlAlchemy, pandas, Django, Flask, Celery, AWS (ECR, RDS (PostgreSQL), S3, Redshift, Kinesis, SQS, ElastiCache), docker, docker-compose, k8s, helm, helmfile, Snowflake, Fivetran, dbt, DataDog, Opsgenie, REST API, GraphQL, Apache Parquet, Kafka

Back-end developer, LookupLeads

April 2019 – February 2020
Description:  Content data scraping from Linkedin and similar websites.
Responsibilities:

• Content data scraping from Linkedin and similar websites.
• Data processing using Celery chords, chains
• Generating datasets with results and providing a download link for users
• Filling a data warehouse (based ElasticSearch) for use by other services
• Conducting technical interviews for new hires.

Technologies: Python, boto3, Selenium, Scrapy, pandas, Django, Celery, AWS (EC2, ECR, RDS (PostgreSQL), S3, SQS, ElastiCache, SES, ElasticSearch Service), docker, docker-compose, nginx, supervisors

Full-stack developer, Verus RR

September 2017 - March 2019

Description:  Users’ property accounting system

Responsibilities:

• Building of a system for loading and processing user data
• Building pipelines for data transformation/aggregation and saving reports on S3, sending emails to end users with a download link 
• Conducting technical interviews for new hires.

Technologies: Python, Django, Django Rest Framework, Django Channels, Celery, Pandas, python-docx, AWS (EC2, RDS (PostgreSQL), S3, ElastiCache, SES), Vue.js, docker, docker-compose, nginx, Kafka

Full-stack developer, Daunert

March 2015 - August 2017

Description:  Tracking system of customers` requests and the progress of their execution (current status, report, statistics, etc.).

Responsibilities:

• development of architecture;
• front-end development
• Database design
• Bug fixing
• Adding new functionality and modifying existing ones to adapt to new requirements
• Coaching less experienced team members
• Processing business requirements in the development of the project
• GIS integration for saving routes, all kinds of points (such as service stations, hotels, etc.)
• Integration Google Maps

Technologies: PHP, Laravel, Google SDK, AWS (EC2, RDS (MySQL), S3, ElastiCache, SES), Vue.js, docker, docker-compose, nginx, Google Maps, Postgis, GIS

Full-stack developer, Mevo

May 2013 – February 2015

Description:  B2B system designed for distribution among sales managers. Main goals are: help sales managers to know better goods they sale, help brand owners to increase the amount of sales.

Responsibilities:

• development of architecture;
• front-end development
• Database design
• Bug fixing
• Adding new functionality and modifying existing ones to adapt to new requirements
• Coaching less experienced team members
• Processing business requirements in the development of the project

Technologies: PHP, Laravel, Ratchet, AWS (EC2, RDS (MySQL), S3, SES), Vue.js, Ionic + Angular (for moblie development), docker, docker-compose, nginx, MemCahced

Education

Master’s degree in Computer Science and Technology, Zaporizhzhya National Technical University