Database/BigData Engineer (Postgres, MySQL, Mongo) on Azure

Database Development.

Senior

Upper-Intermediate English

Required Skills

MongoDB

4.0 yr.

Cosmos DB

2.0 yr.

SQL

5.0 yr.

Nice to Have

PostgreSQL

5.0 yr.

MySQL

5.0 yr.

Summary

The team's goal is to build a full-cycle data management platform, which will include data ingestion, ETL, data quality, data enrichment, data processing pipelines orchestrated into an "elastic data fabric" and most importantly, utilizing federated learning. (ML on multiple data sources with managed models and data interchange)

The client is a new venture, established by a well-established international business with expertise in the industrial manufacturing domain. The project’s ambition is to build an innovative data space platform for the industry, which will take the leading position because of its usefulnчess and performance.

Are you a talented developer looking for a remote job that lets you show your skills and get decent compensation? Join Upstaff.com, a platform that connects you with hand-picked startups and scale-ups in the US and Europe.

Our Ideal candidate required skills are: experience with Mongo / Cosmos DB,

Strong knowledge of SQL databases (Ideally Azure SQL Database or similar - Postgres, MySQL), PostgreSQL, MySQL and derivative managed and unmanaged services.

Nice to have:

Federated data, Knowledge of metadata management, data lineage and governance tools (e.g., Microsoft Purview or simialr tools)
familiarity with ML, Data pipelines
familiarity with Azure ecosystem ADLS Gen2 or equivalent for data lake, Databricks - would be a plus!

Project and Client:

The project is about building a data spaces platform for industrial manufacturing.

Data spaces is a shared distributed data management system combining multiple data sources applying ML models and managed data exchange.

The platform we're building aims to automate data ingestion, processing, and sharing with user-friendly, privacy-preserving, and scalable solutions for industrial manufacturing.

The platform will incorporate scalable and dynamic tools for creating and managing data spaces, handling complex data workflows, ensuring modularity and privacy compliance.

Project’s Technical Stack:

Backend: Python, Flask/FastAPI, Go
Frontend: ReactJS, Angular.
AI/ML: Azure Machine Learning, Azure Databricks, TensorFlow Federated, PyTorch, and privacy-enhancing techniques.
Cloud and DevOps: Kubernetes, Docker, Azure DevOps, CI/CD Data pipelines on Azure
Data Engineering: Apache NiFi. Kafka Connect, Databricks - on Azure.
Database: Cosmos DB, Postgres/Hyperscale or MySQL/Healwave

*the stack may change during the hiring process of qualified specialists in their areas

Not your tech stack?

Join the Upstaff community and we are looking for the best project for you. Be ready for the next steps:

Create your profile on our website (import from LinkedIn)
20-30-minute screening call
Technical interview
Feedback
Project Selection (we are looking for the best project for you).

We work with developers from 50+ countries in different regions: Europe, LATAM, the U.S. (W-9 form owners), Canada, Asia (Philippines, Indonesia), Oceania (Australia, New Zealand, Papua New Guinea), and the the UK.

We don’t have a legal and ethical basis to accept applicants from the following countries: Russia, Belarus, Iran, North Korea
We do not provide visa assistance, and our cooperation model does not include the benefits typically offered with direct hire.