Senior Data Engineer (Data platform for Industrial manufacturing)
Are you a talented developer looking for a remote job that lets you show your skills and get decent compensation? Join Upstaff.com, a platform that connects you with hand-picked startups and scale-ups in the US and Europe.
Summary
- We are looking for a Senior Data Engineer with Data Architecture, AI/ML skills, who will work on developing AI platform with federated learning, to exchange data between European industrial organizations, create value chains, enabling standartizations automated, digital product passports etc.
- Duration: Long-term
Project Description
We are looking for a Senior Data Engineer with Data Architecture, AI/ML skills, who will work on developing Data platform with federated learning features and privacy-preserving techniques.
You will work on a platform that automates data ingestion, processing, and sharing with user-friendly, privacy-preserving, and scalable solutions for industrial manufacturing.
The platform will incorporate scalable and dynamic tools for creating and managing data spaces, handling complex data workflows, and ensuring modularity and privacy compliance.
Your primary responsibilities will include
- Product development and arcitecture: Platform from scratch, include integrating contextualization and schema mapping models into real-time workflows, mata-data management and end-to-end integration of system components.
- Collaboration with stakeholder, mentoring team members, contributing to system design, and managing AI/ML complexities.
Project:
- Focus is on developing data management system for industrial manufacturing companies targeted at both IT/OT (Information Technology/Operational Technology) data.
- UseCases: Compliance, EU certification. Digital Passports., missions reporting. Collaboration between organizations, product development etc - our solution will supply the data layer (according to RAMI 4.0 - see below)
- Example: BMW/Audi/other EU auto manufacturers have thousands of suppliers, and one product certification implies following up dozens of thousands of links in thousands of supply chains! (via https://catena-x.net/en/)
- Automation & integration of data will be implemented with metadata only and without central permanent storage (such as data lakes), so Machine Learning / federated learning, and AI features are key.
- In our platform pilot use-cases, the plan is to start with compliance-related use cases (e.g., digital product passports for compliance, or supply chain inspection).
Team Skills Coverage:
- Ontologies, Semantics, Knowledge Graphs
- Data contextualization & transformation
- Data federation & correlation
- Time-series data & streaming data handling
- RPA - Robotic Process Automation
- MLOps
- Policy engines
- Backend (Go, Python)
- Databases
- Infrastructure and DevOps (presumably Azure but can be changed to AWS or GCP)
- Security & Zero Trust (attribute-based access control, role-based access control, encryption, SSO)
- API & codeless integration agents (Zapier-like functionality)
Domain and Reference Companies
Broader Industrial Robotics Application areas which need to be covered (additional areas to look experience for):
- IT/OT (Information Technology/Operational Technology) integration
- Digital twins, master data, single source of truth
- Data spaces (industrial data traceability)
- Manufacturing & compliance (digital product passports, emissions reporting)
- Predictive maintenance for industrial processes
- Industrial automation & data quality automation
Reference Industrial Data Management Technologies and Companies, including competitors and look-alikes:
- Siemens Digital Thread (proprietary)
- Cognite (Data platform for industry 4.0) https://cognite.com
- Litmus (Industrial data management) https://litmus.io
- Catena-X (EU automotive data space) https://catena-x.net/en/1
- Rami 4.0 (Reference architecture for Industry 4.0) https://ec.europa.eu/futurium/en/system/files/ged/a2-schweichhart-reference_architectural_model_industrie_4.0_rami_4.0.pdf