Hire PySpark Developer

- Data Engineer with a Ph.D. degree in Measurement methods, Master of industrial automation - 16+ years experience with data-driven projects - Strong background in statistics, machine learning, AI, and predictive modeling of big data sets. - AWS Certified Data Analytics. AWS Certified Cloud Practitioner. Microsoft Azure services. - Experience in ETL operations and data curation - PostgreSQL, SQL, Microsoft SQL, MySQL, Snowflake - Big Data Fundamentals via PySpark, Google Cloud, AWS. - Python, Scala, C#, C++ - Skills and knowledge to design and build analytics reports, from data preparation to visualization in BI systems.

- 12+ years of experience in IT, with 12+ years in Data Engineering and Data Architecture, including Oracle Databases, Data Warehousing, Big Data, and real-time streaming systems; - Experience in designing and maintaining enterprise Data Warehouses, leading cloud migration initiatives across Azure, AWS, and GCP; - Strong architectural expertise in ETL/ELT pipelines, batch/real-time processing, and data governance/quality frameworks; - Deep knowledge of Big Data ecosystems (Cloudera, Hadoop, Databricks, Synapse Analytics, HDInsight, AWS EMR); - Skilled in multi-cloud architecture design using Snowflake, DBT, Cosmos DB, Redshift, BigQuery, Athena, and Data Lake solutions; - Experienced in data streaming and integration with Apache Kafka, Apache Spark, PySpark, and Airflow; - Expertise in BI and reporting systems with Power BI and Tableau for data visualization and analytics delivery; - Strong foundation in database administration and security: Oracle EBS R12, RAC/ASM, WebLogic, SOA Suite, ERP systems, database audits and compliance; - Certified in Azure Data Engineer, AWS Data Analytics Specialty, Confluent Kafka, Oracle DBA.

Engineer with 10+ years’ experience in AI, excelling in NLP, GenAI, computer vision, and model deployment. Expertise in Python, SQL, PySpark, and cloud platforms. Established record of enhancing AI-driven services, improving workflows, and boosting efficiency. Proven capabilities in developing robust ML pipelines, integrating state-of-the-art technologies like LLMs and RAG pipelines, and delivering solutions across diverse industries. Advanced in web and full-stack development, data engineering, analysis, and DevOps practices, underpinned by solid formal education in computer science.

- 3+ years of experience as a BI Engineer; - Strong abilities in Power BI, SSIS, Tableau, and Google Data Studio; - Deep skills in developing and optimizing ETL processes within business intelligence; - Experience with SQL, Python; - Familiar with Docker, Apache Airflow, and PySpark; - Good knowledge of data warehousing and business intelligence principles.

- 8 years experience with various data disciplines: Data Engineer, Data Quality Engineer, Data Analyst, Data Management, ETL Engineer - Automated Web scraping (Beautiful Soup and Scrapy, CAPTCHAs and User agent management) - Data QA, SQL, Pipelines, ETL - Data Analytics/Engineering with Cloud Service Providers (AWS, GCP) - Extensive experience with Spark and Hadoop, Databricks - 6 years of experience working with MySQL, SQL, and PostgreSQL; - 5 years of experience with Amazon Web Services (AWS), Google Cloud Platform (GCP) including Data Analytics/Engineering services, Kubernetes (K8s) - 5 years of experience with PowerBI - 4 years of experience with Tableau and other visualization tools like Spotfire and Sisense; - 3+ years of experience with AI/ML projects, background with TensorFlow, Scikit-learn and PyTorch; - Extensive hands-on expertise with Reltio MDM, including configuration, workflows, match rules, survivorship rules, troubleshooting, and integration using APIs and connectors (Databricks, Reltio Integration Hub), Data Modeling, Data Integration, Data Analyses, Data Validation, and Data Cleansing) - Upper-intermediate to advanced English, - Henry is comfortable and has proven track record working with North American timezones (4hour+ overlap)

Expertise in Python, SQL, and ML web applications with a focus on practical applications in fields like AI and computer vision. Proven track record in AI with projects at DeepLogic AI, including creating an ML Web app for classifying blastocyst embryos and implementing LLM for in-house document processing. Further experience includes feature extraction via PySpark at Tata Digital, and exploiting BERT for NLP tasks at Concentrix. Academically grounded with a Master's in CS from Columbia University and a BTech in Math/CS from Delhi University. Holds AWS accreditations, demonstrating proficiency in cloud architectures, EC2, S3, and RDS. Skilled in end-to-end software development, including full-stack abilities shown at Datacultr with Django and PostgreSQL.

- 2+ years of experience with Python as a Data Engineer and Deep/Machine Learning Intern - Experience with Data Vault modeling and AWS cloud services (S3, Lambda, and Batch) - Cloud Services: Sagemaker, Google BigQuery, Google Data Studio, MS Azure Databricks, IBM Spectrum LSF, Slurm - Data Science Frameworks: PyTorch, TensorFlow, PySpark, NumPy, SciPy, scikit-learn, Pandas, Matplotlib, NLTK, OpenCV - Proficient in SQL, Python, Linux, Git, and Bash scripting. - Had experience leading a BI development team and served as a Scrum Master. - Native English - Native German

- Senior Data Engineer with 10+ of experience specializing in designing, optimizing, and maintaining data infrastructures, data flow automation, and algorithm development. - Has expertise in Python, SQL/NoSQL, ETL processes, PySpark, Apache Airflow, and an array of AWS services, complemented by a strong foundation in database systems and cloud-based solutions. Proven capability in handling large-scale data analytics and processing with a focus on performance and cost efficiency in cloud environments. Proficient in developing robust ETL pipelines, performing data migrations, and optimizing complex queries and storage procedures, leveraging extensive experience across multiple industries and platforms. - Start: ASAP - English: Upper-Intermediate