[Remote] Data Engineering with Python

Note: The job is a remote job and is open to candidates in USA. Diverse Lynx is seeking a Data Engineer with strong expertise in Python-based data engineering and CI/CD automation using GitHub Actions. The candidate will play a critical role in building scalable data pipelines, automating deployments, and enabling high-quality data delivery across cloud platforms.

Responsibilities

• Design, build, and maintain scalable data pipelines using Python
• Develop ETL/ELT workflows for batch and near real-time data processing
• Work with structured and unstructured data across cloud/platform ecosystems
• Implement data transformation, validation, and quality frameworks
• Build and maintain CI/CD pipelines using GitHub Actions to automate: Code builds and validations, Unit/integration testing, Deployment of data pipelines and services
• Implement branching strategies, code reviews, and version control using Git
• Enable automated testing and release management for data workflows
• Integrate CI/CD pipelines with cloud platforms and data services

Skills

• Strong expertise in Python-based data engineering
• CI/CD automation using GitHub Actions
• Building scalable data pipelines
• Automating deployments
• Enabling high-quality data delivery across cloud platforms
• Designing, building, and maintaining scalable data pipelines using Python
• Developing ETL/ELT workflows for batch and near real-time data processing
• Working with structured and unstructured data across cloud/platform ecosystems
• Implementing data transformation, validation, and quality frameworks
• Building and maintaining CI/CD pipelines using GitHub Actions
• Automating code builds and validations
• Unit/integration testing
• Deployment of data pipelines and services
• Implementing branching strategies, code reviews, and version control using Git
• Enabling automated testing and release management for data workflows
• Integrating CI/CD pipelines with cloud platforms and data services

Company Overview

Company H1B Sponsorship