Data Engineer

About the Role

You'll be part of a hands-on consulting team building data platforms that clients actually run in production. This isn't a support role — you'll own workstreams, collaborate with client engineers, and be expected to ship. You'll work across the full data lifecycle: ingestion, modeling, orchestration, and serving — on real enterprise-scale data.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines for large-scale processing on cloud platforms (primarily Databricks on AWS)
• Build and optimize pipelines using Databricks — Delta Lake, Auto Loader, Workflows, and SQL Warehouses
• Implement medallion architecture (bronze/silver/gold) with proper data modeling practices
• Collaborate with client stakeholders and internal team to ensure data quality and deliver actionable insights via BI tools
• Monitor and troubleshoot data workflows in production, resolving issues in a structured and timely manner
• Write clean, maintainable PySpark and SQL code that other engineers can read and extend
• Support technical scoping and solution design during pre-sales or project kickoff phases

Requirements

Must Have

• Min. 2 years of experience as a Data Engineer, preferably in a consulting or multi-client environment
• Strong proficiency in Python (PySpark, Pandas) and SQL — CTAS, Window Functions, CTEs
• Hands-on Databricks experience: notebooks, Delta Lake, Workflows, and SQL Warehouses
• Experience with AWS data services: S3, Glue, RDS/Aurora, or Redshift
• Solid understanding of data modeling — star schema, medallion architecture, normalization
• Version control with GitHub or GitLab in a team environment
• Strong communication skills — ability to explain technical decisions to non-technical stakeholders

Nice to Have

• Databricks Unity Catalog, Auto Loader, Delta Sharing, or lakehouse concepts
• Snowflake experience — schema design, data sharing, cost optimization
• SAP data extraction via OData, PyRFC, or similar connectors
• BI tools: Power BI, QuickSight, Tableau, or Databricks AI/BI
• Databricks Certified Data Engineer Associate or above
• API-based data ingestion patterns
• Hands-on experience with MySQL, PostgreSQL, or MariaDB