Architect - Data Engineering

Architect- Data Engineering (Architect - 2 )

Location: Bangalore

Mode: Hybrid

Notice period: Max 30 days (Immediate / short notice preferred)

Experience: Architect: 14+ Years

Job Description:

Role Summary

We are looking for an experienced Architect - Data Engineering with 14+ years of experience and strong expertise in AWS, Databricks, PySpark, and Python. The ideal candidate will lead the design, development, and optimization of scalable data platforms and pipelines, provide technical leadership to a team of data engineers, and collaborate with business and technology stakeholders to deliver secure, reliable, and high-performing data solutions.

Key Skills

• AWS Cloud Services (S3, Glue, EMR, Redshift, Lambda, IAM, EC2)
• Databricks
• PySpark
• Apache Airflow
• ETL/ELT Pipeline Development
• Data Modeling & Data Architecture
• SQL & Performance Optimization
• Python

Must-Have Skills

• Experience with Databricks for data engineering and analytics
• Strong expertise in AWS Cloud Services (S3, Glue, EMR, Lambda, Redshift, IAM, EC2)
• Expert-level proficiency in PySpark for large-scale data processing
• Hands-on experience in designing and developing ETL/ELT pipelines
• Advanced SQL and Data Modeling skills
• Strong programming experience in Python

Good-to-Have Skills

• Big Data technologies such as Spark Streaming, Hadoop, and Kafka
• CI/CD and DevOps practices
• Infrastructure as Code (Terraform, CloudFormation)
• Docker and Kubernetes
• Agile methodologies
• Exposure to Snowflake, Azure Data Factory, or GCP Data Services

Roles & Responsibilities

• Lead the design, development, and implementation of scalable data engineering solutions
• Define architecture standards and best practices for data platforms
• Review technical designs and ensure adherence to engineering standards
• Drive technical innovation and continuous improvement initiatives
• Mentor and guide a team of Data Engineers
• Foster a collaborative and high-performance engineering culture
• Build and maintain data pipelines using AWS, Databricks, PySpark, and Airflow
• Develop high-performance ETL/ELT processes for structured and unstructured data
• Design and optimize data models to support reporting, analytics, and machine learning use cases
• Ensure high availability, reliability, and scalability of data platforms

Regards

Vibha Patel

[Confidential Information]