In this role, you will play a crucial role in building and evolve backend services and scalable pipelines for batch and near-real-time data ingestion. You will design robust data models, optimize platform performance, and ensure cost efficiency within a GCP environment. In this role, you will champion engineering best practices across CI/CD, observability, and automation to power a reliable, high-performance data infrastructure.
Responsibilities
- Architecture & Evolution: Collaborate on designing and improving the data platform's core architecture, scalability, reliability, and performance.
- Best Practices Champion: Drive engineering excellence by contributing to and implementing best practices in CI/CD, data observability, versioning, and automation.
- Cost Optimization: Apply a cost-conscious mindset to cloud resource utilization (specifically within GCP) to ensure efficient data operations.
- Pipeline & Model Management: Design, build, and maintain scalable data pipelines and models for both batch and near-real-time data processing.
- Backend Engineering: Build and evolve backend services specifically tailored for data ingestion and analytics.
- Platform Maintenance: Ensure the day-to-day stability, monitoring, and API-based connectivity of the data platform stack.
- Collaborative Autonomy: Function effectively as an Individual Contributor (IC) who is comfortable working independently while maintaining close, proactive collaboration with peers.
- Cross-Functional Alignment: Work alongside team members to align data ingestion and pipeline capabilities with broader business analytics needs.
Requirements
Minimum Qualifications:
- Years of Experience: 3–4+ years of professional experience operating as an Individual Contributor (IC) Data Engineer.
- Technical Foundations: Proven track record working with distributed systems, data modeling, observability frameworks, and API-based architectures.
- Programming Language Proficiency: Strong, production-level proficiency in backend languages such as Python or Golang.
- Modern Data Stack Expertise: Deep knowledge of modern data tools and infrastructure, specifically:
- Orchestration & Streaming: Airflow, Pub/Sub
- Data Warehousing & Storage: BigQuery, ClickHouse
- Serverless/Compute: Cloud Run, Cloud Functions
- Cloud Ecosystems: Strong conceptual and practical knowledge of cloud environments, with a heavy preference for Google Cloud Platform (GCP).
