Data Pipeline Development: Design, build, and maintain scalable and robust data pipelines (ETL/ELT) to ingest, clean, and transform data from multiple sources.
Data Integration & Architecture: Manage multi-database synchronization strategies, data migrations, and contribute to the structuring of data warehouses or data lakes.
Data Quality & Reliability: Ensure data integrity, availability, and security across the data infrastructure. Debug scripts, resolve data conflicts, and troubleshoot pipeline failures.
Infrastructure Monitoring: Utilize the ELK stack (Elasticsearch, Logstash, Kibana) or similar tools for log and event data management to monitor pipeline health and performance.
Cross-Functional Collaboration: Work closely with software developers, analysts, and Business Intelligence teams to provide reliable, optimized data that drives strategic business decisions.
Requirements:
Data Engineering & ETL Savvy: Proven experience building data pipelines and working with orchestration/ETL tools (e.g., Apache Airflow, SSIS, Informatica).
Expert in SQL & Programming: Extensive experience with complex SQL queries and relational database design. Strong proficiency in programming languages commonly used in data engineering (Python or Golang).
Hands-On Database Experience: Solid experience working with relational databases such as PostgreSQL, MySQL, and MSSQL Server. Experience with Data Warehouse architecture is highly preferred. Proven experience in designing or building a Data Lakehouse is a strong plus.
Streaming & Event Processing: Familiarity with message brokers and streaming platforms like Apache Kafka.
Cloud & Monitoring Tools: Knowledge of the ELK stack. Experience with DBaaS (Database as a Service) on Cloud platforms (AWS, GCP, or Azure) is a strong plus.
Work Arrangement: Willing to Work From Office (WFO - BSD CITY).
Personal Attributes:
Exceptional communication skills, able to explain complex technical data concepts clearly to both technical and non-technical stakeholders.
Strong analytical and problem-solving abilities with a keen eye for detail.
Proactive, self-motivated, and a great team player.
