Data Engineer
Job Description
Job Description:
- Design and develop data marts and data warehouses to support business intelligence and data analytics initiatives within the project lifecycle.
- Handle end-to-end data pipelines from multiple internal and external sources, ensuring high performance, scalability, and establishing a Single Source of Truth for project deliverables.
- Collaborate closely with Data PMs, Business Intelligence Analysts, and engineering team members to transform complex data into actionable business insights within designated project timelines.
- Implement and maintain data management best practices across all project pipelines, focusing on data quality, security, and privacy.
- Execute software development best practices, including test-driven development and code documentation, to ensure robust and maintainable data infrastructure.
Requirements:
- Hold a Master's or Bachelor's degree in Computer Science, Information Systems, or other quantitative-based majors (Science or Engineering)
- Possess 2 to 4 years of hands-on working experience in data engineering, data pipeline development, or data architecture.
- Demonstrated expert-level proficiency in SQL, Spark, HDFS, or big data processing frameworks.
- Expert knowledge of Object-Oriented and Functional Programming, with a strong preference for Python or Scala.
- Solid experience working in UNIX environments, with deep understanding of batch processing, lambda architecture, and Kimball data modeling concepts.
- Proficient in Software Development Life Cycle (SDLC) practices, including version control, test-driven development, and technical documentation.
More Info
Job Type:
Industry:
Function:
Employment Type:
Key Skills
test-driven development
Kimball data modeling
About Company
Skills:
Data Warehouse, Sql, Python, ELT, Etl
Skills:
Spark, Selenium, Python, Sql, data processing engineering tools, Airflow, Agentic tools, Playwright, Llm
Skills:
Dml, Pl Sql, Sql, ELT, Oracle Rdbms, Sql Loader, Python, Kubernetes, Etl, Debezium, AWS cloud data services, Performance tuning of high-throughput data pipelines, Solace messaging
Skills:
Data Governance, Data Modeling, Sql, Python, Feature pipelines, Data quality checks, Documentation, Data pipelines, Data contracts
Skills:
Big Data, Sql, Big Data Management Framework