Data Engineer - Search & Recommendation

0-2 years
7 months ago
Job Description

Job Description:

  • Design, build, and maintain the real-time streaming system to support timely and accurate ingestion of various types of data (e.g. User behavior, RDS, NoSQL DB, and others) into storage.
  • Design, build, and maintain data warehouse for Shopee's e-commerce search and recommendation.
  • Translate data requirements into scalable technical data service with low latency and high concurrency.
  • Design, build, and maintain batch or real-time data pipeline for algorithm, analytics, and operational purposes using Hadoop big data technology.
  • Analyze and improve the efficiency, scalability, and stability of the data pipelines and related platforms or tools.
  • Define and manage service level agreement and data quality for all data sets.


  • Minimum Bachelor's degree in Computer Science or a related technical field.
  • Proficient in experience working with programming languages, such as Java and/or Scala.
  • Proficient in Hadoop, Spark, MapReduce, and Hive/Spark/Presto SQL, experience with PB batch data processing is a plus.
  • Familiar with data warehouse standards, experience with live system data warehouse is a big plus.
  • Familiar with streaming data processing using Flink and Kafka.
  • Experience in designing and operating a robust distributed system is a plus.
  • Understanding data mining or machine learning is a plus.
  • Passionate to working intimately with data.
  • Self-motivated and takes ownership.



Job Source:

Shopee Pte. Ltd. is a Singaporean multinational technology company that specialises in e-commerce. The company was launched in Singapore in 2015, before it expanded abroad. As of 2021, Shopee is considered the largest e-commerce platform in Southeast Asia with 343 million monthly visitors.

Career Advice to Find Better