Develop data pipelines for batch and streaming applications
Develop data products and services like user profiling, realtime reconciliation, etc.
Keep learning big data technologies and continuously optimize products and services
Requirements
3+ years experiences of developing big data related applications
Proficient Java / Scala / Python (at least one language) and SQL programming languages
Proficient in big data pipeline development including batch and streaming processing
Familiar with big data components like Hadoop, Hive, Spark, Spark streaming, Flink, Presto, Hbase, Kafka, Zookeeper, Airflow, Elasticsearch, Redis
Experience in big data product development is a plus, including data warehouse, query platform, streaming, recommendation, user profiling, risk management, etc.
Experience in API developing with at least one of the frames: Spring boot, AWS lambda + API gateway
Team working, analytical mind, be optimistic, willing to accept challenges and quick response against incidents