Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements
Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data
Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development
Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc.
Unify, enrich, and analyze variety of data to derive insights and opportunities
Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities
Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform
Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org
Requirement & Qualification
Bachelor’s degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science. Master is a plus.
2 to 5 years experiences on Cloud Computing (Aliyun, Azure), DataBricks, Micro Services, Streaming Technologies, Network, Security, etc.
2 to 5 years experiences on development experience using SQL, Python-spark, Spark Streaming, MySQL/SQL Server, Cosmos DB/Mongo DB.
Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams
Independent and able to manage, prioritize & lead workload