Our company is at the forefront of innovation in financial technology. Our mission is to develop an AI-driven platform that predicts market trends, empowering retail investors with the tools previously only accessible to professionals. We're looking for a talented Senior Data Engineer with an interest in finance to join our growing team in our Singapore office.
Responsibilities:
- Focuses on DataOps from ingesting to deployment. Portfolio to be requested here.
- Maintain and enhance trading pipelines.
- Combine various data sources via API or scraping into a singular source of truth.
- Develop feature store and automate features cleaning, processing, engineering, and monitoring of drifts.
- Develop data lineage of each feature and also ensuring features integrity with data fingerprinting techniques.
- Work with batch and streaming data and reduce anomalies in data or gaps in different financial data sources.
- Build, maintain, enhance data pipelines using DataOps, MLOps and AIOps frameworks.
- Work with full suite of data and model infrastructure and integration i.e. SageMaker Studio
- Part of a sub-team focusing on full autonomous AI Trading under a larger trading division.
Requirements:
- Bachelor’s and above
- Must have languages: Python, SQL & NoSQL
- Must have database experiences: Mapreduce, Partitioning, Spark etc.
- Must have data ingestion tools: Kafka, Flume or Sqoop etc.
- Must have data management experiences: Feature Store, Engineering Reproducibility etc.
- Must have testing experiences: Unit Tests, Errors Handling to ensure trading system remains fault tolerance and robust
- Must have orchestration tools: DAGs in Airflow or Prefect, MLFlow
- Must have deployment experiences: Docker, AWS S3, Lambda etc
- Extremely good to have languages: C, C# C++, Java
- Extremely good to have capabilities: Scripting in vim, Neo4j
