Job Responsibilities:
- Focuses on DataOps from ingesting to deployment. Portfolio to be requested here.
- Maintain and enhance trading pipelines.
- Combine various data sources via API or scraping into a singular source of truth.
- Develop feature store and automate features cleaning, processing, engineering, and monitoring of drifts.
- Develop data lineage of each feature and also ensuring features integrity with data fingerprinting techniques.
- Work with batch and streaming data and reduce anomalies in data or gaps in different financial data sources.
- Build, maintain, enhance data pipelines using DataOps, MLOps and AIOps frameworks.
- Work with full suite of data and model infrastructure and integration i.e. SageMaker Studio
Job Requirements:
- Bachelor's in relavent qualifications or above.
- Must have languages: Python, SQL & NoSQL
- Must have database experiences: Mapreduce, Partitioning, Spark etc.
- Must have data ingestion tools: Kafka, Flume or Sqoop etc.
- Must have data management experiences: Feature Store, Engineering Reproducibility etc.
- Must have testing experiences: Unit Tests, Errors Handling to ensure trading system remains fault tolerance and robust
- Must have orchestration tools: DAGs in Airflow or Prefect, MLFlow
- Must have deployment experiences: Docker, AWS S3, Lambda etc
- Extremely good to have languages: C, C# C++, Java
- Extremely good to have capabilities: Scripting in vim, Neo4j
**Salary will be commensurate with the candidate's level of experience and qualifications, reflecting our commitment to equitable compensation practices
