ASTEK has been providing IT and Engineering solutions for some of the world’s largest industrial and services groups for more than 35 years with 7200 passionate experts in 22 countries throughout Europe, APAC, Middle East and the Americas.
Currently, we are looking for Data Engineer for Cloud project which would be based in Singapore.
Responsibilities:
• Design and build robust and scalable data ingestion and data management solutions for batch loading and streaming data sources using Python.
• Work closely with data stewards, data analysts, and business end users to implement and support data solutions.
• Familiar with SDLC process such as requirement gathering, design and development, SIT testing, support UAT and CICD deployment for enhancement and new ingestion pipeline.
• Ensure compliance with IT security standards, policies, and procedures.
• Provide BAU support in terms of production job monitoring, issue resolution, and bug fixes.
• Enable ingestion checks and data quality checks for all data sets in the data platform and ensure the data issues are actively detected, tracked, and fixed without breaching SLA.
Requirements:
• Minimum 3 years of experience in a role focusing on development and support of data injection pipelines with a possess a degree in Computer Science/Information Technology or related fields.
• Experience with building on data platforms using Snowflake.
• Proficient in SQL and Python.
• Experience with continuous integration and continuous deployment (CICD).
• Experience with Software Development Life Cycle (SDLC) methodology.
• Experience with data warehousing concepts.
• Strong problem solving and troubleshooting skills.
• Strong communication and collaboration skills.
• Able to design and implement solution and perform code review.
• Agile, fast learner and able to adapt to change.
Good to have:
• Cloud environment (e.g. AWS).
• Experience in ETL tools (such as Informatica), and data virtualization platforms such as Denodo is preferred.
• Experience in Java.
