Job Description:
We are looking for a skilled and experienced Data Engineer to join our team. The ideal candidate will be responsible for designing, building, and maintaining robust data pipelines to support the processing and analysis of clinical study and digital device sensor data. As a Data Engineer, you will work closely with data scientists and software engineers to ensure the efficient and reliable flow of data from source systems to analytical tools and platforms.
Responsibilities:
- Design, develop, and maintain scalable data pipelines to ingest, transform, and load clinical study data from various sources, including digital device sensors.
- Optimize data storage and retrieval processes in cloud-based platforms to ensure high performance and reliability.
- Collaborate with data scientists to integrate data processing pipelines with AI-powered algorithms and third-party analytical tools or platforms.
- Implement data quality checks and monitoring mechanisms to ensure the integrity and accuracy of the data.
- Troubleshoot and resolve issues related to data pipeline performance, reliability, and scalability.
- Work closely with software developers, system architects and other cross-functional teams to develop data-driven business solutions.
- Stay up-to-date with emerging technologies in AI & Cloud computing and best practices in data engineering to continuously improve data processing pipelines and infrastructure.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Proven experience in designing and building data pipelines using ETL tools and frameworks such as Apache Spark, Apache Beam, or Apache Airflow.
- Proficiency in programming languages such as Python, Java, or Scala.
- Strong understanding of database systems, data warehousing concepts, SQL and NoSQL.
- Experience with AI and Cloud Computing: Hands-on experience with cloud platforms like AWS and familiarity with AI solutions in these environments.
- Excellent problem-solving and troubleshooting skills with a strong attention to detail and quality.
- Effective communication and collaboration skills with the ability to work in a team environment.
Preferred Qualifications:
- Experience with containerization and orchestration tools such as Docker and Kubernetes.
- Familiarity with big data technologies such as Hadoop, Hive, or Presto.
- Knowledge of distributed computing frameworks such as Apache Hadoop or Apache Spark.
- Prior experience working with healthcare or clinical data is a plus.
Kindly include your expected salary in your resume.
