Data Engineer

We’re a cognitive science company on a mission to optimize the measurement of brain health to advance the development of new medicines and to enable earlier clinical insights in healthcare.

That’s why we’re seeking a Data Engineer responsible for building and maintaining data pipelines in Azure Databricks to enable Cogstate’s data strategy. The candidate will have a central critical role for establishing and maintaining the Azure Databricks infrastructure, provide best practices and recommendations for using these tools, and will working closely with data scientists, data managers, clinical scientists, and other business stakeholders as a point of contact for the data platform infrastructure.

*Salary Range: $125,000 - $155,000 per year (commensurate with experience). Benefits listed below.

Core Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory, and integrate data from various sources and systems into a coherent data ecosystem.
• Utilize Azure Databricks for complex data transformation and processing tasks, ensuring high performance and alignment with business requirements.
• Manage Azure Functions for efficient data ingestion and processing, optimizing for performance and cost.
• Implement and maintain data storage solutions in Azure Data Lake Storage Gen2 (ADLS2), applying best practices for data lifecycle management, security, and access control.
• Collaborate with cross-functional teams to define data requirements, deliver comprehensive data solutions, and support data-driven decision-making processes.
• Employ GitHub for source control management, adhering to best practices for code reviews, branching, and versioning.
• Facilitate CI/CD pipelines using GitHub Actions to automate testing, integration, and deployment processes, enhancing productivity and ensuring high-quality deployments.
• Monitor, troubleshoot, and optimize data systems and pipelines for performance, reliability, and scalability.
• Stay abreast of industry trends and advancements in cloud and data engineering technologies, continuously seeking opportunities for innovation and improvement.
• Maintain Data Governance infrastructure to ensure appropriate data access rights and restrictions for internal and external users.

Qualifications

• BS/BA in Computer Science, Data Science, or a related field or relevant experience
• 5+ years’ experience as a data engineer.
• Must have 2+ years experience in implementing data engineering solutions with Apache Spark, preferably Databricks.
• Must have deep expertise in one of the programming languages for data processes (Python, Scala). Experience with Python, PySpark, Hadoop, Hive and/or Spark to write data pipelines and data processing layers.
• Direct experience with relational databases like SQL Server. Excellent SQL experience for writing complex SQL transformations.
• Strong knowledge on Databricks configuration, troubleshooting and performance tuning.
• Experience with development tools for CI/CD, unit and integration testing, automation and orchestration, including GitHub, Azure Data Factory and Azure functions.
• Familiarity with Agile workflow methodologies, demonstrating an ability to work effectively in a dynamic, iterative development environment and manage tasks with varying priorities.

Benefits

• Remote Work Practices: Cogstate is a virtual first company. Cogstate employees can work from anywhere where Cogstate is registered to business within the United States, Australia, or the United Kingdom!
• Generous Paid Time-off: Cogstate employees receive 20 days of vacation leave, 10 days of personal leave and 10 paid public holidays, unused days roll over year after year.
• 401(k) Matching: As you invest in yourself and your future, Cogstate invests in you too: we match up to 3% of your yearly salary in Cogstate’s 401k program
• Competitive Salary: We offer competitive base salaries plus additional earning opportunities based on the position.
• Health, Dental & Vision Coverage: We've invested in comprehensive health & dental insurance options with competitive company contributions to help when you need it most. We also offer free vision insurance for all full-time employees.
• Short-Term & Long Term Disability Life Insurance: 100% employer sponsored
• Pre-Tax Benefits: Healthcare and Dependent Care Flexible Spending Accounts
• Learning & Development Opportunities: Cogstate offers a robust learning program from mentorships to scholarships to Coursera to improve knowledge or obtain certifications in applicable areas of interest.

Why We Do What We Do

Our mission at Cogstate is to optimize the measurement of brain health to advance the development of new medicines and to enable earlier clinical insights in healthcare. We are driven by the notion that through our work we impact the health of communities by delivering solutions that combine breakthrough science with advanced technologies. We are inspired by the dedication of researchers and the resilience of patients, and we are strengthened by our 20-year heritage supporting them. We are always working to simplify the complex with solutions that offer insights and hope for the future of healthcare, particularly dementia-related diseases and rare and pediatric disorders.

Commitment to Diversity & Inclusion

Cogstate is committed to building and maintaining a fair, diverse and inclusive workplace where the personal worth of each employee is recognized, and all are respected and valued for their differences. Applicants with disabilities may be entitled to reasonable accommodation under the terms of the Americans with Disabilities Act and certain state or local laws. If you need assistance in applying please email [email protected].