Jobs
Jobs
Director, Clinical Informatics - AI Cohort Validation & RWD
Summary
Company: Norstella
Location: Remote, United States
Date Posted: Nov 19, 2026
Employment Type: Full Time
Job ID: R-2248
Description
Norstella is a premier and critical global life sciences data and AI solutions provider dedicated to improving patient access to life-saving therapies. Norstella supports pharmaceutical and biotech companies across the full drug development lifecycle - from pipeline to patient. Our mission is simple: to help our clients bring therapies to market faster and more efficiently, ultimately impacting patient lives.
Norstella unites market-leading brands - Citeline, Evaluate, MMIT, Panalgo, Skipta and The Dedham Group and delivers must-have answers and insights, leveraging AI, for critical strategic, clinical, and commercial decision-making. We help our clients:
+ Accelerate the drug development cycle
+ Assess competition and bring the right drugs to market
+ Make data driven commercial and financial decisions
+ Match and recruit patients for clinical trials
+ Identify and address barriers to therapies
Norstella serves most pharmaceutical and biotech companies around the world, along with regulators like the FDA, and payers. By providing critical proprietary data supporting AI-driven workflows, Norstella helps clients make decisions faster and with greater confidence. Norstella's investments in AI are transforming how data is consumed and decisions are made, disrupting inefficient legacy workflows and helping the industry become more efficient, innovative, and responsive to patient needs.
About the Role
Norstella's Real World Data (RWD) team develops high-quality, clinically meaningful healthcare data solutions supporting pharmaceutical and biotechnology research.
We are seeking a Director of Clinical Informatics who combines strong clinical knowledge, medical coding expertise, and technical understanding of healthcare data to support the development and validation of AI-driven RWD solutions.
A primary focus of this role is to evaluate the clinical accuracy of AI-generated patient cohorts and develop and maintain gold standard datasets used to test, benchmark, and improve AI performance.
This individual will bridge clinical, engineering, and data science teams, translating complex clinical requirements into structured definitions, technical specifications, and validation frameworks. The ideal candidate understands both the clinical meaning of healthcare data and how it is represented, transformed, and validated across structured and unstructured data sources.
Key Responsibilities
AI Cohort Validation & Gold Standard Datasets
+ Evaluate AI-generated patient cohorts for clinical accuracy, including inclusion/exclusion criteria, coding logic, treatment patterns, and patient identification.
+ Develop, validate, and maintain gold standard clinical datasets and reference code lists used to benchmark AI-generated outputs.
+ Identify false positives, false negatives, clinical inconsistencies, and gaps in AI-generated cohort definitions.
+ Establish clinical validation criteria and assess AI performance using metrics such as precision, recall, sensitivity, and specificity.
+ Review discrepancies between AI outputs and gold standard datasets, adjudicate complex cases, and recommend improvements.
+ Maintain reference datasets through clinical guideline updates, coding changes, and version-controlled documentation.
Clinical Coding & Logic Development
+ Develop and validate clinically accurate code lists across therapeutic areas, including oncology, hematology, rare diseases, cardiology, and neurology.
+ Translate clinical definitions, diagnostic criteria, and study requirements into structured, computable cohort logic.
+ Apply knowledge of ICD-10-CM/PCS, CPT, HCPCS, RxNorm, NDC, LOINC, and SNOMED CT to ensure accurate clinical representation.
+ Define clinical rules, terminology mappings, inclusion/exclusion criteria, and temporal relationships for patient identification.
+ Document clinical rationale, coding decisions, and validation methodology.
Clinical Data & Technical Collaboration
+ Translate clinical requirements into clear technical specifications and guide engineering teams through implementation.
+ Evaluate structured and unstructured healthcare data, including EHR/EMR, claims, pharmacy, laboratory, genomic, and cancer registry data.
+ Collaborate with data engineers and data scientists to resolve clinical data inconsistencies and improve data quality.
+ Review technical outputs to ensure alignment with clinical requirements and expected outcomes.
+ Support AI/NLP workflows involving clinical entity extraction, terminology normalization, and automated coding validation.
+ Identify opportunities to improve clinical validation processes through automation and scalable data workflows.
+ All other duties, as assigned.
Required Qualifications
+ Bachelor's or advanced degree in medicine, pharmacy, nursing, biomedical informatics, health informatics, or a related field.
+ 4+ years of experience in clinical informatics, medical coding, real-world data, healthcare analytics, or clinical data product development.
+ Strong clinical knowledge of disease processes, diagnostic criteria, treatment pathways, and medical terminology.
+ Experience with standardized medical coding systems, including ICD-10-CM, CPT, HCPCS, RxNorm, LOINC, or SNOMED CT.
+ Experience developing clinical cohort definitions, code lists, or validation rules.
+ Strong understanding of EHR/EMR, claims, and other real-world healthcare datasets.
+ Technical fluency in SQL, relational databases, and clinical data workflows, with the ability to guide engineering implementation.
+ Strong analytical, documentation, and cross-functional communication skills.
Preferred Qualifications
+ Advanced clinical credentials (MD, PharmD, RN) or professional coding certification (CPC, CCS, RHIA).
+ Experience developing gold standard datasets or validating AI-generated clinical outputs.
+ Familiarity with AI/NLP, LLMs, machine learning evaluation, and clinical annotation.
+ Experience in oncology, hematology, rare diseases, or specialty therapeutic areas.
+ Knowledge of OMOP CDM, FHIR, HL7, or other healthcare data standards.
+ Familiarity with Python, clinical data pipelines, or healthcare data architecture.
+ Experience working with RWD vendors, pharmaceutical companies, CROs, or life sciences organizations.
What Success Looks Like
+ AI-generated patient cohorts are clinically accurate and aligned with research requirements.
+ Gold standard datasets are reliable, well-documented, and maintained for ongoing AI evaluation.
+ Clinical requirements are effectively translated into technical specifications and implemented correctly.
+ Clinical inconsistencies and...
Do You Have a MO Jobs Account?
Your HireMO account is created from your MO Jobs account, so we need to connect the two.
First, Let's Find Your MO Jobs Account
HireMO works together with MO Jobs, the State of Missouri jobs website.
Create an Account to Apply
Your MO Jobs account is connected. Choose an email and password to sign in to HireMO.
You Need a MO Jobs Account First
HireMO works together with MO Jobs, the State of Missouri jobs website. You need a MO Jobs account before you can create your HireMO account.
- Create your MO Jobs account at jobs.mo.gov. It is free.
- Log in to MO Jobs, then click HireMO from there. That link brings you back here to finish your registration.
- Complete your HireMO account.