Epic: HEAL Harmonization

[oihawkins]

Introduction

The HEAL program includes hundreds of projects generating highly diverse data sets.

RTI and RENCI, the HEAL Stewards, will

• Use semantic knowledge graphs to link data from disparate studies to facilitate analysis.
• Perform preliminary harmonization of data sets towards the HEAL Common Data Elements (CDE)s
• Provide user friendly interfaces with a biological lens on the data.

HEAL Harmonization

To accomplish this:

• Ingest HEAL CDEs
  • Create (or locate) machine readable versions of the HEAL CDEs @gaurav
  • Discuss w/NIH how to publish machine readable HEAL CDEs @gaurav
  • Annotate with controlled vocabulary and ontology identifiers @gaurav
    • Map provided NCI Metathesaurus ids to Human Phenotype Ontology (etc) identifiers @gaurav
    • Annotate HEAL CDEs with Monarch's BioLink API SciGraph Named Entity Recognition service (NER) @gaurav
    • Look into alternate NER tools for finding terms in HEAL CDEs (https://github.com/helxplatform/development/issues/804) @gaurav
    • Convert to Biolink and KGX compliant artifacts @gaurav
    • Apply the SRI Normalizer to use Translator preferred identifiers @YaphetKG @gaurav
    • Clean up SciGraph annotations and resend cleaned KGX files to Yaphet @gaurav
  • Optimize TranQL queries to take advantage of Redisgraph performance. @YaphetKG
  • Create new harmonization and translational TranQL queries to @YaphetKG
  • Link variables through phenotypes, chemicals, diseases to CDEs @YaphetKG
  • Index, recording harmonization connections to enable display
• Ingest HEAL study data as it becomes available @waTeim
  • Begin with
    • The SPARC knowledge graph. @HowardLander
    • NIDA (as transformed/provided by Comp 2) @warrenstephens
  • Create a Dug parser for each data format.
• Update the HeLx/Dug UI to @mbwatson
  • Render
    • An "All" tab for generic search results including lexical matches.
    • A "Harmonized" tab for everything else with markers for CDE, PhenX, Biolink, and other groupings.
  • Allow deployment with or without
    • Authentication
    • App Workspaces
  • Present an information dense display minimizing paging and scrolling

Design

@alexwaldrop @jcheadle-rti

PM Tracking

@hhiles to work with @vgardner-renci and Kathy to report on the following HEAL epics and their related GitHub tickets; give updates directly to PMs or plug into Monday.com

• Index Data Dictionaries
  • #169
  • #768
  • Collect remaining 500-600 HEAL studies
• Develop Automated Curation and Search
  • CDE Harmonization #221
  • Harmonization phase2 #218
• Enable Cloud Based Semantic Search
  • Simplification and Normalization of Standard Apps Deployment #225
  • Variable View #230
  • A user can create a json file of Dug search results to take outside of Dug for further analyses #249

[oihawkins]

View past comments and figures for this work https://github.com/helxplatform/development/issues/742

[krobasky]

@hhiles Questions:

1. Is it possible to give me at least read-access to helxplatform/development repo so that I can see a) context of this ticket on 742, and the PM tracking tickets 169, 768?
2. Alex is gathering metadata using either CEDAR or REDCap, and I could be wrong but I believe this is outside the RTI CDE, HEAL CDE, and PhenX ingestion. Should this be added to the design?
3. I don't see a task or owner for creating, integrating, and deploying the HEAL pipeline, handler & test suite, or is that part of the "Create a Dug parser for each data format" task owned by 'waTeim'?

[oihawkins]

@krobasky

1. Done - read access
2. I can't comment on what should change with the design - this is something that could be discussed at a future scrum @alexwaldrop
3. I read through the descriptions of the 3 epics in Monday as they relate to the 3 above (epic 20, 22, and 23), and I am not entirely sure which one creating, integrating, and deploying the HEAL pipeline, handler & test suite relates to. Can you clarify?

[krobasky]

@hhiles

I think we've come to the conclusion that the Monday epics don't map well operational tasks, so I'm not able to answer your question other than saying it maps broadly across all the epics.

But before we go further, let me confirm -

@alexwaldrop

Do I correctly understand that a HEAL parser needs to be created (with handler & test suite) in order to deploy the HEAL data dictionaries to heal.renci.org? Or are you thinking there will need to be one parser per dictionary provided by each PI?

[oihawkins]

@cbizon @YaphetKG @gaurav I'd like to avoid just blanket recreating this ticket in jira - can we identify what components of this work still need to be done, and what still needs to be recreated?

[oihawkins]

@mbwatson and @YaphetKG - what is left to do on this ticket?

[oihawkins]

most of this ticket is in jira or is done. confirming with @mbwatson on the UI portion