- Overview
- A dynamic graph to construct the relationships between diseases and organs
- Represent Image with medical knowledge graph embeddings with Cross Attention
- Dynamic Graph
- $$G_{pre}$$: 27 nodes, 20 disease keywords, 7 organs or tissues
- $$G_{specific}$$: retrieve top-N reports, extract keywords using Stanza, retrieve knowledge in RadGraph, add nodes to $$G_{pre}$$,
- Embedding: use SciBert to get the initial embedding, add level embedding to show which level the node is at (disease, organ, tissue and so on)
- Learning
- IRC：Report embedding& Image(Graph) embedding -> Contrastive Learning
- IRM: whether image and report match,
- Q: Report
- K, V : Image
- CE Loss
- RG: Self Regression, Teacher Forcing