whaleloops
commited on
Commit
•
b798553
1
Parent(s):
eada6dd
Update README.md
Browse files
README.md
CHANGED
@@ -2,10 +2,10 @@
|
|
2 |
license: cc-by-nc-4.0
|
3 |
---
|
4 |
|
5 |
-
KEPTlongfomer using [contrastive learning](https://arxiv.org/pdf/2210.03304.pdf).
|
6 |
|
7 |
|
8 |
-
First,
|
9 |
|
10 |
And then pretrained with Hierarchical Self-Alignment Pretrainumls (HSAP) using Knowledge Graph UMLS.
|
11 |
This includes (a) Hierarchy, (b) Synonym, (c) Abbreviation. For more info, see section 3.3 in [paper](https://arxiv.org/pdf/2210.03304.pdf).
|
|
|
2 |
license: cc-by-nc-4.0
|
3 |
---
|
4 |
|
5 |
+
KEPTlongfomer pretrained using [contrastive learning](https://arxiv.org/pdf/2210.03304.pdf).
|
6 |
|
7 |
|
8 |
+
First, The model was first inited from RoBERTa-base-PM-M3-Voc-distill from [bio-lm](https://github.com/facebookresearch/bio-lm/blob/main/README.md).
|
9 |
|
10 |
And then pretrained with Hierarchical Self-Alignment Pretrainumls (HSAP) using Knowledge Graph UMLS.
|
11 |
This includes (a) Hierarchy, (b) Synonym, (c) Abbreviation. For more info, see section 3.3 in [paper](https://arxiv.org/pdf/2210.03304.pdf).
|