CluelessNovice commited on
Commit
a5c935a
1 Parent(s): 4c16328

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+
5
+ **Base model:** [westlake-repl/SaProt_650M_AF2](https://huggingface.co/westlake-repl/SaProt_650M_AF2)
6
+
7
+ **Dataset:** [SaProtHub/Dataset-Beta_Lactamase-PEER](https://huggingface.co/datasets/SaProtHub/Dataset-Beta_Lactamase-PEER)
8
+
9
+ **Performance (on test set):** 0.809 Spearman's ρ
10
+
11
+ **LoRA config:**
12
+ - **r:** 8
13
+ - **lora_dropout:** 0
14
+ - **lora_alpha:** 8
15
+ - **target_modules:** ["query", "key", "value", "intermediate.dense", "output.dense"]
16
+ - **modules_to_save:** ["classifier"]
17
+
18
+ **Training config:**
19
+
20
+ - **optimizer:**
21
+ - **class:** AdamW
22
+ - **betas:** (0.9, 0.98)
23
+ - **weight_decay:** 0.01
24
+ - **learning rate:** 1e-4
25
+ - **epoch:** 50
26
+ - **batch size:** 64
27
+ - **precision:** 16-mixed