jupyterjazz
commited on
Commit
•
837f25a
1
Parent(s):
ebf2504
readme: finetuning (#43)
Browse files- finetuning details (562d3a6d7afdcf505718163e9503561f060a40d8)
- minor change (d2adb21c39ae110976287aeecf030a06fe2a0658)
- simplify setting a task (bd2d9c1701c033fdbd661c40ebaa6156d3c6757b)
README.md
CHANGED
@@ -25178,6 +25178,25 @@ embeddings = model.encode(
|
|
25178 |
)
|
25179 |
```
|
25180 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25181 |
**<details><summary>ONNX Inference.</summary>**
|
25182 |
<p>
|
25183 |
|
|
|
25178 |
)
|
25179 |
```
|
25180 |
|
25181 |
+
You can fine-tune `jina-embeddings-v3` using [SentenceTransformerTrainer](https://sbert.net/docs/package_reference/sentence_transformer/trainer.html).
|
25182 |
+
To fine-tune for a specific task, you should set the task before passing the model to the ST Trainer, either during initialization:
|
25183 |
+
```python
|
25184 |
+
model = SentenceTransformer("jinaai/jina-embeddings-v3", trust_remote_code=True, model_kwargs={'default_task': 'classification'})
|
25185 |
+
```
|
25186 |
+
Or afterwards:
|
25187 |
+
```python
|
25188 |
+
model = SentenceTransformer("jinaai/jina-embeddings-v3", trust_remote_code=True)
|
25189 |
+
model[0].default_task = 'classification'
|
25190 |
+
```
|
25191 |
+
This way you can fine-tune the LoRA adapter for the chosen task.
|
25192 |
+
|
25193 |
+
However, If you want to fine-tune the entire model, make sure the main parameters are set as trainable when loading the model:
|
25194 |
+
```python
|
25195 |
+
model = SentenceTransformer("jinaai/jina-embeddings-v3", trust_remote_code=True, model_kwargs={'lora_main_params_trainable': True})
|
25196 |
+
```
|
25197 |
+
This will allow fine-tuning the whole model instead of just the LoRA adapters.
|
25198 |
+
|
25199 |
+
|
25200 |
**<details><summary>ONNX Inference.</summary>**
|
25201 |
<p>
|
25202 |
|