cassador commited on
Commit
3322423
1 Parent(s): 5fdc060

Add new SentenceTransformer model.

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,500 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: indobenchmark/indobert-base-p2
3
+ datasets:
4
+ - afaji/indonli
5
+ language:
6
+ - id
7
+ library_name: sentence-transformers
8
+ metrics:
9
+ - pearson_cosine
10
+ - spearman_cosine
11
+ - pearson_manhattan
12
+ - spearman_manhattan
13
+ - pearson_euclidean
14
+ - spearman_euclidean
15
+ - pearson_dot
16
+ - spearman_dot
17
+ - pearson_max
18
+ - spearman_max
19
+ pipeline_tag: sentence-similarity
20
+ tags:
21
+ - sentence-transformers
22
+ - sentence-similarity
23
+ - feature-extraction
24
+ - generated_from_trainer
25
+ - dataset_size:6915
26
+ - loss:SoftmaxLoss
27
+ widget:
28
+ - source_sentence: Pesta Olahraga Asia Tenggara atau Southeast Asian Games, biasa
29
+ disingkat SEA Games, adalah ajang olahraga yang diadakan setiap dua tahun dan
30
+ melibatkan 11 negara Asia Tenggara.
31
+ sentences:
32
+ - Sekarang tahun 2017.
33
+ - Warna kulit tidak mempengaruhi waktu berjemur yang baik untuk mengatifkan pro-vitamin
34
+ D3.
35
+ - Pesta Olahraga Asia Tenggara diadakan setiap tahun.
36
+ - source_sentence: Menjalani aktivitas Ramadhan di tengah wabah Corona tentunya tidak
37
+ mudah.
38
+ sentences:
39
+ - Tidak ada observasi yang pernah dilansir oleh Business Insider.
40
+ - Wabah Corona membuat aktivitas Ramadhan tidak mudah dijalani.
41
+ - Piala Sudirman pertama digelar pada tahun 1989.
42
+ - source_sentence: Dalam bidang politik, partai ini memperjuangkan agar kekuasaan
43
+ sepenuhnya berada di tangan rakyat.
44
+ sentences:
45
+ - Galileo tidak berhasil mengetes hasil dari Hukum Inert.
46
+ - Kudeta 14 Februari 1946 gagal merebut kekuasaan Belanda.
47
+ - Partai ini berusaha agar kekuasaan sepenuhnya berada di tangan rakyat.
48
+ - source_sentence: Keluarga mendiang Prince menuduh layanan musik streaming Tidal
49
+ memasukkan karya milik sang penyanyi legendaris tanpa izin .
50
+ sentences:
51
+ - Rosier adalah pelayan setia Lord Voldemort.
52
+ - Bangunan ini digunakan untuk penjualan.
53
+ - Keluarga mendiang Prince sudah memberi izin kepada TImbal untuk menggunakan lagu
54
+ milik Prince.
55
+ - source_sentence: Tujuan dari acara dengar pendapat CRTC adalah untuk mengumpulkan
56
+ respons dari pada pemangku kepentingan industri ini dan dari masyarakat umum.
57
+ sentences:
58
+ - Pembuat Rooms hanya bisa membuat meeting yang terbuka.
59
+ - Masyarakat umum dilibatkan untuk memberikan respon dalam acara dengar pendapat
60
+ CRTC.
61
+ - Eminem dirasa tidak akan memulai kembali kariernya tahun ini.
62
+ model-index:
63
+ - name: SentenceTransformer based on indobenchmark/indobert-base-p2
64
+ results:
65
+ - task:
66
+ type: semantic-similarity
67
+ name: Semantic Similarity
68
+ dataset:
69
+ name: sts dev
70
+ type: sts-dev
71
+ metrics:
72
+ - type: pearson_cosine
73
+ value: 0.5829898836235055
74
+ name: Pearson Cosine
75
+ - type: spearman_cosine
76
+ value: 0.5604880880211627
77
+ name: Spearman Cosine
78
+ - type: pearson_manhattan
79
+ value: 0.5703534992812126
80
+ name: Pearson Manhattan
81
+ - type: spearman_manhattan
82
+ value: 0.5499989364166947
83
+ name: Spearman Manhattan
84
+ - type: pearson_euclidean
85
+ value: 0.5753323630988341
86
+ name: Pearson Euclidean
87
+ - type: spearman_euclidean
88
+ value: 0.552442969754755
89
+ name: Spearman Euclidean
90
+ - type: pearson_dot
91
+ value: 0.5620113473718095
92
+ name: Pearson Dot
93
+ - type: spearman_dot
94
+ value: 0.5624324325309726
95
+ name: Spearman Dot
96
+ - type: pearson_max
97
+ value: 0.5829898836235055
98
+ name: Pearson Max
99
+ - type: spearman_max
100
+ value: 0.5624324325309726
101
+ name: Spearman Max
102
+ - task:
103
+ type: semantic-similarity
104
+ name: Semantic Similarity
105
+ dataset:
106
+ name: sts test
107
+ type: sts-test
108
+ metrics:
109
+ - type: pearson_cosine
110
+ value: 0.27661444766220145
111
+ name: Pearson Cosine
112
+ - type: spearman_cosine
113
+ value: 0.25397061268923804
114
+ name: Spearman Cosine
115
+ - type: pearson_manhattan
116
+ value: 0.22893950626786405
117
+ name: Pearson Manhattan
118
+ - type: spearman_manhattan
119
+ value: 0.2295445814901059
120
+ name: Spearman Manhattan
121
+ - type: pearson_euclidean
122
+ value: 0.23773763148887356
123
+ name: Pearson Euclidean
124
+ - type: spearman_euclidean
125
+ value: 0.23225044424139019
126
+ name: Spearman Euclidean
127
+ - type: pearson_dot
128
+ value: 0.2930559400528471
129
+ name: Pearson Dot
130
+ - type: spearman_dot
131
+ value: 0.28163535345836893
132
+ name: Spearman Dot
133
+ - type: pearson_max
134
+ value: 0.2930559400528471
135
+ name: Pearson Max
136
+ - type: spearman_max
137
+ value: 0.28163535345836893
138
+ name: Spearman Max
139
+ ---
140
+
141
+ # SentenceTransformer based on indobenchmark/indobert-base-p2
142
+
143
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [indobenchmark/indobert-base-p2](https://huggingface.co/indobenchmark/indobert-base-p2) on the [afaji/indonli](https://huggingface.co/datasets/afaji/indonli) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
144
+
145
+ ## Model Details
146
+
147
+ ### Model Description
148
+ - **Model Type:** Sentence Transformer
149
+ - **Base model:** [indobenchmark/indobert-base-p2](https://huggingface.co/indobenchmark/indobert-base-p2) <!-- at revision 94b4e0a82081fa57f227fcc2024d1ea89b57ac1f -->
150
+ - **Maximum Sequence Length:** 512 tokens
151
+ - **Output Dimensionality:** 768 tokens
152
+ - **Similarity Function:** Cosine Similarity
153
+ - **Training Dataset:**
154
+ - [afaji/indonli](https://huggingface.co/datasets/afaji/indonli)
155
+ - **Language:** id
156
+ <!-- - **License:** Unknown -->
157
+
158
+ ### Model Sources
159
+
160
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
161
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
162
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
163
+
164
+ ### Full Model Architecture
165
+
166
+ ```
167
+ SentenceTransformer(
168
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
169
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
170
+ )
171
+ ```
172
+
173
+ ## Usage
174
+
175
+ ### Direct Usage (Sentence Transformers)
176
+
177
+ First install the Sentence Transformers library:
178
+
179
+ ```bash
180
+ pip install -U sentence-transformers
181
+ ```
182
+
183
+ Then you can load this model and run inference.
184
+ ```python
185
+ from sentence_transformers import SentenceTransformer
186
+
187
+ # Download from the 🤗 Hub
188
+ model = SentenceTransformer("cassador/2bs32lr2")
189
+ # Run inference
190
+ sentences = [
191
+ 'Tujuan dari acara dengar pendapat CRTC adalah untuk mengumpulkan respons dari pada pemangku kepentingan industri ini dan dari masyarakat umum.',
192
+ 'Masyarakat umum dilibatkan untuk memberikan respon dalam acara dengar pendapat CRTC.',
193
+ 'Pembuat Rooms hanya bisa membuat meeting yang terbuka.',
194
+ ]
195
+ embeddings = model.encode(sentences)
196
+ print(embeddings.shape)
197
+ # [3, 768]
198
+
199
+ # Get the similarity scores for the embeddings
200
+ similarities = model.similarity(embeddings, embeddings)
201
+ print(similarities.shape)
202
+ # [3, 3]
203
+ ```
204
+
205
+ <!--
206
+ ### Direct Usage (Transformers)
207
+
208
+ <details><summary>Click to see the direct usage in Transformers</summary>
209
+
210
+ </details>
211
+ -->
212
+
213
+ <!--
214
+ ### Downstream Usage (Sentence Transformers)
215
+
216
+ You can finetune this model on your own dataset.
217
+
218
+ <details><summary>Click to expand</summary>
219
+
220
+ </details>
221
+ -->
222
+
223
+ <!--
224
+ ### Out-of-Scope Use
225
+
226
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
227
+ -->
228
+
229
+ ## Evaluation
230
+
231
+ ### Metrics
232
+
233
+ #### Semantic Similarity
234
+ * Dataset: `sts-dev`
235
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
236
+
237
+ | Metric | Value |
238
+ |:--------------------|:-----------|
239
+ | pearson_cosine | 0.583 |
240
+ | **spearman_cosine** | **0.5605** |
241
+ | pearson_manhattan | 0.5704 |
242
+ | spearman_manhattan | 0.55 |
243
+ | pearson_euclidean | 0.5753 |
244
+ | spearman_euclidean | 0.5524 |
245
+ | pearson_dot | 0.562 |
246
+ | spearman_dot | 0.5624 |
247
+ | pearson_max | 0.583 |
248
+ | spearman_max | 0.5624 |
249
+
250
+ #### Semantic Similarity
251
+ * Dataset: `sts-test`
252
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
253
+
254
+ | Metric | Value |
255
+ |:--------------------|:----------|
256
+ | pearson_cosine | 0.2766 |
257
+ | **spearman_cosine** | **0.254** |
258
+ | pearson_manhattan | 0.2289 |
259
+ | spearman_manhattan | 0.2295 |
260
+ | pearson_euclidean | 0.2377 |
261
+ | spearman_euclidean | 0.2323 |
262
+ | pearson_dot | 0.2931 |
263
+ | spearman_dot | 0.2816 |
264
+ | pearson_max | 0.2931 |
265
+ | spearman_max | 0.2816 |
266
+
267
+ <!--
268
+ ## Bias, Risks and Limitations
269
+
270
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
271
+ -->
272
+
273
+ <!--
274
+ ### Recommendations
275
+
276
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
277
+ -->
278
+
279
+ ## Training Details
280
+
281
+ ### Training Dataset
282
+
283
+ #### afaji/indonli
284
+
285
+ * Dataset: [afaji/indonli](https://huggingface.co/datasets/afaji/indonli)
286
+ * Size: 6,915 training samples
287
+ * Columns: <code>premise</code>, <code>hypothesis</code>, and <code>label</code>
288
+ * Approximate statistics based on the first 1000 samples:
289
+ | | premise | hypothesis | label |
290
+ |:--------|:------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
291
+ | type | string | string | int |
292
+ | details | <ul><li>min: 12 tokens</li><li>mean: 29.26 tokens</li><li>max: 135 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.13 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>0: ~51.00%</li><li>1: ~49.00%</li></ul> |
293
+ * Samples:
294
+ | premise | hypothesis | label |
295
+ |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------|:---------------|
296
+ | <code>Presiden Joko Widodo (Jokowi) menyampaikan prediksi bahwa wabah virus Corona (COVID-19) di Indonesia akan selesai akhir tahun ini.</code> | <code>Prediksi akhir wabah tidak disampaikan Jokowi.</code> | <code>0</code> |
297
+ | <code>Meski biasanya hanya digunakan di fasilitas kesehatan, saat ini masker dan sarung tangan sekali pakai banyak dipakai di tingkat rumah tangga.</code> | <code>Masker sekali pakai banyak dipakai di tingkat rumah tangga.</code> | <code>1</code> |
298
+ | <code>Seperti namanya, paket internet sahur Telkomsel ini ditujukan bagi pengguna yang menginginkan kuota ekstra, untuk menemani momen sahur sepanjang bulan puasa.</code> | <code>Paket internet sahur tidak ditujukan untuk saat sahur.</code> | <code>0</code> |
299
+ * Loss: [<code>SoftmaxLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#softmaxloss)
300
+
301
+ ### Evaluation Dataset
302
+
303
+ #### afaji/indonli
304
+
305
+ * Dataset: [afaji/indonli](https://huggingface.co/datasets/afaji/indonli)
306
+ * Size: 1,556 evaluation samples
307
+ * Columns: <code>premise</code>, <code>hypothesis</code>, and <code>label</code>
308
+ * Approximate statistics based on the first 1000 samples:
309
+ | | premise | hypothesis | label |
310
+ |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
311
+ | type | string | string | int |
312
+ | details | <ul><li>min: 9 tokens</li><li>mean: 28.07 tokens</li><li>max: 179 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.15 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>0: ~47.90%</li><li>1: ~52.10%</li></ul> |
313
+ * Samples:
314
+ | premise | hypothesis | label |
315
+ |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------|:---------------|
316
+ | <code>Manuskrip tersebut berisi tiga catatan yang menceritakan bagaimana peristiwa jatuhnya meteorit serta laporan kematian akibat kejadian tersebut seperti dilansir dari Science Alert, Sabtu (25/4/2020).</code> | <code>Manuskrip tersebut tidak mencatat laporan kematian.</code> | <code>0</code> |
317
+ | <code>Dilansir dari Business Insider, menurut observasi dari Mauna Loa Observatory di Hawaii pada karbon dioksida (CO2) di level mencapai 410 ppm tidak langsung memberikan efek pada pernapasan, karena tubuh manusia juga masih membutuhkan CO2 dalam kadar tertentu.</code> | <code>Tidak ada observasi yang pernah dilansir oleh Business Insider.</code> | <code>0</code> |
318
+ | <code>Seorang wanita asal New York mengaku sangat benci air putih.</code> | <code>Tidak ada orang dari New York yang membenci air putih.</code> | <code>0</code> |
319
+ * Loss: [<code>SoftmaxLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#softmaxloss)
320
+
321
+ ### Training Hyperparameters
322
+ #### Non-Default Hyperparameters
323
+
324
+ - `eval_strategy`: epoch
325
+ - `per_device_train_batch_size`: 32
326
+ - `per_device_eval_batch_size`: 32
327
+ - `learning_rate`: 2e-05
328
+ - `num_train_epochs`: 2
329
+ - `warmup_ratio`: 0.1
330
+ - `fp16`: True
331
+
332
+ #### All Hyperparameters
333
+ <details><summary>Click to expand</summary>
334
+
335
+ - `overwrite_output_dir`: False
336
+ - `do_predict`: False
337
+ - `eval_strategy`: epoch
338
+ - `prediction_loss_only`: True
339
+ - `per_device_train_batch_size`: 32
340
+ - `per_device_eval_batch_size`: 32
341
+ - `per_gpu_train_batch_size`: None
342
+ - `per_gpu_eval_batch_size`: None
343
+ - `gradient_accumulation_steps`: 1
344
+ - `eval_accumulation_steps`: None
345
+ - `learning_rate`: 2e-05
346
+ - `weight_decay`: 0.0
347
+ - `adam_beta1`: 0.9
348
+ - `adam_beta2`: 0.999
349
+ - `adam_epsilon`: 1e-08
350
+ - `max_grad_norm`: 1.0
351
+ - `num_train_epochs`: 2
352
+ - `max_steps`: -1
353
+ - `lr_scheduler_type`: linear
354
+ - `lr_scheduler_kwargs`: {}
355
+ - `warmup_ratio`: 0.1
356
+ - `warmup_steps`: 0
357
+ - `log_level`: passive
358
+ - `log_level_replica`: warning
359
+ - `log_on_each_node`: True
360
+ - `logging_nan_inf_filter`: True
361
+ - `save_safetensors`: True
362
+ - `save_on_each_node`: False
363
+ - `save_only_model`: False
364
+ - `restore_callback_states_from_checkpoint`: False
365
+ - `no_cuda`: False
366
+ - `use_cpu`: False
367
+ - `use_mps_device`: False
368
+ - `seed`: 42
369
+ - `data_seed`: None
370
+ - `jit_mode_eval`: False
371
+ - `use_ipex`: False
372
+ - `bf16`: False
373
+ - `fp16`: True
374
+ - `fp16_opt_level`: O1
375
+ - `half_precision_backend`: auto
376
+ - `bf16_full_eval`: False
377
+ - `fp16_full_eval`: False
378
+ - `tf32`: None
379
+ - `local_rank`: 0
380
+ - `ddp_backend`: None
381
+ - `tpu_num_cores`: None
382
+ - `tpu_metrics_debug`: False
383
+ - `debug`: []
384
+ - `dataloader_drop_last`: False
385
+ - `dataloader_num_workers`: 0
386
+ - `dataloader_prefetch_factor`: None
387
+ - `past_index`: -1
388
+ - `disable_tqdm`: False
389
+ - `remove_unused_columns`: True
390
+ - `label_names`: None
391
+ - `load_best_model_at_end`: False
392
+ - `ignore_data_skip`: False
393
+ - `fsdp`: []
394
+ - `fsdp_min_num_params`: 0
395
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
396
+ - `fsdp_transformer_layer_cls_to_wrap`: None
397
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
398
+ - `deepspeed`: None
399
+ - `label_smoothing_factor`: 0.0
400
+ - `optim`: adamw_torch
401
+ - `optim_args`: None
402
+ - `adafactor`: False
403
+ - `group_by_length`: False
404
+ - `length_column_name`: length
405
+ - `ddp_find_unused_parameters`: None
406
+ - `ddp_bucket_cap_mb`: None
407
+ - `ddp_broadcast_buffers`: False
408
+ - `dataloader_pin_memory`: True
409
+ - `dataloader_persistent_workers`: False
410
+ - `skip_memory_metrics`: True
411
+ - `use_legacy_prediction_loop`: False
412
+ - `push_to_hub`: False
413
+ - `resume_from_checkpoint`: None
414
+ - `hub_model_id`: None
415
+ - `hub_strategy`: every_save
416
+ - `hub_private_repo`: False
417
+ - `hub_always_push`: False
418
+ - `gradient_checkpointing`: False
419
+ - `gradient_checkpointing_kwargs`: None
420
+ - `include_inputs_for_metrics`: False
421
+ - `eval_do_concat_batches`: True
422
+ - `fp16_backend`: auto
423
+ - `push_to_hub_model_id`: None
424
+ - `push_to_hub_organization`: None
425
+ - `mp_parameters`:
426
+ - `auto_find_batch_size`: False
427
+ - `full_determinism`: False
428
+ - `torchdynamo`: None
429
+ - `ray_scope`: last
430
+ - `ddp_timeout`: 1800
431
+ - `torch_compile`: False
432
+ - `torch_compile_backend`: None
433
+ - `torch_compile_mode`: None
434
+ - `dispatch_batches`: None
435
+ - `split_batches`: None
436
+ - `include_tokens_per_second`: False
437
+ - `include_num_input_tokens_seen`: False
438
+ - `neftune_noise_alpha`: None
439
+ - `optim_target_modules`: None
440
+ - `batch_eval_metrics`: False
441
+ - `batch_sampler`: batch_sampler
442
+ - `multi_dataset_batch_sampler`: proportional
443
+
444
+ </details>
445
+
446
+ ### Training Logs
447
+ | Epoch | Step | Training Loss | loss | sts-dev_spearman_cosine | sts-test_spearman_cosine |
448
+ |:------:|:----:|:-------------:|:------:|:-----------------------:|:------------------------:|
449
+ | 0 | 0 | - | - | 0.1277 | - |
450
+ | 0.4608 | 100 | 0.5694 | - | - | - |
451
+ | 0.9217 | 200 | 0.4754 | - | - | - |
452
+ | 1.0 | 217 | - | 0.4349 | 0.5410 | - |
453
+ | 1.3825 | 300 | 0.3829 | - | - | - |
454
+ | 1.8433 | 400 | 0.3507 | - | - | - |
455
+ | 2.0 | 434 | - | 0.4254 | 0.5605 | 0.2540 |
456
+
457
+
458
+ ### Framework Versions
459
+ - Python: 3.10.12
460
+ - Sentence Transformers: 3.0.1
461
+ - Transformers: 4.41.2
462
+ - PyTorch: 2.3.0+cu121
463
+ - Accelerate: 0.31.0
464
+ - Datasets: 2.20.0
465
+ - Tokenizers: 0.19.1
466
+
467
+ ## Citation
468
+
469
+ ### BibTeX
470
+
471
+ #### Sentence Transformers and SoftmaxLoss
472
+ ```bibtex
473
+ @inproceedings{reimers-2019-sentence-bert,
474
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
475
+ author = "Reimers, Nils and Gurevych, Iryna",
476
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
477
+ month = "11",
478
+ year = "2019",
479
+ publisher = "Association for Computational Linguistics",
480
+ url = "https://arxiv.org/abs/1908.10084",
481
+ }
482
+ ```
483
+
484
+ <!--
485
+ ## Glossary
486
+
487
+ *Clearly define terms in order to be accessible across audiences.*
488
+ -->
489
+
490
+ <!--
491
+ ## Model Card Authors
492
+
493
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
494
+ -->
495
+
496
+ <!--
497
+ ## Model Card Contact
498
+
499
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
500
+ -->
config.json ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "indobenchmark/indobert-base-p2",
3
+ "_num_labels": 5,
4
+ "architectures": [
5
+ "BertModel"
6
+ ],
7
+ "attention_probs_dropout_prob": 0.1,
8
+ "classifier_dropout": null,
9
+ "directionality": "bidi",
10
+ "hidden_act": "gelu",
11
+ "hidden_dropout_prob": 0.1,
12
+ "hidden_size": 768,
13
+ "id2label": {
14
+ "0": "LABEL_0",
15
+ "1": "LABEL_1",
16
+ "2": "LABEL_2",
17
+ "3": "LABEL_3",
18
+ "4": "LABEL_4"
19
+ },
20
+ "initializer_range": 0.02,
21
+ "intermediate_size": 3072,
22
+ "label2id": {
23
+ "LABEL_0": 0,
24
+ "LABEL_1": 1,
25
+ "LABEL_2": 2,
26
+ "LABEL_3": 3,
27
+ "LABEL_4": 4
28
+ },
29
+ "layer_norm_eps": 1e-12,
30
+ "max_position_embeddings": 512,
31
+ "model_type": "bert",
32
+ "num_attention_heads": 12,
33
+ "num_hidden_layers": 12,
34
+ "output_past": true,
35
+ "pad_token_id": 0,
36
+ "pooler_fc_size": 768,
37
+ "pooler_num_attention_heads": 12,
38
+ "pooler_num_fc_layers": 3,
39
+ "pooler_size_per_head": 128,
40
+ "pooler_type": "first_token_transform",
41
+ "position_embedding_type": "absolute",
42
+ "torch_dtype": "float32",
43
+ "transformers_version": "4.41.2",
44
+ "type_vocab_size": 2,
45
+ "use_cache": true,
46
+ "vocab_size": 50000
47
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.0.1",
4
+ "transformers": "4.41.2",
5
+ "pytorch": "2.3.0+cu121"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6670ee08f867384186aa107882dca0d2c5c68cceb93ef4546f3a956f83368e4c
3
+ size 497787752
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "4": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 1000000000000000019884624838656,
50
+ "never_split": null,
51
+ "pad_token": "[PAD]",
52
+ "sep_token": "[SEP]",
53
+ "strip_accents": null,
54
+ "tokenize_chinese_chars": true,
55
+ "tokenizer_class": "BertTokenizer",
56
+ "unk_token": "[UNK]"
57
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff