End of training

README.md

@@ -0,0 +1,79 @@
+---
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6832
+- Answer: {'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809}
+- Header: {'precision': 0.31932773109243695, 'recall': 0.31932773109243695, 'f1': 0.31932773109243695, 'number': 119}
+- Question: {'precision': 0.784366576819407, 'recall': 0.819718309859155, 'f1': 0.8016528925619835, 'number': 1065}
+- Overall Precision: 0.7259
+- Overall Recall: 0.7812
+- Overall F1: 0.7525
+- Overall Accuracy: 0.8047
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                         | Header                                                                                                      | Question                                                                                                     | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.8109        | 1.0   | 10   | 1.6140          | {'precision': 0.013268998793727383, 'recall': 0.013597033374536464, 'f1': 0.013431013431013432, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.10519645120405577, 'recall': 0.07793427230046948, 'f1': 0.08953613807982741, 'number': 1065} | 0.0581            | 0.0472         | 0.0521     | 0.3634           |
+| 1.468         | 2.0   | 20   | 1.2385          | {'precision': 0.13105413105413105, 'recall': 0.11372064276885044, 'f1': 0.12177365982792852, 'number': 809}    | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.4378029079159935, 'recall': 0.5089201877934272, 'f1': 0.4706904038211029, 'number': 1065}    | 0.3268            | 0.3181         | 0.3224     | 0.5793           |
+| 1.0973        | 3.0   | 30   | 0.9328          | {'precision': 0.41563275434243174, 'recall': 0.41409147095179233, 'f1': 0.41486068111455104, 'number': 809}    | {'precision': 0.03125, 'recall': 0.008403361344537815, 'f1': 0.013245033112582781, 'number': 119}           | {'precision': 0.6483720930232558, 'recall': 0.6544600938967137, 'f1': 0.6514018691588785, 'number': 1065}    | 0.5400            | 0.5183         | 0.5289     | 0.7016           |
+| 0.8233        | 4.0   | 40   | 0.7582          | {'precision': 0.6106290672451193, 'recall': 0.695920889987639, 'f1': 0.6504910456383594, 'number': 809}        | {'precision': 0.17543859649122806, 'recall': 0.08403361344537816, 'f1': 0.11363636363636363, 'number': 119} | {'precision': 0.6941176470588235, 'recall': 0.72018779342723, 'f1': 0.7069124423963133, 'number': 1065}      | 0.6430            | 0.6724         | 0.6573     | 0.7595           |
+| 0.6573        | 5.0   | 50   | 0.6894          | {'precision': 0.6411332633788038, 'recall': 0.7552533992583437, 'f1': 0.6935300794551646, 'number': 809}       | {'precision': 0.2696629213483146, 'recall': 0.20168067226890757, 'f1': 0.23076923076923078, 'number': 119}  | {'precision': 0.7267080745341615, 'recall': 0.7690140845070422, 'f1': 0.7472627737226277, 'number': 1065}    | 0.6704            | 0.7296         | 0.6987     | 0.7838           |
+| 0.5504        | 6.0   | 60   | 0.6623          | {'precision': 0.6652675760755509, 'recall': 0.7836835599505563, 'f1': 0.7196367763904654, 'number': 809}       | {'precision': 0.19736842105263158, 'recall': 0.12605042016806722, 'f1': 0.15384615384615385, 'number': 119} | {'precision': 0.731626754748142, 'recall': 0.831924882629108, 'f1': 0.7785588752196836, 'number': 1065}      | 0.6853            | 0.7702         | 0.7253     | 0.7933           |
+| 0.4731        | 7.0   | 70   | 0.6464          | {'precision': 0.6681127982646421, 'recall': 0.761433868974042, 'f1': 0.7117273252455227, 'number': 809}        | {'precision': 0.22641509433962265, 'recall': 0.20168067226890757, 'f1': 0.21333333333333335, 'number': 119} | {'precision': 0.7656794425087108, 'recall': 0.8253521126760563, 'f1': 0.7943967464979665, 'number': 1065}    | 0.6981            | 0.7622         | 0.7287     | 0.8003           |
+| 0.428         | 8.0   | 80   | 0.6407          | {'precision': 0.6865671641791045, 'recall': 0.796044499381953, 'f1': 0.7372638809387521, 'number': 809}        | {'precision': 0.22321428571428573, 'recall': 0.21008403361344538, 'f1': 0.21645021645021645, 'number': 119} | {'precision': 0.7692307692307693, 'recall': 0.8262910798122066, 'f1': 0.7967406066093254, 'number': 1065}    | 0.7060            | 0.7772         | 0.7399     | 0.8053           |
+| 0.3776        | 9.0   | 90   | 0.6475          | {'precision': 0.7108843537414966, 'recall': 0.7750309023485785, 'f1': 0.7415730337078651, 'number': 809}       | {'precision': 0.23770491803278687, 'recall': 0.24369747899159663, 'f1': 0.24066390041493776, 'number': 119} | {'precision': 0.7615780445969125, 'recall': 0.8338028169014085, 'f1': 0.796055580457194, 'number': 1065}     | 0.7115            | 0.7747         | 0.7418     | 0.8022           |
+| 0.3434        | 10.0  | 100  | 0.6694          | {'precision': 0.6895074946466809, 'recall': 0.796044499381953, 'f1': 0.7389558232931727, 'number': 809}        | {'precision': 0.2831858407079646, 'recall': 0.2689075630252101, 'f1': 0.27586206896551724, 'number': 119}   | {'precision': 0.7693661971830986, 'recall': 0.8206572769953052, 'f1': 0.79418446160836, 'number': 1065}      | 0.7100            | 0.7777         | 0.7423     | 0.8007           |
+| 0.3082        | 11.0  | 110  | 0.6749          | {'precision': 0.6961206896551724, 'recall': 0.7985166872682324, 'f1': 0.7438111686816349, 'number': 809}       | {'precision': 0.2905982905982906, 'recall': 0.2857142857142857, 'f1': 0.288135593220339, 'number': 119}     | {'precision': 0.7794779477947795, 'recall': 0.8131455399061033, 'f1': 0.7959558823529411, 'number': 1065}    | 0.7171            | 0.7757         | 0.7452     | 0.7985           |
+| 0.2933        | 12.0  | 120  | 0.6635          | {'precision': 0.7130242825607064, 'recall': 0.7985166872682324, 'f1': 0.7533527696793003, 'number': 809}       | {'precision': 0.28448275862068967, 'recall': 0.2773109243697479, 'f1': 0.28085106382978725, 'number': 119}  | {'precision': 0.78125, 'recall': 0.8215962441314554, 'f1': 0.8009153318077803, 'number': 1065}               | 0.7255            | 0.7797         | 0.7516     | 0.8056           |
+| 0.278         | 13.0  | 130  | 0.6760          | {'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809}        | {'precision': 0.3170731707317073, 'recall': 0.3277310924369748, 'f1': 0.32231404958677684, 'number': 119}   | {'precision': 0.7897111913357401, 'recall': 0.8215962441314554, 'f1': 0.8053382420616658, 'number': 1065}    | 0.7297            | 0.7827         | 0.7553     | 0.8049           |
+| 0.2699        | 14.0  | 140  | 0.6824          | {'precision': 0.7041484716157205, 'recall': 0.7972805933250927, 'f1': 0.7478260869565218, 'number': 809}       | {'precision': 0.3305785123966942, 'recall': 0.33613445378151263, 'f1': 0.33333333333333337, 'number': 119}  | {'precision': 0.7845601436265709, 'recall': 0.8206572769953052, 'f1': 0.8022028453419, 'number': 1065}       | 0.7248            | 0.7822         | 0.7524     | 0.8045           |
+| 0.2645        | 15.0  | 150  | 0.6832          | {'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809}       | {'precision': 0.31932773109243695, 'recall': 0.31932773109243695, 'f1': 0.31932773109243695, 'number': 119} | {'precision': 0.784366576819407, 'recall': 0.819718309859155, 'f1': 0.8016528925619835, 'number': 1065}      | 0.7259            | 0.7812         | 0.7525     | 0.8047           |
+
+
+### Framework versions
+
+- Transformers 4.21.3
+- Pytorch 1.12.1+cu102
+- Datasets 2.4.0
+- Tokenizers 0.12.1

logs/events.out.tfevents.1665152364.ip-172-30-1-140.ec2.internal.18409.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7d08e12ea593cee782c070a384b86fcef6568c839dc2061090ec9311f7cb8f8b
-size 12385
+oid sha256:e010cb43a87b941a0a9c01c54441b62d567b395e94331f1ed1097caace3b3410
+size 14061

preprocessor_config.json

@@ -0,0 +1,9 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": 224
+}

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": null,
+  "apply_ocr": false,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "name_or_path": "microsoft/layoutlmv2-base-uncased",
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "special_tokens_map_file": null,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}