--- language: en license: apache-2.0 tags: - sagemaker - bart - summarization datasets: - samsum widget: - text: "Jeff: Can I train a \U0001F917 Transformers model on Amazon SageMaker? \n\ Philipp: Sure you can use the new Hugging Face Deep Learning Container. \nJeff:\ \ ok.\nJeff: and how can I get started? \nJeff: where can I find documentation?\ \ \nPhilipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face " model-index: - name: philschmid/distilbart-cnn-12-6-samsum results: - task: type: summarization name: Summarization dataset: name: samsum type: samsum config: samsum split: test metrics: - type: rouge value: 41.0895 name: ROUGE-1 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTBlZmQzZDFmNzY5YTBjMTI3ZjRkNDk4NTI3YzQxOGY4MjlkMTU4ZGJkZWE4YjQ4ZDFhOTIxM2M1YWEyMDQ4MCIsInZlcnNpb24iOjF9.Nw7idRmEmjS-c91HthjVGw6YxttVA_tRB2QRkGwVSVABR3_BY84HvwLOZVstc6a9gUHopMj_W9SRfa_6xTWcBA - type: rouge value: 20.7459 name: ROUGE-2 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjRhN2E4ZDNiNWNkY2RkNjMxMjhiYjcxYjc4OWM4NWQ5MDNjZDMwOGIwNWI2NWJiYzljMzc0NzY1ZDBmMTRmZCIsInZlcnNpb24iOjF9.nYxNimi33AW0T8T1JhqFUukxe4W4niXj4UzLRTuc40NeZveDTSpPS8QzR4rF1gK-r2irqIX5FrvG4dwQHrESBA - type: rouge value: 31.5952 name: ROUGE-L verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjYzYTRjOTU1MDVhN2ZlOGE3YTIxMjk1NDBiY2E2ZWI1MDA5ZTJkOTQ4MzgxNThkOGU4OTUzODU0YWE1OTQ5MiIsInZlcnNpb24iOjF9.G2EtxIlJ86AcNx2bqw2nu1UbdczQ-anl1c02EopQyC81BEcEAbnY-liPvHXLjPVQvP97GGGjqTDLZYjYJ71hDQ - type: rouge value: 38.3389 name: ROUGE-LSUM verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWExZmM2YWI1MDU0ZjE1YWY1NDNiY2E4YTkwZjA5MTE0YmM1NzI4YTc1YjI0MWFmZDlkYTBlZjVlMDk2ZGQxZSIsInZlcnNpb24iOjF9.jjBghJ66Gj_95AdDpWG2TR_MnuUtj8Fzc0M3KS9vqsM0iqtlu9khY8lXrFpMaIeDxVBYKltMMFdZWH8mVv2wCg - type: loss value: 1.4566329717636108 name: loss verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDAyYjFmNzRlYzkyNmUxMWM5YWYxNjgwZGQ1ZDc1Njg0MzU1ZjM4MjI5MTJlODRiNDdiNGRjYTkzZGUxNGMyZiIsInZlcnNpb24iOjF9.2eH5b7DlPeVQ_zFGlvKyRvqrc7yyT8vcf3koJGKGysV00vCQew8sOmFEmDegiBka8gq3UL987Dd2yZCU3b64Cw - type: gen_len value: 59.6032 name: gen_len verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNTFkOTlmMjFlODIxZWExZmQyMTQyNDYyY2RmNjY3YmM1NDFlNTI3OTRhZjU0MDQxNGY2NGQ4ZWY2OWFmNDliMyIsInZlcnNpb24iOjF9.K9qwFg3Flnu2-1H-WI9adj7yoBuJ3zBBDyda5BxRpJ1D4L_alLpCweqrVGuynOPl9PAWPuHo7bAG1y2zZNmmDw - task: type: summarization name: Summarization dataset: name: xsum type: xsum config: default split: test metrics: - type: rouge value: 21.1644 name: ROUGE-1 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzVjNWI1MWJmOTAzMDYxNjlkYzg4NjM3MzczMDJiYjNjYTg3NjYzNjQ3YTczYzg5MGU2NDcxOWQwZjdlODU4YSIsInZlcnNpb24iOjF9.CqB-ANpnx0GvwhsjeCzLB_RxaKqbnhc_980RG8fqDb2hNTk4LvDhqdDfkLFQMj8kvW4nQLLDSNUENQ7Uni9kCA - type: rouge value: 4.0659 name: ROUGE-2 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjgzYTc5Y2VjZTliNmEwYTZjNTViODdjYTg2MGMyZTFkMjgyMjk0OGIzZjg1ODgzZDJmOTMwNmU4MzdlNTI2OCIsInZlcnNpb24iOjF9.1AfPtrpJ38Khz5vfRsN4Jwb3J_PdycddRH9DJtEccqmEz9BzDo-AO7Ts94sfVlYfSf3srplLHDcd_XFCwQtlBg - type: rouge value: 13.9414 name: ROUGE-L verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODZlZTM2OTM5NzVlNDBjYjc2NDdhNmIxNDZhNGVjOTk4YzIzZmEwYzEwZmQ2ZmNmMjIzNzYwMzkxMjU1ODcxMCIsInZlcnNpb24iOjF9.vvp5PKmEp-Hyt46zgsvzjGOO8wrV0cDG68Z0VPqW2WfY5Sp3k3krEcKLATdQAQjfy96gKCCkQpBFefpjYWcmDA - type: rouge value: 17.0718 name: ROUGE-LSUM verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWEyN2QyOWFhMjAzZTk4NjU5ODU5YjgxMjczNzc1MTM5OTY1OTVjZDMwZDhjODFlZTVmODNkYzFhOTc5NzZhYiIsInZlcnNpb24iOjF9.PwJT3EYTV3KifWaySiwSTxGyBWTB8bHMuaXG3AyRvWkY2xju1BSaBjPGCcfmlZs1yJwghOH7N4dBW5yJBEp5DA - type: loss value: 3.002755880355835 name: loss verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGY0NjgzOGExZWZmNWI3ZTA1ZjRlYTU2OTZkYzk5NDdmOGVmNDdkNWU5YjViMWQyNGE2MTNkODhkZmQ1ZGE2OSIsInZlcnNpb24iOjF9.pWru9Nhl0aZThHz0qveOHmxTOCrZjHu9ySt5wI9MnGQ5ZEpxfufjpI196EMMn-KSSxAl-s7wHygtGC9_WtC1BQ - type: gen_len value: 71.2969 name: gen_len verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiY2Q4ODZlZDY4YzE1MDEwNzJjNmNiYjI4NTU5NjRlNmY2NWNjYmUwNzcwZmY3NWVlZDA3NTMxY2Y2NWI1ODYwMCIsInZlcnNpb24iOjF9.0Y_lnTKQ5nmjnwAEju9T7xlLObWgwPLMOxlWDpjPBkDeW0bzHYqJcRADtFcvAhznJ3HktIV830QxjqkRYjZTDw - task: type: summarization name: Summarization dataset: name: cnn_dailymail type: cnn_dailymail config: 3.0.0 split: test metrics: - type: rouge value: 42.9764 name: ROUGE-1 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzQ0NzAyNDM5YWIyZmY5OTUwNWZjNzRkYTliYWI0YzdmZTg5ZGVhZjJlMDBiZjg2YmE0ZjkxNzU4YjRkYTJjNCIsInZlcnNpb24iOjF9.dymlXdITNpMZpOaYvif-LcxaSRWKh8_RxV6mdBpuvlThPPi3-TwKCW20Fowor8H5RPsC0M1cfvNNzINCyApKCA - type: rouge value: 19.8711 name: ROUGE-2 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTY5ZTI0OTEzNzI1NDRjYWE5MDk2NzVhZDEyNGEzNzU3OTE0NmJmYzhkMTU0MTI2MDVlOTdlNGU0OGI2MTdlNSIsInZlcnNpb24iOjF9.7xN2u4HjPL78CkkihB9I0befTn04IQqimvNlSHpc888arBm_qCtTGl7q7389ArpWUKEdkhvZ94BgB-Z_cXsqCg - type: rouge value: 29.5196 name: ROUGE-L verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGY3YzRiYjljZWJhNGJiMTdhNDY5OTk0ODJhYWMxOGMwYjY4ZTBlNmQ0YTUxMzQ3YmZiNzg0ZDJiNTg4MzdiYyIsInZlcnNpb24iOjF9.Yj0ZaJelYcMJ-8SIon9x7GxRityWR3p0vcLNctTfcg6eCClalTQKBclCVgpDCO8WQyVxSz8EyCDb2qedRgF9CQ - type: rouge value: 39.959 name: ROUGE-LSUM verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmI2MTE3NjQ2M2MwYjJmN2JmMTEwN2JkMDVhMWRiMDBiM2FlMzM5NjAxYWM1N2Q2MTA5MGRhZmI5ZDNjNzMwYSIsInZlcnNpb24iOjF9.A-2Ch4-M691OBAp4KmsYut10K3sF0fjw5ztutK_LTtn68Ne0x8w-u-7pEyjTuWJrJx4Q3Yb1eW3yeHPTnFI0DA - type: loss value: 3.014679193496704 name: loss verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmNiNWZlOGZmOWZhNWZjNTg1ODIzMzY2ZGYxNzBiOWIxMWRiZTQ5MzM5NjRlYTQwMzRjOGI3YzNmZDhmYjQ3MCIsInZlcnNpb24iOjF9.BZkiJxZG0RdFzNgxgcS8U6_zPT1t7rvs-603cnC1tjMMYF3Lbae7rExRb-fVHN_ofZV_w5vl4uRLQ3OxZUY5Ag - type: gen_len value: 81.956 name: gen_len verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTdhNGYxYWRlNTU0MzAxMWU1NzNmMTBjMmY3NzkzODAyYTMzZWYyZmNiMTViMzNmYTE0ZmFmNDdhMzQwMmJkNyIsInZlcnNpb24iOjF9.8lm84JtbCh-diuNQ01oXK6P8vV9CPyA8y-7D9o_OHb9Vk3pNEFM1jMSZVdEG9wFuMpWL3ARbXLadEPQB5HN8AQ - task: type: summarization name: Summarization dataset: name: banking77 type: banking77 config: default split: train metrics: - type: rouge value: 0.0004 name: ROUGE-1 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWRhYTYwZjAzZTUxYjE5MThhODZiM2QzODcyYTU2ODRmNjgzNTg4YTMyMTVlZGE4N2ZlYzk0MzgxZTgzOGM5MSIsInZlcnNpb24iOjF9.Bzhpi1LGPsx5dEHUbTFlBDM5je5k1TkEHnNzANAYeyrRYs032xJTx8fmjxTnHUfikGxkAhe5Xw3JCWBtsMShCA - type: rouge value: 0.0 name: ROUGE-2 verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWQ2NDNlYzE1NDFmOTZmYjc4OTlkYTIyOWVlOTJmMmM3ZWQzOWJkYWYxYzZhNmU0ZWZhNDgwNjM3MDhkNDRlZiIsInZlcnNpb24iOjF9.xTGMh_3YQKmx1HVTfhHS7lm_4zY4bu6IUaxAcA5JinRVn7w6lxmuqoH5D9b_OnYQYEYh1TAr4QG4Twq0MFWWCQ - type: rouge value: 0.0004 name: ROUGE-L verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjVhMjE2ZGIyNjg3MjQwNjNlYWNhZTk1NDY5NmJlOTFiMjM2MzU4Yjg4ZGE2NTE5YTQ3YmY4NDIwMTg0ZjZkNyIsInZlcnNpb24iOjF9.CvGzeC-pH5KQnjrvDM1LIPsDFFSFOSCGWQPUJHIzMz9eyfWskt7Vry2P9EdKgufN8TELnwfAvqDSBXafgoljAA - type: rouge value: 0.0004 name: ROUGE-LSUM verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiN2IzZjQ0NThmMmIxOWZhYjdiY2ZjYzZmMDBiNjAxZTMwM2MzZjQ5NDA5Y2ZmYTgxYjFiMmM1ZTAyZGFkZmNmMSIsInZlcnNpb24iOjF9.5mj1n2uFX1FTP9oKbRVB43gakUuAOXakia6f0ZIVpt7BtUXWS2GdTz2cTd2kCaIua633iGdsYZEl-kmeYysiBw - type: loss value: 7.779788494110107 name: loss verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODUwZGNiMGI4ZjEyMDVjZGRjYzc4Zjc1OTdlMzU2M2RhM2RiN2NiNmYyYTgwNDY2M2ZkNWZiMThiNjZkMmUyMCIsInZlcnNpb24iOjF9.3nt7FwKpXhgf5d_rWi1Pk6eDU27jEvaCF_c_rdhFne0S62ajO0a3BpCmWWrbdwsvhyX1zvqJ2-r0FpRDj49WDQ - type: gen_len value: 59.0362 name: gen_len verified: true verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGQ0ODI4YjU4NzdhMjNjNjI1OGY2MWMyNDA1NWMyMTU5MDg2MWQ0NmUyM2IzZjkyODhkNDc5ZmQyNWQzZGRiYSIsInZlcnNpb24iOjF9._4SyY434kEelLfRIwf-MvRvZkc3NuGwk6S5IRAhxWHhbJgXp-6qGgSAkgs3GLnQkA5PXdVPy3oSQ91LGdqrnDg --- ## `distilbart-cnn-12-6-samsum` This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container. For more information look at: - [🤗 Transformers Documentation: Amazon SageMaker](https://huggingface.co/transformers/sagemaker.html) - [Example Notebooks](https://github.com/huggingface/notebooks/tree/master/sagemaker) - [Amazon SageMaker documentation for Hugging Face](https://docs.aws.amazon.com/sagemaker/latest/dg/hugging-face.html) - [Python SDK SageMaker documentation for Hugging Face](https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/index.html) - [Deep Learning Container](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-training-containers) ## Hyperparameters ```json { "dataset_name": "samsum", "do_eval": true, "do_train": true, "fp16": true, "learning_rate": 5e-05, "model_name_or_path": "sshleifer/distilbart-cnn-12-6", "num_train_epochs": 3, "output_dir": "/opt/ml/model", "per_device_eval_batch_size": 8, "per_device_train_batch_size": 8, "seed": 7 } ``` ## Train results | key | value | | --- | ----- | | epoch | 3.0 | | init_mem_cpu_alloc_delta | 180338 | | init_mem_cpu_peaked_delta | 18282 | | init_mem_gpu_alloc_delta | 1222242816 | | init_mem_gpu_peaked_delta | 0 | | train_mem_cpu_alloc_delta | 6971403 | | train_mem_cpu_peaked_delta | 640733 | | train_mem_gpu_alloc_delta | 4910897664 | | train_mem_gpu_peaked_delta | 23331969536 | | train_runtime | 155.2034 | | train_samples | 14732 | | train_samples_per_second | 2.242 | ## Eval results | key | value | | --- | ----- | | epoch | 3.0 | | eval_loss | 1.4209576845169067 | | eval_mem_cpu_alloc_delta | 868003 | | eval_mem_cpu_peaked_delta | 18250 | | eval_mem_gpu_alloc_delta | 0 | | eval_mem_gpu_peaked_delta | 328244736 | | eval_runtime | 0.6088 | | eval_samples | 818 | | eval_samples_per_second | 1343.647 | ## Usage ```python from transformers import pipeline summarizer = pipeline("summarization", model="philschmid/distilbart-cnn-12-6-samsum") conversation = '''Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker? Philipp: Sure you can use the new Hugging Face Deep Learning Container. Jeff: ok. Jeff: and how can I get started? Jeff: where can I find documentation? Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face ''' nlp(conversation) ```