{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Igc5itf-xMGj" }, "source": [ "# Masakhane - Machine Translation for African Languages (Using JoeyNMT)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "x4fXCKCf36IK" }, "source": [ "## Note before beginning:\n", "### - The idea is that you should be able to make minimal changes to this in order to get SOME result for your own translation corpus. \n", "\n", "### - The tl;dr: Go to the **\"TODO\"** comments which will tell you what to update to get up and running\n", "\n", "### - If you actually want to have a clue what you're doing, read the text and peek at the links\n", "\n", "### - With 100 epochs, it should take around 7 hours to run in Google Colab\n", "\n", "### - Once you've gotten a result for your language, please attach and email your notebook that generated it to masakhanetranslation@gmail.com\n", "\n", "### - If you care enough and get a chance, doing a brief background on your language would be amazing. See examples in [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "l929HimrxS0a" }, "source": [ "## Retrieve your data & make a parallel corpus\n", "\n", "If you are wanting to use the JW300 data referenced on the Masakhane website or in our GitHub repo, you can use `opus-tools` to convert the data into a convenient format. `opus_read` from that package provides a convenient tool for reading the native aligned XML files and to convert them to TMX format. The tool can also be used to fetch relevant files from OPUS on the fly and to filter the data as necessary. [Read the documentation](https://pypi.org/project/opustools-pkg/) for more details.\n", "\n", "Once you have your corpus files in TMX format (an xml structure which will include the sentences in your target language and your source language in a single file), we recommend reading them into a pandas dataframe. Thankfully, Jade wrote a silly `tmx2dataframe` package which converts your tmx file to a pandas dataframe. " ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "oGRmDELn7Az0" }, "outputs": [], "source": [ "# from google.colab import drive\n", "# drive.mount('/content/drive')" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "Cn3tgQLzUxwn" }, "outputs": [], "source": [ "# TODO: Set your source and target languages. Keep in mind, these traditionally use language codes as found here:\n", "# These will also become the suffix's of all vocab and corpus files used throughout\n", "import os\n", "source_language = \"en\"\n", "target_language = \"luo\" \n", "lc = False # If True, lowercase the data.\n", "seed = 42 # Random seed for shuffling.\n", "tag = \"baseline\" # Give a unique name to your folder - this is to ensure you don't rewrite any models you've already submitted\n", "\n", "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n", "os.environ[\"tgt\"] = target_language\n", "os.environ[\"tag\"] = tag\n", "\n", "# This will save it to a folder in our gdrive instead!\n", "# !mkdir -p \"/content/drive/My Drive/masakhane/$src-$tgt-$tag\"\n", "# os.environ[\"gdrive_path\"] = \"/content/drive/My Drive/masakhane/%s-%s-%s\" % (source_language, target_language, tag)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "kBSgJHEw7Nvx" }, "outputs": [], "source": [ "# !echo $gdrive_path" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "gA75Fs9ys8Y9" }, "outputs": [], "source": [ "# Install opus-tools\n", "#! pip install opustools-pkg" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Uncomment cell below if notebook is being run for the first time and you need to download the data" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": {}, "colab_type": "code", "id": "xq-tDZVks7ZD" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Alignment file /proj/nlpl/data/OPUS/JW300/latest/xml/en-luo.xml.gz not found. The following files are available for downloading:\n", "\n", " 1 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en-luo.xml.gz\n", " 263 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en.zip\n", " 14 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/luo.zip\n", "\n", " 279 MB Total size\n", "./JW300_latest_xml_en-luo.xml.gz ... 100% of 1 MB\n", "./JW300_latest_xml_en.zip ... 100% of 263 MB\n", "./JW300_latest_xml_luo.zip ... 100% of 14 MB\n" ] } ], "source": [ "# Downloading our corpus\n", "! opus_read -d JW300 -s $src -t $tgt -wm moses -w jw300.$src jw300.$tgt -q\n", "\n", "# extract the corpus file\n", "! gunzip JW300_latest_xml_$src-$tgt.xml.gz" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": {}, "colab_type": "code", "id": "n48GDRnP8y2G" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--2020-02-19 06:24:03-- https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.192.133, 151.101.128.133, 151.101.64.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.192.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 277791 (271K) [text/plain]\n", "Saving to: ‘test.en-any.en’\n", "\n", "test.en-any.en 100%[===================>] 271.28K --.-KB/s in 0.04s \n", "\n", "2020-02-19 06:24:04 (6.13 MB/s) - ‘test.en-any.en’ saved [277791/277791]\n", "\n", "--2020-02-19 06:24:04-- https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-luo.en\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.192.133, 151.101.128.133, 151.101.64.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.192.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 205584 (201K) [text/plain]\n", "Saving to: ‘test.en-luo.en’\n", "\n", "test.en-luo.en 100%[===================>] 200.77K --.-KB/s in 0.04s \n", "\n", "2020-02-19 06:24:04 (5.24 MB/s) - ‘test.en-luo.en’ saved [205584/205584]\n", "\n", "--2020-02-19 06:24:04-- https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-luo.luo\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.192.133, 151.101.128.133, 151.101.64.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.192.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 209877 (205K) [text/plain]\n", "Saving to: ‘test.en-luo.luo’\n", "\n", "test.en-luo.luo 100%[===================>] 204.96K --.-KB/s in 0.04s \n", "\n", "2020-02-19 06:24:05 (4.95 MB/s) - ‘test.en-luo.luo’ saved [209877/209877]\n", "\n" ] } ], "source": [ "# Download the global test set.\n", "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n", " \n", "# And the specific test set for this language pair.\n", "os.environ[\"trg\"] = target_language \n", "os.environ[\"src\"] = source_language \n", "\n", "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.en \n", "! mv test.en-$trg.en test.en\n", "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.$trg \n", "! mv test.en-$trg.$trg test.$trg" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "colab": {}, "colab_type": "code", "id": "NqDG-CI28y2L" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loaded 3571 global test sentences to filter from the training/dev data.\n" ] } ], "source": [ "# Read the test data to filter from train and dev splits.\n", "# Store english portion in set for quick filtering checks.\n", "en_test_sents = set()\n", "filter_test_sents = \"test.en-any.en\"\n", "j = 0\n", "with open(filter_test_sents) as f:\n", " for line in f:\n", " en_test_sents.add(line.strip())\n", " j += 1\n", "print('Loaded {} global test sentences to filter from the training/dev data.'.format(j))" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "colab": {}, "colab_type": "code", "id": "3CNdwLBCfSIl" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loaded data and skipped 4654/154941 lines since contained in test set.\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
source_sentencetarget_sentence
0Compassion in a Cruel WorldKecho Ji e Piny ma Ji Ok Dew Jowetegi
1A MAN in Burundi falls seriously ill with mala...MALARIA mager ogoyo dichwo moro e piny Burundi .
2He urgently needs to be transferred to a hospi...Dwarore otere e osiptal mapiyo ahinya .
\n", "
" ], "text/plain": [ " source_sentence \\\n", "0 Compassion in a Cruel World \n", "1 A MAN in Burundi falls seriously ill with mala... \n", "2 He urgently needs to be transferred to a hospi... \n", "\n", " target_sentence \n", "0 Kecho Ji e Piny ma Ji Ok Dew Jowetegi \n", "1 MALARIA mager ogoyo dichwo moro e piny Burundi . \n", "2 Dwarore otere e osiptal mapiyo ahinya . " ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "# TMX file to dataframe\n", "source_file = 'jw300.' + source_language\n", "target_file = 'jw300.' + target_language\n", "\n", "source = []\n", "target = []\n", "skip_lines = [] # Collect the line numbers of the source portion to skip the same lines for the target portion.\n", "with open(source_file) as f:\n", " for i, line in enumerate(f):\n", " # Skip sentences that are contained in the test set.\n", " if line.strip() not in en_test_sents:\n", " source.append(line.strip())\n", " else:\n", " skip_lines.append(i) \n", "with open(target_file) as f:\n", " for j, line in enumerate(f):\n", " # Only add to corpus if corresponding source was not skipped.\n", " if j not in skip_lines:\n", " target.append(line.strip())\n", " \n", "print('Loaded data and skipped {}/{} lines since contained in test set.'.format(len(skip_lines), i))\n", " \n", "df = pd.DataFrame(zip(source, target), columns=['source_sentence', 'target_sentence'])\n", "df.head(3)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "YkuK3B4p2AkN" }, "source": [ "## Pre-processing and export\n", "\n", "It is generally a good idea to remove duplicate translations and conflicting translations from the corpus. In practice, these public corpora include some number of these that need to be cleaned.\n", "\n", "In addition we will split our data into dev/test/train and export to the filesystem." ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "colab": {}, "colab_type": "code", "id": "M_2ouEOH1_1q" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:8: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " \n", "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:9: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " if __name__ == '__main__':\n", "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:12: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " if sys.path[0] == '':\n", "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:13: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " del sys.path[0]\n" ] } ], "source": [ "import numpy as np\n", "# drop duplicate translations\n", "df_pp = df.drop_duplicates()\n", "\n", "#drop empty lines (alp)\n", "df_pp['source_sentence'].replace('', np.nan, inplace=True)\n", "df_pp['target_sentence'].replace('', np.nan, inplace=True)\n", "df_pp.dropna(subset=['source_sentence'], inplace=True)\n", "df_pp.dropna(subset=['target_sentence'], inplace=True)\n", "\n", "# drop conflicting translations\n", "df_pp.drop_duplicates(subset='source_sentence', inplace=True)\n", "df_pp.drop_duplicates(subset='target_sentence', inplace=True)\n", "\n", "# Shuffle the data to remove bias in dev set selection.\n", "df_pp = df_pp.sample(frac=1, random_state=seed).reset_index(drop=True)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "colab": {}, "colab_type": "code", "id": "hxxBOCA-xXhy" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "==> train.bpe.en <==\n", "Paul wrote : “ Let the su@@ n not set with you in a pro@@ v@@ ok@@ ed st@@ ate . ”\n", "Some of these en@@ emi@@ es as@@ sa@@ ul@@ t Jehovah’s Witnesses physi@@ c@@ ally .\n", "S@@ T@@ U@@ D@@ Y A@@ R@@ TI@@ C@@ L@@ E@@ S 3 , 4 PA@@ G@@ E@@ S 19 - 28\n", "From inf@@ an@@ cy , children can be@@ gin lear@@ ning about Jehovah’s Word .\n", "In these ways , youn@@ ger ones and those ne@@ w@@ ly associ@@ ated learn to per@@ form ac@@ ts of kindness for others .\n", "O@@ f@@ fered S@@ el@@ ves in W@@ est A@@ fri@@ c@@ a , 1 / 15\n", "Those resurrec@@ ted to heaven will even@@ tually number 14@@ 4@@ ,000 .\n", "A@@ ga@@ inst the bac@@ k@@ d@@ ro@@ p of Satan’s dis@@ as@@ tr@@ ous rule , Jehovah’s perfect qualities are even more ob@@ vi@@ ous than they o@@ ther@@ wise might have been .\n", "The first Kingdom H@@ all in B@@ a@@ uru ​ — a r@@ ented place with a sig@@ n I pa@@ inted , 195@@ 5\n", "In ad@@ dition , his s@@ mok@@ ing end@@ ang@@ ers the health of those around him .\n", "\n", "==> train.bpe.luo <==\n", "Paulo nondiko kama : “ Kik uwe chieng ’ p@@ odhi ka pod un gi mirima . ”\n", "Moko kuom jo@@ wasi@@ gu@@ gi goyo Joneno mag Jehova e yo ma@@ sie .\n", "S@@ U@@ L@@ A MA@@ G P@@ U@@ ON@@ J@@ R@@ UO@@ K 3 , 4 IT@@ E MA@@ G 19 - 28\n", "A e tin - gi , nyithindo nyalo chako puonjore Wach Jehova .\n", "Ka jodongo otiyo kanyachiel gi rowere kod joma nyi@@ en e konyo jomoko , gin bende gibiro bedo gi chuny mar timo kamano .\n", "Pinyruodh Nyasaye No@@ chako Lo@@ cho K@@ ar@@ ang’o ?\n", "Jogo duto mi@@ biro chi@@ ero mondo odhi e polo kwan - gi en 14@@ 4@@ ,000 .\n", "Nikech wanyalo neno kaka loch Satan en loch manono , kido malong’o chuth mag Jehova koro nyis@@ or@@ en@@ wa e yo maler moloyo .\n", "O@@ d Romo ma ne wak@@ wongo bedogo e taon mar B@@ a@@ uru , en ot ma ne wa@@ p@@ ango ma no@@ ke@@ tie ranyisi ma n@@ an@@ diko , 195@@ 5\n", "E wi mano , ma@@ dh@@ one n@@ da@@ wa bende ketho ngima jogo man machiegni kode .\n", "\n", "==> train.en <==\n", "Paul wrote : “ Let the sun not set with you in a provoked state . ”\n", "Some of these enemies assault Jehovah’s Witnesses physically .\n", "STUDY ARTICLES 3 , 4 PAGES 19 - 28\n", "From infancy , children can begin learning about Jehovah’s Word .\n", "In these ways , younger ones and those newly associated learn to perform acts of kindness for others .\n", "Offered Selves in West Africa , 1 / 15\n", "Those resurrected to heaven will eventually number 144,000 .\n", "Against the backdrop of Satan’s disastrous rule , Jehovah’s perfect qualities are even more obvious than they otherwise might have been .\n", "The first Kingdom Hall in Bauru ​ — a rented place with a sign I painted , 1955\n", "In addition , his smoking endangers the health of those around him .\n", "\n", "==> train.luo <==\n", "Paulo nondiko kama : “ Kik uwe chieng ’ podhi ka pod un gi mirima . ”\n", "Moko kuom jowasigugi goyo Joneno mag Jehova e yo masie .\n", "SULA MAG PUONJRUOK 3 , 4 ITE MAG 19 - 28\n", "A e tin - gi , nyithindo nyalo chako puonjore Wach Jehova .\n", "Ka jodongo otiyo kanyachiel gi rowere kod joma nyien e konyo jomoko , gin bende gibiro bedo gi chuny mar timo kamano .\n", "Pinyruodh Nyasaye Nochako Locho Karang’o ?\n", "Jogo duto mibiro chiero mondo odhi e polo kwan - gi en 144,000 .\n", "Nikech wanyalo neno kaka loch Satan en loch manono , kido malong’o chuth mag Jehova koro nyisorenwa e yo maler moloyo .\n", "Od Romo ma ne wakwongo bedogo e taon mar Bauru , en ot ma ne wapango ma noketie ranyisi ma nandiko , 1955\n", "E wi mano , madhone ndawa bende ketho ngima jogo man machiegni kode .\n", "==> dev.bpe.en <==\n", "Their own willing obedi@@ ence to what Jehovah requ@@ ires should demonstr@@ ate that God’s rul@@ es are reason@@ able .\n", "It also gives me more reason to respect him as my father and spiritual head . ”\n", "Se@@ e@@ ing ourselves in the light of what the Scriptures say can in@@ deed be a powerful a@@ id in identi@@ f@@ ying the inten@@ tions of the heart .\n", "He had written about the “ rest@@ or@@ ation of all things of which God spoke through the m@@ outh of his holy prophe@@ ts of old time . ”\n", "15 O@@ U@@ R R@@ E@@ A@@ D@@ ER@@ S A@@ S@@ K . . .\n", "A@@ M@@ ON@@ G the ze@@ al@@ ous Witnesses serving in l@@ ands where there is a greater need for more Kingdom pre@@ ach@@ ers are num@@ erous single sisters .\n", "But often they are not .\n", "As our Creator , Jehovah deser@@ ves pra@@ ise , glory , than@@ k@@ s@@ giving , and honor .\n", "An@@ cient King Solomon had a lo@@ t of knowledge about Jehovah’s ways .\n", "The writ@@ er of Psalm 1@@ 11 had the cor@@ rec@@ t view of godly fear , st@@ ating : “ The fear of Jehovah is the begin@@ ning of wisdom . All those doing them [ keep@@ ing his comman@@ d@@ ments ] have a good in@@ sight . ” ​ — Ps .\n", "\n", "==> dev.bpe.luo <==\n", "Ka gin giwegi gi@@ ikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo mak@@ gi .\n", "Mano bende miyo am@@ edo miye luor kaka wuon@@ wa kendo kaka ng’at mat@@ ay@@ owa e weche mag lamo . ”\n", "Ka wan@@ on@@ ore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "No@@ yudo D@@ un@@ n osen@@ diko wach “ loso gik moko duto od@@ ok makare , kaka Nyasaye nowacho gi dho j@@ on@@ abi duto maler nyaka a chak@@ ruok piny . ”\n", "15 J@@ O@@ S@@ O@@ M@@ B@@ W@@ A P@@ EN@@ J@@ O . . .\n", "N@@ I@@ TI@@ E nyimine mang’eny ma pok oken@@ di mo@@ chiwore tiyo ne Jehova gi kinda e pinje ma jolendo dwar@@ or@@ ee ahinya .\n", "To ok tim@@ re@@ ga kamano .\n", "Nikech Jehova en Ja@@ chu@@ e@@ ch@@ wa , onego wa@@ miye duong ’ , wa@@ g@@ one erokamano , kendo wa@@ miye luor matut .\n", "Ruoth Solomon nong’eyo gik mathoth e wi Jehova .\n", "Jal ma ne ondiko Zaburi 1@@ 11 ne nigi paro makare kuom wach luoro Nyasaye kane owacho kama : “ Lu@@ oro Jehova en chak@@ r@@ w@@ ok mar rieko ; jo ma timo kamano [ ma@@ rito chi@@ kene ] weche don@@ j@@ oni@@ gi maber . ” ​ ​ —⁠ Zab .\n", "\n", "==> dev.en <==\n", "Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "It also gives me more reason to respect him as my father and spiritual head . ”\n", "Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "15 OUR READERS ASK . . .\n", "AMONG the zealous Witnesses serving in lands where there is a greater need for more Kingdom preachers are numerous single sisters .\n", "But often they are not .\n", "As our Creator , Jehovah deserves praise , glory , thanksgiving , and honor .\n", "Ancient King Solomon had a lot of knowledge about Jehovah’s ways .\n", "The writer of Psalm 111 had the correct view of godly fear , stating : “ The fear of Jehovah is the beginning of wisdom . All those doing them [ keeping his commandments ] have a good insight . ” ​ — Ps .\n", "\n", "==> dev.luo <==\n", "Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "15 JOSOMBWA PENJO . . .\n", "NITIE nyimine mang’eny ma pok okendi mochiwore tiyo ne Jehova gi kinda e pinje ma jolendo dwaroree ahinya .\n", "To ok timrega kamano .\n", "Nikech Jehova en Jachuechwa , onego wamiye duong ’ , wagone erokamano , kendo wamiye luor matut .\n", "Ruoth Solomon nong’eyo gik mathoth e wi Jehova .\n", "Jal ma ne ondiko Zaburi 111 ne nigi paro makare kuom wach luoro Nyasaye kane owacho kama : “ Luoro Jehova en chakrwok mar rieko ; jo ma timo kamano [ marito chikene ] weche donjonigi maber . ” ​ ​ —⁠ Zab .\n" ] } ], "source": [ "# This section does the split between train/dev for the parallel corpora then saves them as separate files\n", "# We use 1000 dev test and the given test set.\n", "import csv\n", "\n", "# Do the split between dev/train and create parallel corpora\n", "num_dev_patterns = 1000\n", "\n", "# Optional: lower case the corpora - this will make it easier to generalize, but without proper casing.\n", "if lc: # Julia: making lowercasing optional\n", " df_pp[\"source_sentence\"] = df_pp[\"source_sentence\"].str.lower()\n", " df_pp[\"target_sentence\"] = df_pp[\"target_sentence\"].str.lower()\n", "\n", "# Julia: test sets are already generated\n", "dev = df_pp.tail(num_dev_patterns) # Herman: Error in original\n", "stripped = df_pp.drop(df_pp.tail(num_dev_patterns).index)\n", "\n", "with open(\"train.\"+source_language, \"w\") as src_file, open(\"train.\"+target_language, \"w\") as trg_file:\n", " for index, row in stripped.iterrows():\n", " src_file.write(row[\"source_sentence\"]+\"\\n\")\n", " trg_file.write(row[\"target_sentence\"]+\"\\n\")\n", " \n", "with open(\"dev.\"+source_language, \"w\") as src_file, open(\"dev.\"+target_language, \"w\") as trg_file:\n", " for index, row in dev.iterrows():\n", " src_file.write(row[\"source_sentence\"]+\"\\n\")\n", " trg_file.write(row[\"target_sentence\"]+\"\\n\")\n", "\n", "#stripped[[\"source_sentence\"]].to_csv(\"train.\"+source_language, header=False, index=False) # Herman: Added `header=False` everywhere\n", "#stripped[[\"target_sentence\"]].to_csv(\"train.\"+target_language, header=False, index=False) # Julia: Problematic handling of quotation marks.\n", "\n", "#dev[[\"source_sentence\"]].to_csv(\"dev.\"+source_language, header=False, index=False)\n", "#dev[[\"target_sentence\"]].to_csv(\"dev.\"+target_language, header=False, index=False)\n", "\n", "\n", "# Doublecheck the format below. There should be no extra quotation marks or weird characters.\n", "! head train.*\n", "! head dev.*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "epeCydmCyS8X" }, "source": [ "\n", "\n", "---\n", "\n", "\n", "## Installation of JoeyNMT\n", "\n", "JoeyNMT is a simple, minimalist NMT package which is useful for learning and teaching. Check out the documentation for JoeyNMT [here](https://joeynmt.readthedocs.io) " ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "iBRMm4kMxZ8L" }, "outputs": [], "source": [ "# Install JoeyNMT\n", "#! git clone https://github.com/joeynmt/joeynmt.git\n", "#! cd joeynmt; pip3 install ." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AaE77Tcppex9" }, "source": [ "# Preprocessing the Data into Subword BPE Tokens\n", "\n", "- One of the most powerful improvements for agglutinative languages (a feature of most Bantu languages) is using BPE tokenization [ (Sennrich, 2015) ](https://arxiv.org/abs/1508.07909).\n", "\n", "- It was also shown that by optimizing the umber of BPE codes we significantly improve results for low-resourced languages [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021) [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)\n", "\n", "- Below we have the scripts for doing BPE tokenization of our data. We use 4000 tokens as recommended by [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021). You do not need to change anything. Simply running the below will be suitable. " ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# One of the huge boosts in NMT performance was to use a different method of tokenizing. \n", "# Usually, NMT would tokenize by words. However, using a method called BPE gave amazing boosts to performance\n", "\n", "# Do subword NMT\n", "from os import path\n", "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n", "os.environ[\"tgt\"] = target_language\n", "\n", "# Learn BPEs on the training data.\n", "os.environ[\"data_path\"] = path.join(\"../../joeynmt\", \"data\", source_language + target_language) # Herman! \n", "! subword-nmt learn-joint-bpe-and-vocab --input train.$src train.$tgt -s 4000 -o bpe.codes.4000 --write-vocabulary vocab.$src vocab.$tgt" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Apply BPE splits to the development and test data.\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < train.$src > train.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < train.$tgt > train.bpe.$tgt\n", "\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < dev.$src > dev.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < dev.$tgt > dev.bpe.$tgt\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < test.$src > test.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": true }, "outputs": [], "source": [ "! sudo chmod 777 ../../joeynmt/data/" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "bpe.codes.4000\tdev.en\t test.bpe.luo test.luo\t train.en\r\n", "dev.bpe.en\tdev.luo test.en\t train.bpe.en train.luo\r\n", "dev.bpe.luo\ttest.bpe.en test.en-any.en train.bpe.luo vocab.txt\r\n" ] } ], "source": [ "# Create directory, move everyone we care about to the correct location\n", "! mkdir -p $data_path\n", "! cp train.* $data_path\n", "! cp test.* $data_path\n", "! cp dev.* $data_path\n", "! cp bpe.codes.4000 $data_path\n", "! ls $data_path" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n", "# ! cp train.* \"$gdrive_path\"\n", "# ! cp test.* \"$gdrive_path\"\n", "# ! cp dev.* \"$gdrive_path\"\n", "# ! cp bpe.codes.4000 \"$gdrive_path\"\n", "# ! ls \"$gdrive_path\"" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Create that vocab using build_vocab\n", "! sudo chmod 777 ../../joeynmt/scripts/build_vocab.py\n", "! ../../joeynmt/scripts/build_vocab.py ../../joeynmt/data/$src$tgt/train.bpe.$src ../../joeynmt/data/$src$tgt/train.bpe.$tgt --output_path ../../joeynmt/data/$src$tgt/vocab.txt" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "colab": {}, "colab_type": "code", "id": "H-TyjtmXB1mL" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "BPE Luo Sentences\n", "O@@ k@@ um@@ ba malach mar yie ( Ne paragraf mar 12 - 14 )\n", "O@@ g@@ ud@@ u mar war@@ ruok ( Ne paragraf mar 15 - 18 )\n", "A@@ se@@ fwenyo ni ji chiko it@@ gi sama gineno ni i@@ hero weche manie Muma , kendo itimo duto ma inyalo mondo i@@ kony@@ gi . ”\n", "L@@ ig@@ ang@@ la mar roho maler ( Ne paragraf mar 19 - 20 )\n", "Kata kamano , kokalo kuom teko mar Jehova wanyalo k@@ wede !\n", "Combined BPE Vocab\n", "portunity\n", "aburi\n", "Q\n", ".E\n", "Ç@@\n", "Ł@@\n", "ʺ\n", "rans@@\n", "oura@@\n", "_\n" ] } ], "source": [ "# Some output\n", "! echo \"BPE Luo Sentences\"\n", "! tail -n 5 test.bpe.$tgt\n", "! echo \"Combined BPE Vocab\"\n", "! tail -n 10 ../../joeynmt/data/$src$tgt/vocab.txt # Herman" ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "IlMitUHR8Qy-" }, "outputs": [], "source": [ "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n", "#! cp train.* \"$gdrive_path\"\n", "#! cp test.* \"$gdrive_path\"\n", "#! cp dev.* \"$gdrive_path\"\n", "#! cp bpe.codes.4000 \"$gdrive_path\"\n", "#! ls \"$gdrive_path\"" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Ixmzi60WsUZ8" }, "source": [ "# Creating the JoeyNMT Config\n", "\n", "JoeyNMT requires a yaml config. We provide a template below. We've also set a number of defaults with it, that you may play with!\n", "\n", "- We used Transformer architecture \n", "- We set our dropout to reasonably high: 0.3 (recommended in [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021))\n", "\n", "Things worth playing with:\n", "- The batch size (also recommended to change for low-resourced languages)\n", "- The number of epochs (we've set it at 30 just so it runs in about an hour, for testing purposes)\n", "- The decoder options (beam_size, alpha)\n", "- Evaluation metrics (BLEU versus Crhf4)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "PIs1lY2hxMsl" }, "outputs": [], "source": [ "# This creates the config file for our JoeyNMT system. It might seem overwhelming so we've provided a couple of useful parameters you'll need to update\n", "# (You can of course play with all the parameters if you'd like!)\n", "\n", "name = '%s%s' % (source_language, target_language)\n", "# gdrive_path = os.environ[\"gdrive_path\"]\n", "\n", "# Create the config\n", "config = \"\"\"\n", "name: \"{name}_transformer\"\n", "\n", "data:\n", " src: \"{source_language}\"\n", " trg: \"{target_language}\"\n", " train: \"data/{name}/train.bpe\"\n", " dev: \"data/{name}/dev.bpe\"\n", " test: \"data/{name}/test.bpe\"\n", " level: \"bpe\"\n", " lowercase: False\n", " max_sent_length: 100\n", " src_vocab: \"data/{name}/vocab.txt\"\n", " trg_vocab: \"data/{name}/vocab.txt\"\n", "\n", "testing:\n", " beam_size: 5\n", " alpha: 1.0\n", "\n", "training:\n", " #load_model: \"models/{name}_transformer/1000.ckpt\" # if uncommented, load a pre-trained model from this checkpoint\n", " random_seed: 42\n", " optimizer: \"adam\"\n", " normalization: \"tokens\"\n", " adam_betas: [0.9, 0.999] \n", " scheduling: \"plateau\" # TODO: try switching from plateau to Noam scheduling\n", " patience: 5 # For plateau: decrease learning rate by decrease_factor if validation score has not improved for this many validation rounds.\n", " learning_rate_factor: 0.5 # factor for Noam scheduler (used with Transformer)\n", " learning_rate_warmup: 1000 # warmup steps for Noam scheduler (used with Transformer)\n", " decrease_factor: 0.7\n", " loss: \"crossentropy\"\n", " learning_rate: 0.0003\n", " learning_rate_min: 0.00000001\n", " weight_decay: 0.0\n", " label_smoothing: 0.1\n", " batch_size: 4096\n", " batch_type: \"token\"\n", " eval_batch_size: 3600\n", " eval_batch_type: \"token\"\n", " batch_multiplier: 1\n", " early_stopping_metric: \"ppl\"\n", " epochs: 30 # TODO: Decrease for when playing around and checking of working. Around 30 is sufficient to check if its working at all\n", " validation_freq: 1000 # TODO: Set to at least once per epoch.\n", " logging_freq: 100\n", " eval_metric: \"bleu\"\n", " model_dir: \"models/{name}_transformer\"\n", " overwrite: False # TODO: Set to True if you want to overwrite possibly existing models. \n", " shuffle: True\n", " use_cuda: True\n", " max_output_length: 100\n", " print_valid_sents: [0, 1, 2, 3]\n", " keep_last_ckpts: 3\n", "\n", "model:\n", " initializer: \"xavier\"\n", " bias_initializer: \"zeros\"\n", " init_gain: 1.0\n", " embed_initializer: \"xavier\"\n", " embed_init_gain: 1.0\n", " tied_embeddings: True\n", " tied_softmax: True\n", " encoder:\n", " type: \"transformer\"\n", " num_layers: 6\n", " num_heads: 4 # TODO: Increase to 8 for larger data.\n", " embeddings:\n", " embedding_dim: 256 # TODO: Increase to 512 for larger data.\n", " scale: True\n", " dropout: 0.2\n", " # typically ff_size = 4 x hidden_size\n", " hidden_size: 256 # TODO: Increase to 512 for larger data.\n", " ff_size: 1024 # TODO: Increase to 2048 for larger data.\n", " dropout: 0.3\n", " decoder:\n", " type: \"transformer\"\n", " num_layers: 6\n", " num_heads: 4 # TODO: Increase to 8 for larger data.\n", " embeddings:\n", " embedding_dim: 256 # TODO: Increase to 512 for larger data.\n", " scale: True\n", " dropout: 0.2\n", " # typically ff_size = 4 x hidden_size\n", " hidden_size: 256 # TODO: Increase to 512 for larger data.\n", " ff_size: 1024 # TODO: Increase to 2048 for larger data.\n", " dropout: 0.3\n", "\"\"\".format(name=name, gdrive_path=\"n/a\", source_language=source_language, target_language=target_language)\n", "with open(\"../../joeynmt/configs/transformer_{name}.yaml\".format(name=name),'w') as f:\n", " f.write(config)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "pIifxE3Qzuvs" }, "source": [ "# Train the Model\n", "\n", "This single line of joeynmt runs the training using the config we made above" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "total 136\r\n", "drwxr-xr-x 11 root root 4096 Oct 25 12:22 .\r\n", "drwxrwxrwx 11 root root 4096 Feb 19 06:15 ..\r\n", "drwxr-xr-x 8 root root 4096 Oct 24 15:14 .git\r\n", "-rw-r--r-- 1 root root 49 Oct 24 15:14 .gitattributes\r\n", "drwxr-xr-x 3 root root 4096 Oct 24 15:14 .github\r\n", "-rw-r--r-- 1 root root 71 Oct 24 15:14 .gitignore\r\n", "-rw-r--r-- 1 root root 13514 Oct 24 15:14 .pylintrc\r\n", "-rw-r--r-- 1 root root 159 Oct 24 15:14 .readthedocs.yml\r\n", "-rw-r--r-- 1 root root 542 Oct 24 15:14 .travis.yml\r\n", "-rwxrw-rwx 1 root root 3354 Oct 24 15:14 CODE_OF_CONDUCT.md\r\n", "-rwxrw-rwx 1 root root 1071 Oct 24 15:14 LICENSE\r\n", "-rwxrw-rwx 1 root root 13286 Oct 24 15:14 README.md\r\n", "-rwxrw-rwx 1 root root 8229 Oct 24 15:14 benchmarks.md\r\n", "drwxrw-rwx 3 root root 4096 Feb 19 06:30 configs\r\n", "drwxrwxrwx 8 root root 4096 Feb 19 06:35 data\r\n", "drwxrw-rwx 4 root root 4096 Oct 24 15:14 docs\r\n", "-rwxrw-rwx 1 root root 14373 Oct 24 15:14 joey-small.png\r\n", "drwxrw-rwx 3 root root 4096 Oct 24 16:35 joeynmt\r\n", "drwxrwxrwx 7 root root 4096 Feb 20 07:38 models\r\n", "-rwxrw-rwx 1 root root 167 Oct 24 15:14 requirements.txt\r\n", "drwxrw-rwx 2 root root 4096 Oct 24 15:14 scripts\r\n", "-rwxrw-rwx 1 root root 810 Oct 24 15:14 setup.py\r\n", "drwxrw-rwx 4 root root 4096 Oct 24 15:14 test\r\n" ] } ], "source": [ "! ls -la ../../joeynmt" ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "collapsed": true }, "outputs": [], "source": [ "! sudo chmod 777 ../../joeynmt/models" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "colab": {}, "colab_type": "code", "id": "6ZBPFwT94WpI" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n", "2020-02-20 12:03:17,437 Hello! This is Joey-NMT.\n", "2020-02-20 12:03:17,444 Total params: 12144896\n", "2020-02-20 12:03:17,445 Trainable parameters: ['decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.dec_layer_norm.bias', 'decoder.layers.0.dec_layer_norm.weight', 'decoder.layers.0.feed_forward.layer_norm.bias', 'decoder.layers.0.feed_forward.layer_norm.weight', 'decoder.layers.0.feed_forward.pwff_layer.0.bias', 'decoder.layers.0.feed_forward.pwff_layer.0.weight', 'decoder.layers.0.feed_forward.pwff_layer.3.bias', 'decoder.layers.0.feed_forward.pwff_layer.3.weight', 'decoder.layers.0.src_trg_att.k_layer.bias', 'decoder.layers.0.src_trg_att.k_layer.weight', 'decoder.layers.0.src_trg_att.output_layer.bias', 'decoder.layers.0.src_trg_att.output_layer.weight', 'decoder.layers.0.src_trg_att.q_layer.bias', 'decoder.layers.0.src_trg_att.q_layer.weight', 'decoder.layers.0.src_trg_att.v_layer.bias', 'decoder.layers.0.src_trg_att.v_layer.weight', 'decoder.layers.0.trg_trg_att.k_layer.bias', 'decoder.layers.0.trg_trg_att.k_layer.weight', 'decoder.layers.0.trg_trg_att.output_layer.bias', 'decoder.layers.0.trg_trg_att.output_layer.weight', 'decoder.layers.0.trg_trg_att.q_layer.bias', 'decoder.layers.0.trg_trg_att.q_layer.weight', 'decoder.layers.0.trg_trg_att.v_layer.bias', 'decoder.layers.0.trg_trg_att.v_layer.weight', 'decoder.layers.0.x_layer_norm.bias', 'decoder.layers.0.x_layer_norm.weight', 'decoder.layers.1.dec_layer_norm.bias', 'decoder.layers.1.dec_layer_norm.weight', 'decoder.layers.1.feed_forward.layer_norm.bias', 'decoder.layers.1.feed_forward.layer_norm.weight', 'decoder.layers.1.feed_forward.pwff_layer.0.bias', 'decoder.layers.1.feed_forward.pwff_layer.0.weight', 'decoder.layers.1.feed_forward.pwff_layer.3.bias', 'decoder.layers.1.feed_forward.pwff_layer.3.weight', 'decoder.layers.1.src_trg_att.k_layer.bias', 'decoder.layers.1.src_trg_att.k_layer.weight', 'decoder.layers.1.src_trg_att.output_layer.bias', 'decoder.layers.1.src_trg_att.output_layer.weight', 'decoder.layers.1.src_trg_att.q_layer.bias', 'decoder.layers.1.src_trg_att.q_layer.weight', 'decoder.layers.1.src_trg_att.v_layer.bias', 'decoder.layers.1.src_trg_att.v_layer.weight', 'decoder.layers.1.trg_trg_att.k_layer.bias', 'decoder.layers.1.trg_trg_att.k_layer.weight', 'decoder.layers.1.trg_trg_att.output_layer.bias', 'decoder.layers.1.trg_trg_att.output_layer.weight', 'decoder.layers.1.trg_trg_att.q_layer.bias', 'decoder.layers.1.trg_trg_att.q_layer.weight', 'decoder.layers.1.trg_trg_att.v_layer.bias', 'decoder.layers.1.trg_trg_att.v_layer.weight', 'decoder.layers.1.x_layer_norm.bias', 'decoder.layers.1.x_layer_norm.weight', 'decoder.layers.2.dec_layer_norm.bias', 'decoder.layers.2.dec_layer_norm.weight', 'decoder.layers.2.feed_forward.layer_norm.bias', 'decoder.layers.2.feed_forward.layer_norm.weight', 'decoder.layers.2.feed_forward.pwff_layer.0.bias', 'decoder.layers.2.feed_forward.pwff_layer.0.weight', 'decoder.layers.2.feed_forward.pwff_layer.3.bias', 'decoder.layers.2.feed_forward.pwff_layer.3.weight', 'decoder.layers.2.src_trg_att.k_layer.bias', 'decoder.layers.2.src_trg_att.k_layer.weight', 'decoder.layers.2.src_trg_att.output_layer.bias', 'decoder.layers.2.src_trg_att.output_layer.weight', 'decoder.layers.2.src_trg_att.q_layer.bias', 'decoder.layers.2.src_trg_att.q_layer.weight', 'decoder.layers.2.src_trg_att.v_layer.bias', 'decoder.layers.2.src_trg_att.v_layer.weight', 'decoder.layers.2.trg_trg_att.k_layer.bias', 'decoder.layers.2.trg_trg_att.k_layer.weight', 'decoder.layers.2.trg_trg_att.output_layer.bias', 'decoder.layers.2.trg_trg_att.output_layer.weight', 'decoder.layers.2.trg_trg_att.q_layer.bias', 'decoder.layers.2.trg_trg_att.q_layer.weight', 'decoder.layers.2.trg_trg_att.v_layer.bias', 'decoder.layers.2.trg_trg_att.v_layer.weight', 'decoder.layers.2.x_layer_norm.bias', 'decoder.layers.2.x_layer_norm.weight', 'decoder.layers.3.dec_layer_norm.bias', 'decoder.layers.3.dec_layer_norm.weight', 'decoder.layers.3.feed_forward.layer_norm.bias', 'decoder.layers.3.feed_forward.layer_norm.weight', 'decoder.layers.3.feed_forward.pwff_layer.0.bias', 'decoder.layers.3.feed_forward.pwff_layer.0.weight', 'decoder.layers.3.feed_forward.pwff_layer.3.bias', 'decoder.layers.3.feed_forward.pwff_layer.3.weight', 'decoder.layers.3.src_trg_att.k_layer.bias', 'decoder.layers.3.src_trg_att.k_layer.weight', 'decoder.layers.3.src_trg_att.output_layer.bias', 'decoder.layers.3.src_trg_att.output_layer.weight', 'decoder.layers.3.src_trg_att.q_layer.bias', 'decoder.layers.3.src_trg_att.q_layer.weight', 'decoder.layers.3.src_trg_att.v_layer.bias', 'decoder.layers.3.src_trg_att.v_layer.weight', 'decoder.layers.3.trg_trg_att.k_layer.bias', 'decoder.layers.3.trg_trg_att.k_layer.weight', 'decoder.layers.3.trg_trg_att.output_layer.bias', 'decoder.layers.3.trg_trg_att.output_layer.weight', 'decoder.layers.3.trg_trg_att.q_layer.bias', 'decoder.layers.3.trg_trg_att.q_layer.weight', 'decoder.layers.3.trg_trg_att.v_layer.bias', 'decoder.layers.3.trg_trg_att.v_layer.weight', 'decoder.layers.3.x_layer_norm.bias', 'decoder.layers.3.x_layer_norm.weight', 'decoder.layers.4.dec_layer_norm.bias', 'decoder.layers.4.dec_layer_norm.weight', 'decoder.layers.4.feed_forward.layer_norm.bias', 'decoder.layers.4.feed_forward.layer_norm.weight', 'decoder.layers.4.feed_forward.pwff_layer.0.bias', 'decoder.layers.4.feed_forward.pwff_layer.0.weight', 'decoder.layers.4.feed_forward.pwff_layer.3.bias', 'decoder.layers.4.feed_forward.pwff_layer.3.weight', 'decoder.layers.4.src_trg_att.k_layer.bias', 'decoder.layers.4.src_trg_att.k_layer.weight', 'decoder.layers.4.src_trg_att.output_layer.bias', 'decoder.layers.4.src_trg_att.output_layer.weight', 'decoder.layers.4.src_trg_att.q_layer.bias', 'decoder.layers.4.src_trg_att.q_layer.weight', 'decoder.layers.4.src_trg_att.v_layer.bias', 'decoder.layers.4.src_trg_att.v_layer.weight', 'decoder.layers.4.trg_trg_att.k_layer.bias', 'decoder.layers.4.trg_trg_att.k_layer.weight', 'decoder.layers.4.trg_trg_att.output_layer.bias', 'decoder.layers.4.trg_trg_att.output_layer.weight', 'decoder.layers.4.trg_trg_att.q_layer.bias', 'decoder.layers.4.trg_trg_att.q_layer.weight', 'decoder.layers.4.trg_trg_att.v_layer.bias', 'decoder.layers.4.trg_trg_att.v_layer.weight', 'decoder.layers.4.x_layer_norm.bias', 'decoder.layers.4.x_layer_norm.weight', 'decoder.layers.5.dec_layer_norm.bias', 'decoder.layers.5.dec_layer_norm.weight', 'decoder.layers.5.feed_forward.layer_norm.bias', 'decoder.layers.5.feed_forward.layer_norm.weight', 'decoder.layers.5.feed_forward.pwff_layer.0.bias', 'decoder.layers.5.feed_forward.pwff_layer.0.weight', 'decoder.layers.5.feed_forward.pwff_layer.3.bias', 'decoder.layers.5.feed_forward.pwff_layer.3.weight', 'decoder.layers.5.src_trg_att.k_layer.bias', 'decoder.layers.5.src_trg_att.k_layer.weight', 'decoder.layers.5.src_trg_att.output_layer.bias', 'decoder.layers.5.src_trg_att.output_layer.weight', 'decoder.layers.5.src_trg_att.q_layer.bias', 'decoder.layers.5.src_trg_att.q_layer.weight', 'decoder.layers.5.src_trg_att.v_layer.bias', 'decoder.layers.5.src_trg_att.v_layer.weight', 'decoder.layers.5.trg_trg_att.k_layer.bias', 'decoder.layers.5.trg_trg_att.k_layer.weight', 'decoder.layers.5.trg_trg_att.output_layer.bias', 'decoder.layers.5.trg_trg_att.output_layer.weight', 'decoder.layers.5.trg_trg_att.q_layer.bias', 'decoder.layers.5.trg_trg_att.q_layer.weight', 'decoder.layers.5.trg_trg_att.v_layer.bias', 'decoder.layers.5.trg_trg_att.v_layer.weight', 'decoder.layers.5.x_layer_norm.bias', 'decoder.layers.5.x_layer_norm.weight', 'encoder.layer_norm.bias', 'encoder.layer_norm.weight', 'encoder.layers.0.feed_forward.layer_norm.bias', 'encoder.layers.0.feed_forward.layer_norm.weight', 'encoder.layers.0.feed_forward.pwff_layer.0.bias', 'encoder.layers.0.feed_forward.pwff_layer.0.weight', 'encoder.layers.0.feed_forward.pwff_layer.3.bias', 'encoder.layers.0.feed_forward.pwff_layer.3.weight', 'encoder.layers.0.layer_norm.bias', 'encoder.layers.0.layer_norm.weight', 'encoder.layers.0.src_src_att.k_layer.bias', 'encoder.layers.0.src_src_att.k_layer.weight', 'encoder.layers.0.src_src_att.output_layer.bias', 'encoder.layers.0.src_src_att.output_layer.weight', 'encoder.layers.0.src_src_att.q_layer.bias', 'encoder.layers.0.src_src_att.q_layer.weight', 'encoder.layers.0.src_src_att.v_layer.bias', 'encoder.layers.0.src_src_att.v_layer.weight', 'encoder.layers.1.feed_forward.layer_norm.bias', 'encoder.layers.1.feed_forward.layer_norm.weight', 'encoder.layers.1.feed_forward.pwff_layer.0.bias', 'encoder.layers.1.feed_forward.pwff_layer.0.weight', 'encoder.layers.1.feed_forward.pwff_layer.3.bias', 'encoder.layers.1.feed_forward.pwff_layer.3.weight', 'encoder.layers.1.layer_norm.bias', 'encoder.layers.1.layer_norm.weight', 'encoder.layers.1.src_src_att.k_layer.bias', 'encoder.layers.1.src_src_att.k_layer.weight', 'encoder.layers.1.src_src_att.output_layer.bias', 'encoder.layers.1.src_src_att.output_layer.weight', 'encoder.layers.1.src_src_att.q_layer.bias', 'encoder.layers.1.src_src_att.q_layer.weight', 'encoder.layers.1.src_src_att.v_layer.bias', 'encoder.layers.1.src_src_att.v_layer.weight', 'encoder.layers.2.feed_forward.layer_norm.bias', 'encoder.layers.2.feed_forward.layer_norm.weight', 'encoder.layers.2.feed_forward.pwff_layer.0.bias', 'encoder.layers.2.feed_forward.pwff_layer.0.weight', 'encoder.layers.2.feed_forward.pwff_layer.3.bias', 'encoder.layers.2.feed_forward.pwff_layer.3.weight', 'encoder.layers.2.layer_norm.bias', 'encoder.layers.2.layer_norm.weight', 'encoder.layers.2.src_src_att.k_layer.bias', 'encoder.layers.2.src_src_att.k_layer.weight', 'encoder.layers.2.src_src_att.output_layer.bias', 'encoder.layers.2.src_src_att.output_layer.weight', 'encoder.layers.2.src_src_att.q_layer.bias', 'encoder.layers.2.src_src_att.q_layer.weight', 'encoder.layers.2.src_src_att.v_layer.bias', 'encoder.layers.2.src_src_att.v_layer.weight', 'encoder.layers.3.feed_forward.layer_norm.bias', 'encoder.layers.3.feed_forward.layer_norm.weight', 'encoder.layers.3.feed_forward.pwff_layer.0.bias', 'encoder.layers.3.feed_forward.pwff_layer.0.weight', 'encoder.layers.3.feed_forward.pwff_layer.3.bias', 'encoder.layers.3.feed_forward.pwff_layer.3.weight', 'encoder.layers.3.layer_norm.bias', 'encoder.layers.3.layer_norm.weight', 'encoder.layers.3.src_src_att.k_layer.bias', 'encoder.layers.3.src_src_att.k_layer.weight', 'encoder.layers.3.src_src_att.output_layer.bias', 'encoder.layers.3.src_src_att.output_layer.weight', 'encoder.layers.3.src_src_att.q_layer.bias', 'encoder.layers.3.src_src_att.q_layer.weight', 'encoder.layers.3.src_src_att.v_layer.bias', 'encoder.layers.3.src_src_att.v_layer.weight', 'encoder.layers.4.feed_forward.layer_norm.bias', 'encoder.layers.4.feed_forward.layer_norm.weight', 'encoder.layers.4.feed_forward.pwff_layer.0.bias', 'encoder.layers.4.feed_forward.pwff_layer.0.weight', 'encoder.layers.4.feed_forward.pwff_layer.3.bias', 'encoder.layers.4.feed_forward.pwff_layer.3.weight', 'encoder.layers.4.layer_norm.bias', 'encoder.layers.4.layer_norm.weight', 'encoder.layers.4.src_src_att.k_layer.bias', 'encoder.layers.4.src_src_att.k_layer.weight', 'encoder.layers.4.src_src_att.output_layer.bias', 'encoder.layers.4.src_src_att.output_layer.weight', 'encoder.layers.4.src_src_att.q_layer.bias', 'encoder.layers.4.src_src_att.q_layer.weight', 'encoder.layers.4.src_src_att.v_layer.bias', 'encoder.layers.4.src_src_att.v_layer.weight', 'encoder.layers.5.feed_forward.layer_norm.bias', 'encoder.layers.5.feed_forward.layer_norm.weight', 'encoder.layers.5.feed_forward.pwff_layer.0.bias', 'encoder.layers.5.feed_forward.pwff_layer.0.weight', 'encoder.layers.5.feed_forward.pwff_layer.3.bias', 'encoder.layers.5.feed_forward.pwff_layer.3.weight', 'encoder.layers.5.layer_norm.bias', 'encoder.layers.5.layer_norm.weight', 'encoder.layers.5.src_src_att.k_layer.bias', 'encoder.layers.5.src_src_att.k_layer.weight', 'encoder.layers.5.src_src_att.output_layer.bias', 'encoder.layers.5.src_src_att.output_layer.weight', 'encoder.layers.5.src_src_att.q_layer.bias', 'encoder.layers.5.src_src_att.q_layer.weight', 'encoder.layers.5.src_src_att.v_layer.bias', 'encoder.layers.5.src_src_att.v_layer.weight', 'src_embed.lut.weight']\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 12:03:20,118 cfg.name : enluo_transformer\n", "2020-02-20 12:03:20,118 cfg.data.src : en\n", "2020-02-20 12:03:20,118 cfg.data.trg : luo\n", "2020-02-20 12:03:20,118 cfg.data.train : data/enluo/train.bpe\n", "2020-02-20 12:03:20,118 cfg.data.dev : data/enluo/dev.bpe\n", "2020-02-20 12:03:20,118 cfg.data.test : data/enluo/test.bpe\n", "2020-02-20 12:03:20,118 cfg.data.level : bpe\n", "2020-02-20 12:03:20,118 cfg.data.lowercase : False\n", "2020-02-20 12:03:20,118 cfg.data.max_sent_length : 100\n", "2020-02-20 12:03:20,118 cfg.data.src_vocab : data/enluo/vocab.txt\n", "2020-02-20 12:03:20,118 cfg.data.trg_vocab : data/enluo/vocab.txt\n", "2020-02-20 12:03:20,118 cfg.testing.beam_size : 5\n", "2020-02-20 12:03:20,118 cfg.testing.alpha : 1.0\n", "2020-02-20 12:03:20,119 cfg.training.random_seed : 42\n", "2020-02-20 12:03:20,119 cfg.training.optimizer : adam\n", "2020-02-20 12:03:20,119 cfg.training.normalization : tokens\n", "2020-02-20 12:03:20,119 cfg.training.adam_betas : [0.9, 0.999]\n", "2020-02-20 12:03:20,119 cfg.training.scheduling : plateau\n", "2020-02-20 12:03:20,119 cfg.training.patience : 5\n", "2020-02-20 12:03:20,119 cfg.training.learning_rate_factor : 0.5\n", "2020-02-20 12:03:20,119 cfg.training.learning_rate_warmup : 1000\n", "2020-02-20 12:03:20,119 cfg.training.decrease_factor : 0.7\n", "2020-02-20 12:03:20,119 cfg.training.loss : crossentropy\n", "2020-02-20 12:03:20,119 cfg.training.learning_rate : 0.0003\n", "2020-02-20 12:03:20,119 cfg.training.learning_rate_min : 1e-08\n", "2020-02-20 12:03:20,119 cfg.training.weight_decay : 0.0\n", "2020-02-20 12:03:20,119 cfg.training.label_smoothing : 0.1\n", "2020-02-20 12:03:20,119 cfg.training.batch_size : 4096\n", "2020-02-20 12:03:20,119 cfg.training.batch_type : token\n", "2020-02-20 12:03:20,120 cfg.training.eval_batch_size : 3600\n", "2020-02-20 12:03:20,120 cfg.training.eval_batch_type : token\n", "2020-02-20 12:03:20,120 cfg.training.batch_multiplier : 1\n", "2020-02-20 12:03:20,120 cfg.training.early_stopping_metric : ppl\n", "2020-02-20 12:03:20,120 cfg.training.epochs : 30\n", "2020-02-20 12:03:20,120 cfg.training.validation_freq : 1000\n", "2020-02-20 12:03:20,120 cfg.training.logging_freq : 100\n", "2020-02-20 12:03:20,120 cfg.training.eval_metric : bleu\n", "2020-02-20 12:03:20,120 cfg.training.model_dir : models/enluo_transformer\n", "2020-02-20 12:03:20,120 cfg.training.overwrite : False\n", "2020-02-20 12:03:20,120 cfg.training.shuffle : True\n", "2020-02-20 12:03:20,120 cfg.training.use_cuda : True\n", "2020-02-20 12:03:20,120 cfg.training.max_output_length : 100\n", "2020-02-20 12:03:20,120 cfg.training.print_valid_sents : [0, 1, 2, 3]\n", "2020-02-20 12:03:20,120 cfg.training.keep_last_ckpts : 3\n", "2020-02-20 12:03:20,121 cfg.model.initializer : xavier\n", "2020-02-20 12:03:20,121 cfg.model.bias_initializer : zeros\n", "2020-02-20 12:03:20,121 cfg.model.init_gain : 1.0\n", "2020-02-20 12:03:20,121 cfg.model.embed_initializer : xavier\n", "2020-02-20 12:03:20,121 cfg.model.embed_init_gain : 1.0\n", "2020-02-20 12:03:20,121 cfg.model.tied_embeddings : True\n", "2020-02-20 12:03:20,121 cfg.model.tied_softmax : True\n", "2020-02-20 12:03:20,121 cfg.model.encoder.type : transformer\n", "2020-02-20 12:03:20,121 cfg.model.encoder.num_layers : 6\n", "2020-02-20 12:03:20,121 cfg.model.encoder.num_heads : 4\n", "2020-02-20 12:03:20,121 cfg.model.encoder.embeddings.embedding_dim : 256\n", "2020-02-20 12:03:20,121 cfg.model.encoder.embeddings.scale : True\n", "2020-02-20 12:03:20,121 cfg.model.encoder.embeddings.dropout : 0.2\n", "2020-02-20 12:03:20,121 cfg.model.encoder.hidden_size : 256\n", "2020-02-20 12:03:20,121 cfg.model.encoder.ff_size : 1024\n", "2020-02-20 12:03:20,122 cfg.model.encoder.dropout : 0.3\n", "2020-02-20 12:03:20,122 cfg.model.decoder.type : transformer\n", "2020-02-20 12:03:20,122 cfg.model.decoder.num_layers : 6\n", "2020-02-20 12:03:20,122 cfg.model.decoder.num_heads : 4\n", "2020-02-20 12:03:20,122 cfg.model.decoder.embeddings.embedding_dim : 256\n", "2020-02-20 12:03:20,122 cfg.model.decoder.embeddings.scale : True\n", "2020-02-20 12:03:20,122 cfg.model.decoder.embeddings.dropout : 0.2\n", "2020-02-20 12:03:20,122 cfg.model.decoder.hidden_size : 256\n", "2020-02-20 12:03:20,122 cfg.model.decoder.ff_size : 1024\n", "2020-02-20 12:03:20,122 cfg.model.decoder.dropout : 0.3\n", "2020-02-20 12:03:20,122 Data set sizes: \n", "\ttrain 136375,\n", "\tvalid 1000,\n", "\ttest 2705\n", "2020-02-20 12:03:20,122 First training example:\n", "\t[SRC] Paul wrote : “ Let the su@@ n not set with you in a pro@@ v@@ ok@@ ed st@@ ate . ”\n", "\t[TRG] Paulo nondiko kama : “ Kik uwe chieng ’ p@@ odhi ka pod un gi mirima . ”\n", "2020-02-20 12:03:20,122 First 10 words (src): (0) (1) (2) (3) (4) . (5) , (6) the (7) to (8) e (9) ne\n", "2020-02-20 12:03:20,123 First 10 words (trg): (0) (1) (2) (3) (4) . (5) , (6) the (7) to (8) e (9) ne\n", "2020-02-20 12:03:20,123 Number of Src words (types): 4237\n", "2020-02-20 12:03:20,123 Number of Trg words (types): 4237\n", "2020-02-20 12:03:20,123 Model(\n", "\tencoder=TransformerEncoder(num_layers=6, num_heads=4),\n", "\tdecoder=TransformerDecoder(num_layers=6, num_heads=4),\n", "\tsrc_embed=Embeddings(embedding_dim=256, vocab_size=4237),\n", "\ttrg_embed=Embeddings(embedding_dim=256, vocab_size=4237))\n", "2020-02-20 12:03:20,127 EPOCH 1\n", "2020-02-20 12:03:49,513 Epoch 1 Step: 100 Batch Loss: 5.409370 Tokens per Sec: 7544, Lr: 0.000300\n", "2020-02-20 12:04:18,216 Epoch 1 Step: 200 Batch Loss: 5.257635 Tokens per Sec: 7619, Lr: 0.000300\n", "2020-02-20 12:04:47,204 Epoch 1 Step: 300 Batch Loss: 5.064311 Tokens per Sec: 7625, Lr: 0.000300\n", "2020-02-20 12:05:15,922 Epoch 1 Step: 400 Batch Loss: 4.967086 Tokens per Sec: 7592, Lr: 0.000300\n", "2020-02-20 12:05:45,060 Epoch 1 Step: 500 Batch Loss: 4.752475 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 12:06:13,895 Epoch 1 Step: 600 Batch Loss: 4.454239 Tokens per Sec: 7485, Lr: 0.000300\n", "2020-02-20 12:06:42,961 Epoch 1 Step: 700 Batch Loss: 4.371806 Tokens per Sec: 7564, Lr: 0.000300\n", "2020-02-20 12:07:11,959 Epoch 1 Step: 800 Batch Loss: 4.450417 Tokens per Sec: 7661, Lr: 0.000300\n", "2020-02-20 12:07:40,856 Epoch 1 Step: 900 Batch Loss: 4.141620 Tokens per Sec: 7539, Lr: 0.000300\n", "2020-02-20 12:08:09,675 Epoch 1 Step: 1000 Batch Loss: 4.208232 Tokens per Sec: 7618, Lr: 0.000300\n", "2020-02-20 12:09:37,993 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:09:37,993 Saving new checkpoint.\n", "2020-02-20 12:09:38,204 Example #0\n", "2020-02-20 12:09:38,204 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:09:38,204 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:09:38,204 \tHypothesis: Mano kaka wanyalo bedo gi Jehova , Jehova nyalo bedo gi Jehova .\n", "2020-02-20 12:09:38,204 Example #1\n", "2020-02-20 12:09:38,205 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:09:38,205 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:09:38,205 \tHypothesis: Kata kamano , ne ok wabedo gi gik ma ne ok ok wabedo gi Nyasaye . ”\n", "2020-02-20 12:09:38,205 Example #2\n", "2020-02-20 12:09:38,205 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:09:38,205 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:09:38,205 \tHypothesis: Kata kamano , wanyalo bedo gi gik ma ne wabedo gi gik ma ne watimo kamano .\n", "2020-02-20 12:09:38,205 Example #3\n", "2020-02-20 12:09:38,205 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:09:38,205 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:09:38,205 \tHypothesis: Ne en ni “ Nyasaye nowacho ni “ Nyasaye , ” ma ne en gima ne en gima ne en gima ne en gima ne en gima ne en gima ne en gima ne en gima ne en gima ne en . ”\n", "2020-02-20 12:09:38,206 Validation result at epoch 1, step 1000: bleu: 1.01, loss: 98175.9766, ppl: 53.6843, duration: 88.5302s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 12:10:07,191 Epoch 1 Step: 1100 Batch Loss: 3.762488 Tokens per Sec: 7596, Lr: 0.000300\n", "2020-02-20 12:10:36,062 Epoch 1 Step: 1200 Batch Loss: 4.396629 Tokens per Sec: 7597, Lr: 0.000300\n", "2020-02-20 12:11:04,709 Epoch 1 Step: 1300 Batch Loss: 4.256168 Tokens per Sec: 7514, Lr: 0.000300\n", "2020-02-20 12:11:33,621 Epoch 1 Step: 1400 Batch Loss: 3.945779 Tokens per Sec: 7441, Lr: 0.000300\n", "2020-02-20 12:12:02,358 Epoch 1 Step: 1500 Batch Loss: 3.935235 Tokens per Sec: 7623, Lr: 0.000300\n", "2020-02-20 12:12:07,028 Epoch 1: total training loss 6731.63\n", "2020-02-20 12:12:07,029 EPOCH 2\n", "2020-02-20 12:12:31,139 Epoch 2 Step: 1600 Batch Loss: 3.035681 Tokens per Sec: 7441, Lr: 0.000300\n", "2020-02-20 12:12:59,846 Epoch 2 Step: 1700 Batch Loss: 3.587291 Tokens per Sec: 7593, Lr: 0.000300\n", "2020-02-20 12:13:28,871 Epoch 2 Step: 1800 Batch Loss: 3.648604 Tokens per Sec: 7519, Lr: 0.000300\n", "2020-02-20 12:13:57,438 Epoch 2 Step: 1900 Batch Loss: 2.682493 Tokens per Sec: 7478, Lr: 0.000300\n", "2020-02-20 12:14:26,622 Epoch 2 Step: 2000 Batch Loss: 3.596354 Tokens per Sec: 7753, Lr: 0.000300\n", "2020-02-20 12:15:54,913 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:15:54,913 Saving new checkpoint.\n", "2020-02-20 12:15:55,109 Example #0\n", "2020-02-20 12:15:55,110 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:15:55,110 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:15:55,110 \tHypothesis: Obiro konyowa mondo wakaw okang ’ ma Jehova dwaro ni wayie motegno e wi Nyasaye .\n", "2020-02-20 12:15:55,110 Example #1\n", "2020-02-20 12:15:55,110 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:15:55,110 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:15:55,110 \tHypothesis: Mano kaka ne ok en gima ne ok nyal bedo gi yie kendo ne ok giyie gi Nyasaye . ”\n", "2020-02-20 12:15:55,110 Example #2\n", "2020-02-20 12:15:55,110 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:15:55,110 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:15:55,111 \tHypothesis: Kokalo kuom weche ma Muma nyalo konyowa mondo wakaw okang ’ mar Muma , kendo waparo ni wayie motegno e wi gik ma nyalo miyo ng’ato obed gi geno mar timo .\n", "2020-02-20 12:15:55,111 Example #3\n", "2020-02-20 12:15:55,111 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:15:55,111 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:15:55,111 \tHypothesis: Ne giwuoyo kuom “ oganda Nyasaye ” ma ne dhi timore e wi Yesu , kendo ne dhi bedo gi geno mar Yesu .\n", "2020-02-20 12:15:55,111 Validation result at epoch 2, step 2000: bleu: 2.74, loss: 83177.7891, ppl: 29.2134, duration: 88.4880s\n", "2020-02-20 12:16:23,645 Epoch 2 Step: 2100 Batch Loss: 3.629211 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 12:16:52,681 Epoch 2 Step: 2200 Batch Loss: 3.622419 Tokens per Sec: 7664, Lr: 0.000300\n", "2020-02-20 12:17:21,336 Epoch 2 Step: 2300 Batch Loss: 3.560351 Tokens per Sec: 7579, Lr: 0.000300\n", "2020-02-20 12:17:50,095 Epoch 2 Step: 2400 Batch Loss: 3.292273 Tokens per Sec: 7494, Lr: 0.000300\n", "2020-02-20 12:18:19,258 Epoch 2 Step: 2500 Batch Loss: 3.300863 Tokens per Sec: 7592, Lr: 0.000300\n", "2020-02-20 12:18:48,170 Epoch 2 Step: 2600 Batch Loss: 2.710793 Tokens per Sec: 7468, Lr: 0.000300\n", "2020-02-20 12:19:16,980 Epoch 2 Step: 2700 Batch Loss: 3.404884 Tokens per Sec: 7585, Lr: 0.000300\n", "2020-02-20 12:19:45,680 Epoch 2 Step: 2800 Batch Loss: 2.982437 Tokens per Sec: 7570, Lr: 0.000300\n", "2020-02-20 12:20:14,533 Epoch 2 Step: 2900 Batch Loss: 2.992396 Tokens per Sec: 7690, Lr: 0.000300\n", "2020-02-20 12:20:43,253 Epoch 2 Step: 3000 Batch Loss: 2.899553 Tokens per Sec: 7606, Lr: 0.000300\n", "2020-02-20 12:22:11,433 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:22:11,433 Saving new checkpoint.\n", "2020-02-20 12:22:11,627 Example #0\n", "2020-02-20 12:22:11,627 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:22:11,627 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:22:11,627 \tHypothesis: Jonyuol ma Jehova dwaro ni wabed gi yie motegno e wi Nyasaye , onego omi wabed gi paro ma Nyasaye nigo .\n", "2020-02-20 12:22:11,627 Example #1\n", "2020-02-20 12:22:11,627 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:22:11,627 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:22:11,628 \tHypothesis: Bende , ne awinjo ni ne amor gi Nyasaye kendo ne amor . ”\n", "2020-02-20 12:22:11,628 Example #2\n", "2020-02-20 12:22:11,628 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:22:11,628 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:22:11,628 \tHypothesis: Tem ane kaka wanyalo ng’eyo weche manie Muma e wi gik ma nyalo miyo ng’ato obed gi chuny mar timo .\n", "2020-02-20 12:22:11,628 Example #3\n", "2020-02-20 12:22:11,628 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:22:11,628 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:22:11,628 \tHypothesis: Ne giparo ni “ Nyasaye non ji duto ma ne ochiwo e wi roho maler . ”\n", "2020-02-20 12:22:11,628 Validation result at epoch 2, step 3000: bleu: 4.86, loss: 75435.2109, ppl: 21.3382, duration: 88.3752s\n", "2020-02-20 12:22:22,268 Epoch 2: total training loss 5145.10\n", "2020-02-20 12:22:22,269 EPOCH 3\n", "2020-02-20 12:22:40,557 Epoch 3 Step: 3100 Batch Loss: 3.012194 Tokens per Sec: 7509, Lr: 0.000300\n", "2020-02-20 12:23:09,222 Epoch 3 Step: 3200 Batch Loss: 2.736670 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 12:23:38,282 Epoch 3 Step: 3300 Batch Loss: 3.120730 Tokens per Sec: 7632, Lr: 0.000300\n", "2020-02-20 12:24:06,960 Epoch 3 Step: 3400 Batch Loss: 3.273985 Tokens per Sec: 7542, Lr: 0.000300\n", "2020-02-20 12:24:35,743 Epoch 3 Step: 3500 Batch Loss: 3.094509 Tokens per Sec: 7650, Lr: 0.000300\n", "2020-02-20 12:25:04,392 Epoch 3 Step: 3600 Batch Loss: 3.046881 Tokens per Sec: 7579, Lr: 0.000300\n", "2020-02-20 12:25:33,262 Epoch 3 Step: 3700 Batch Loss: 3.162375 Tokens per Sec: 7671, Lr: 0.000300\n", "2020-02-20 12:26:02,164 Epoch 3 Step: 3800 Batch Loss: 3.190458 Tokens per Sec: 7655, Lr: 0.000300\n", "2020-02-20 12:26:30,733 Epoch 3 Step: 3900 Batch Loss: 3.196185 Tokens per Sec: 7489, Lr: 0.000300\n", "2020-02-20 12:26:59,709 Epoch 3 Step: 4000 Batch Loss: 3.097071 Tokens per Sec: 7623, Lr: 0.000300\n", "2020-02-20 12:28:27,888 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:28:27,889 Saving new checkpoint.\n", "2020-02-20 12:28:28,107 Example #0\n", "2020-02-20 12:28:28,107 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:28:28,108 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:28:28,108 \tHypothesis: Ne giyie mondo Jehova otagi mondo omi giwinj tiend wach Nyasaye .\n", "2020-02-20 12:28:28,108 Example #1\n", "2020-02-20 12:28:28,108 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:28:28,108 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:28:28,108 \tHypothesis: Bende , ne awachona ni , kendo ne awinjo ka an gi yie . ”\n", "2020-02-20 12:28:28,108 Example #2\n", "2020-02-20 12:28:28,108 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:28:28,108 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:28:28,108 \tHypothesis: Tem ane gik ma Muma wacho e wi gik ma nyalo timore e wi gik ma nyalo miyo ng’ato obed gi chuny maler .\n", "2020-02-20 12:28:28,109 Example #3\n", "2020-02-20 12:28:28,109 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:28:28,109 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:28:28,109 \tHypothesis: Ne owachone ni “ Nyasaye duto ma ne osetiedho ji duto ma ne ni e ndalo mag giko . ”\n", "2020-02-20 12:28:28,109 Validation result at epoch 3, step 4000: bleu: 7.70, loss: 69503.4297, ppl: 16.7742, duration: 88.3992s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 12:28:56,806 Epoch 3 Step: 4100 Batch Loss: 2.952160 Tokens per Sec: 7502, Lr: 0.000300\n", "2020-02-20 12:29:25,371 Epoch 3 Step: 4200 Batch Loss: 3.406082 Tokens per Sec: 7499, Lr: 0.000300\n", "2020-02-20 12:29:54,297 Epoch 3 Step: 4300 Batch Loss: 3.000547 Tokens per Sec: 7585, Lr: 0.000300\n", "2020-02-20 12:30:22,684 Epoch 3 Step: 4400 Batch Loss: 3.101769 Tokens per Sec: 7585, Lr: 0.000300\n", "2020-02-20 12:30:51,661 Epoch 3 Step: 4500 Batch Loss: 2.918369 Tokens per Sec: 7768, Lr: 0.000300\n", "2020-02-20 12:31:07,842 Epoch 3: total training loss 4516.30\n", "2020-02-20 12:31:07,843 EPOCH 4\n", "2020-02-20 12:31:20,580 Epoch 4 Step: 4600 Batch Loss: 2.786486 Tokens per Sec: 7292, Lr: 0.000300\n", "2020-02-20 12:31:49,217 Epoch 4 Step: 4700 Batch Loss: 2.668926 Tokens per Sec: 7503, Lr: 0.000300\n", "2020-02-20 12:32:18,079 Epoch 4 Step: 4800 Batch Loss: 2.563730 Tokens per Sec: 7651, Lr: 0.000300\n", "2020-02-20 12:32:47,159 Epoch 4 Step: 4900 Batch Loss: 2.711700 Tokens per Sec: 7726, Lr: 0.000300\n", "2020-02-20 12:33:16,085 Epoch 4 Step: 5000 Batch Loss: 2.843105 Tokens per Sec: 7747, Lr: 0.000300\n", "2020-02-20 12:34:44,283 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:34:44,284 Saving new checkpoint.\n", "2020-02-20 12:34:44,499 Example #0\n", "2020-02-20 12:34:44,500 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:34:44,500 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:34:44,500 \tHypothesis: Ne gitemo nyiso ni Jehova dwaro ni giher ni giher ni giluw chike Nyasaye .\n", "2020-02-20 12:34:44,500 Example #1\n", "2020-02-20 12:34:44,500 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:34:44,500 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:34:44,500 \tHypothesis: Bende , amor ahinya mondo awinj luor kendo amor gi osiepena . ”\n", "2020-02-20 12:34:44,500 Example #2\n", "2020-02-20 12:34:44,501 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:34:44,501 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:34:44,501 \tHypothesis: Bedo gi paro matut kuom weche manie Ndiko nyalo miyo waket pachwa kuom gik ma nyalo miyo ng’ato obed gi chuny maler .\n", "2020-02-20 12:34:44,501 Example #3\n", "2020-02-20 12:34:44,501 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:34:44,501 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:34:44,501 \tHypothesis: Ne owuoyo kuom weche ma ne okor e wi “ gik moko duto ma Nyasaye notiyogo e kinde machon . ”\n", "2020-02-20 12:34:44,501 Validation result at epoch 4, step 5000: bleu: 9.56, loss: 65853.0234, ppl: 14.4651, duration: 88.4158s\n", "2020-02-20 12:35:12,990 Epoch 4 Step: 5100 Batch Loss: 2.572210 Tokens per Sec: 7391, Lr: 0.000300\n", "2020-02-20 12:35:42,102 Epoch 4 Step: 5200 Batch Loss: 2.858901 Tokens per Sec: 7599, Lr: 0.000300\n", "2020-02-20 12:36:11,060 Epoch 4 Step: 5300 Batch Loss: 2.993764 Tokens per Sec: 7639, Lr: 0.000300\n", "2020-02-20 12:36:39,920 Epoch 4 Step: 5400 Batch Loss: 2.127422 Tokens per Sec: 7607, Lr: 0.000300\n", "2020-02-20 12:37:08,685 Epoch 4 Step: 5500 Batch Loss: 2.690960 Tokens per Sec: 7572, Lr: 0.000300\n", "2020-02-20 12:37:37,664 Epoch 4 Step: 5600 Batch Loss: 3.039052 Tokens per Sec: 7527, Lr: 0.000300\n", "2020-02-20 12:38:06,762 Epoch 4 Step: 5700 Batch Loss: 2.761341 Tokens per Sec: 7523, Lr: 0.000300\n", "2020-02-20 12:38:35,581 Epoch 4 Step: 5800 Batch Loss: 2.505060 Tokens per Sec: 7476, Lr: 0.000300\n", "2020-02-20 12:39:04,462 Epoch 4 Step: 5900 Batch Loss: 2.251678 Tokens per Sec: 7545, Lr: 0.000300\n", "2020-02-20 12:39:33,184 Epoch 4 Step: 6000 Batch Loss: 2.194507 Tokens per Sec: 7611, Lr: 0.000300\n", "2020-02-20 12:41:01,319 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:41:01,320 Saving new checkpoint.\n", "2020-02-20 12:41:01,534 Example #0\n", "2020-02-20 12:41:01,534 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:41:01,534 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:41:01,534 \tHypothesis: Geno ma Jehova dwaro ni watim gik ma dwaro ni watim mondo omi dhano obed gi paro makare .\n", "2020-02-20 12:41:01,535 Example #1\n", "2020-02-20 12:41:01,535 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:41:01,535 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:41:01,535 \tHypothesis: Bende , amor ahinya mondo awinj tiend luor kod luor ma en - go mar Nyasaye . ”\n", "2020-02-20 12:41:01,535 Example #2\n", "2020-02-20 12:41:01,535 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:41:01,535 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:41:01,535 \tHypothesis: Kuom adier , ka waparo matut kuom Ndiko nyalo miyo ng’ato obed gi chuny maler .\n", "2020-02-20 12:41:01,535 Example #3\n", "2020-02-20 12:41:01,536 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:41:01,536 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:41:01,536 \tHypothesis: Ne ooro weche ma ne okor e wi “ gik moko duto ma Nyasaye nowacho e wi ndalo mag giko . ”\n", "2020-02-20 12:41:01,536 Validation result at epoch 4, step 6000: bleu: 11.00, loss: 63031.2188, ppl: 12.9004, duration: 88.3515s\n", "2020-02-20 12:41:23,076 Epoch 4: total training loss 4153.75\n", "2020-02-20 12:41:23,076 EPOCH 5\n", "2020-02-20 12:41:30,384 Epoch 5 Step: 6100 Batch Loss: 2.903340 Tokens per Sec: 7358, Lr: 0.000300\n", "2020-02-20 12:41:59,188 Epoch 5 Step: 6200 Batch Loss: 2.727892 Tokens per Sec: 7711, Lr: 0.000300\n", "2020-02-20 12:42:27,997 Epoch 5 Step: 6300 Batch Loss: 2.482589 Tokens per Sec: 7594, Lr: 0.000300\n", "2020-02-20 12:42:57,040 Epoch 5 Step: 6400 Batch Loss: 2.914473 Tokens per Sec: 7581, Lr: 0.000300\n", "2020-02-20 12:43:25,492 Epoch 5 Step: 6500 Batch Loss: 2.596678 Tokens per Sec: 7547, Lr: 0.000300\n", "2020-02-20 12:43:54,212 Epoch 5 Step: 6600 Batch Loss: 2.626672 Tokens per Sec: 7422, Lr: 0.000300\n", "2020-02-20 12:44:22,613 Epoch 5 Step: 6700 Batch Loss: 2.924554 Tokens per Sec: 7495, Lr: 0.000300\n", "2020-02-20 12:44:51,306 Epoch 5 Step: 6800 Batch Loss: 2.206463 Tokens per Sec: 7455, Lr: 0.000300\n", "2020-02-20 12:45:20,424 Epoch 5 Step: 6900 Batch Loss: 2.519378 Tokens per Sec: 7674, Lr: 0.000300\n", "2020-02-20 12:45:49,385 Epoch 5 Step: 7000 Batch Loss: 2.784849 Tokens per Sec: 7641, Lr: 0.000300\n", "2020-02-20 12:47:17,551 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:47:17,552 Saving new checkpoint.\n", "2020-02-20 12:47:17,769 Example #0\n", "2020-02-20 12:47:17,769 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:47:17,769 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:47:17,769 \tHypothesis: Gweth ma Jehova dwaro ni giluw gima onego omi omi wabed gi paro mowinjore kuom Nyasaye .\n", "2020-02-20 12:47:17,769 Example #1\n", "2020-02-20 12:47:17,769 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:47:17,770 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:47:17,770 \tHypothesis: Bende , amor ahinya ni en gi luor ma ne an - go e lamo Nyasaye kendo ma en - go . ”\n", "2020-02-20 12:47:17,770 Example #2\n", "2020-02-20 12:47:17,770 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:47:17,770 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:47:17,770 \tHypothesis: Kuom adier , waparo matut kuom Ndiko nyalo konyowa ng’eyo kaka wanyalo bedo gi chuny mokuwe .\n", "2020-02-20 12:47:17,770 Example #3\n", "2020-02-20 12:47:17,770 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:47:17,770 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:47:17,770 \tHypothesis: Ne ooro weche ma ne okor e wi “ gik moko duto ma Nyasaye nowacho kuom janabi Isaya . ”\n", "2020-02-20 12:47:17,771 Validation result at epoch 5, step 7000: bleu: 11.96, loss: 60268.7461, ppl: 11.5326, duration: 88.3848s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 12:47:46,809 Epoch 5 Step: 7100 Batch Loss: 2.377353 Tokens per Sec: 7536, Lr: 0.000300\n", "2020-02-20 12:48:15,570 Epoch 5 Step: 7200 Batch Loss: 1.793351 Tokens per Sec: 7642, Lr: 0.000300\n", "2020-02-20 12:48:44,278 Epoch 5 Step: 7300 Batch Loss: 2.368781 Tokens per Sec: 7535, Lr: 0.000300\n", "2020-02-20 12:49:12,767 Epoch 5 Step: 7400 Batch Loss: 3.025920 Tokens per Sec: 7526, Lr: 0.000300\n", "2020-02-20 12:49:41,451 Epoch 5 Step: 7500 Batch Loss: 2.123299 Tokens per Sec: 7625, Lr: 0.000300\n", "2020-02-20 12:50:10,079 Epoch 5 Step: 7600 Batch Loss: 2.233503 Tokens per Sec: 7551, Lr: 0.000300\n", "2020-02-20 12:50:10,224 Epoch 5: total training loss 3924.77\n", "2020-02-20 12:50:10,224 EPOCH 6\n", "2020-02-20 12:50:38,666 Epoch 6 Step: 7700 Batch Loss: 2.196222 Tokens per Sec: 7480, Lr: 0.000300\n", "2020-02-20 12:51:07,374 Epoch 6 Step: 7800 Batch Loss: 2.038606 Tokens per Sec: 7539, Lr: 0.000300\n", "2020-02-20 12:51:36,278 Epoch 6 Step: 7900 Batch Loss: 2.617388 Tokens per Sec: 7576, Lr: 0.000300\n", "2020-02-20 12:52:04,641 Epoch 6 Step: 8000 Batch Loss: 2.530022 Tokens per Sec: 7480, Lr: 0.000300\n", "2020-02-20 12:53:32,809 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:53:32,810 Saving new checkpoint.\n", "2020-02-20 12:53:33,028 Example #0\n", "2020-02-20 12:53:33,029 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:53:33,029 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:53:33,029 \tHypothesis: Ginyiso ni giikore luwo gima Jehova dwaro ni mondo omi dhano obed gi paro mowinjore .\n", "2020-02-20 12:53:33,029 Example #1\n", "2020-02-20 12:53:33,029 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:53:33,029 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:53:33,029 \tHypothesis: Bende , achiwo luor ne wuonwa kaka wuoro kendo ni en gi winjruok maber gi Nyasaye . ”\n", "2020-02-20 12:53:33,029 Example #2\n", "2020-02-20 12:53:33,029 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:53:33,029 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:53:33,029 \tHypothesis: Kuom adier , sama wawuoyo e Ndiko e yo ma kamano nyalo miyo ng’ato obed gi chuny maler .\n", "2020-02-20 12:53:33,030 Example #3\n", "2020-02-20 12:53:33,030 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:53:33,030 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:53:33,030 \tHypothesis: Ne okoro weche ma ne okor e wi “ gik moko duto ma Nyasaye nowacho kokalo kuom janabi maler , ma ne okor e kinde ma ne okor . ”\n", "2020-02-20 12:53:33,030 Validation result at epoch 6, step 8000: bleu: 12.98, loss: 58475.8125, ppl: 10.7235, duration: 88.3884s\n", "2020-02-20 12:54:01,854 Epoch 6 Step: 8100 Batch Loss: 2.635342 Tokens per Sec: 7650, Lr: 0.000300\n", "2020-02-20 12:54:30,728 Epoch 6 Step: 8200 Batch Loss: 2.719745 Tokens per Sec: 7628, Lr: 0.000300\n", "2020-02-20 12:54:59,789 Epoch 6 Step: 8300 Batch Loss: 2.731328 Tokens per Sec: 7610, Lr: 0.000300\n", "2020-02-20 12:55:28,959 Epoch 6 Step: 8400 Batch Loss: 2.633184 Tokens per Sec: 7656, Lr: 0.000300\n", "2020-02-20 12:55:57,197 Epoch 6 Step: 8500 Batch Loss: 2.693359 Tokens per Sec: 7364, Lr: 0.000300\n", "2020-02-20 12:56:26,347 Epoch 6 Step: 8600 Batch Loss: 2.647988 Tokens per Sec: 7670, Lr: 0.000300\n", "2020-02-20 12:56:55,051 Epoch 6 Step: 8700 Batch Loss: 2.646337 Tokens per Sec: 7531, Lr: 0.000300\n", "2020-02-20 12:57:24,148 Epoch 6 Step: 8800 Batch Loss: 1.519602 Tokens per Sec: 7492, Lr: 0.000300\n", "2020-02-20 12:57:53,096 Epoch 6 Step: 8900 Batch Loss: 2.236732 Tokens per Sec: 7625, Lr: 0.000300\n", "2020-02-20 12:58:22,081 Epoch 6 Step: 9000 Batch Loss: 2.326519 Tokens per Sec: 7572, Lr: 0.000300\n", "2020-02-20 12:59:50,300 Hooray! New best validation result [ppl]!\n", "2020-02-20 12:59:50,300 Saving new checkpoint.\n", "2020-02-20 12:59:50,516 Example #0\n", "2020-02-20 12:59:50,516 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 12:59:50,517 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 12:59:50,517 \tHypothesis: Gombo ma gichiwo ne Jehova dwaro ni otim gik moko duto ma Nyasaye dwaro ni otim .\n", "2020-02-20 12:59:50,517 Example #1\n", "2020-02-20 12:59:50,517 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 12:59:50,517 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 12:59:50,517 \tHypothesis: Bende , en gima ber mondo awinj ka wuon mare kendo omakore gi Nyasaye . ”\n", "2020-02-20 12:59:50,517 Example #2\n", "2020-02-20 12:59:50,517 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 12:59:50,517 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 12:59:50,517 \tHypothesis: Ket pachwa e Ndiko , nyalo bedo ni ng’ato nyalo ng’eyo tiend weche manie Ndiko .\n", "2020-02-20 12:59:50,517 Example #3\n", "2020-02-20 12:59:50,518 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 12:59:50,518 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 12:59:50,518 \tHypothesis: Ne ooro weche ma ne okor e wi “ gik moko duto ma Nyasaye nowacho kokalo kuom janabi maler . ”\n", "2020-02-20 12:59:50,518 Validation result at epoch 6, step 9000: bleu: 13.53, loss: 57032.0273, ppl: 10.1134, duration: 88.4363s\n", "2020-02-20 13:00:19,367 Epoch 6 Step: 9100 Batch Loss: 2.202389 Tokens per Sec: 7487, Lr: 0.000300\n", "2020-02-20 13:00:26,260 Epoch 6: total training loss 3733.66\n", "2020-02-20 13:00:26,260 EPOCH 7\n", "2020-02-20 13:00:48,498 Epoch 7 Step: 9200 Batch Loss: 1.776751 Tokens per Sec: 7591, Lr: 0.000300\n", "2020-02-20 13:01:17,676 Epoch 7 Step: 9300 Batch Loss: 2.848773 Tokens per Sec: 7698, Lr: 0.000300\n", "2020-02-20 13:01:46,183 Epoch 7 Step: 9400 Batch Loss: 2.161055 Tokens per Sec: 7468, Lr: 0.000300\n", "2020-02-20 13:02:15,160 Epoch 7 Step: 9500 Batch Loss: 2.258479 Tokens per Sec: 7584, Lr: 0.000300\n", "2020-02-20 13:02:44,095 Epoch 7 Step: 9600 Batch Loss: 2.699597 Tokens per Sec: 7486, Lr: 0.000300\n", "2020-02-20 13:03:13,116 Epoch 7 Step: 9700 Batch Loss: 2.499038 Tokens per Sec: 7706, Lr: 0.000300\n", "2020-02-20 13:03:41,895 Epoch 7 Step: 9800 Batch Loss: 2.388414 Tokens per Sec: 7500, Lr: 0.000300\n", "2020-02-20 13:04:10,344 Epoch 7 Step: 9900 Batch Loss: 1.981571 Tokens per Sec: 7464, Lr: 0.000300\n", "2020-02-20 13:04:39,375 Epoch 7 Step: 10000 Batch Loss: 2.537346 Tokens per Sec: 7635, Lr: 0.000300\n", "2020-02-20 13:06:07,385 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:06:07,385 Saving new checkpoint.\n", "2020-02-20 13:06:07,601 Example #0\n", "2020-02-20 13:06:07,601 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:06:07,601 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:06:07,601 \tHypothesis: Ginyalo yie luwo gima Jehova dwaro ni watim mondo omi wabed joma kare .\n", "2020-02-20 13:06:07,602 Example #1\n", "2020-02-20 13:06:07,602 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:06:07,602 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:06:07,602 \tHypothesis: Bende , amiyo abedo gi luor kaka wuoro kod wuonwa . ”\n", "2020-02-20 13:06:07,602 Example #2\n", "2020-02-20 13:06:07,602 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:06:07,602 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:06:07,602 \tHypothesis: Ng’e ni mondo wamed ler e Ndiko , nyalo bedo gi teko mar ng’eyo kaka chuny ng’ato nyalo nenore e chunye .\n", "2020-02-20 13:06:07,602 Example #3\n", "2020-02-20 13:06:07,603 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:06:07,603 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:06:07,603 \tHypothesis: Ne ondiko ni “ giduto ma Nyasaye ne ochiwo , ma ne owachone kuom janabi maler . ”\n", "2020-02-20 13:06:07,603 Validation result at epoch 7, step 10000: bleu: 15.22, loss: 54667.5508, ppl: 9.1883, duration: 88.2267s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 13:06:36,315 Epoch 7 Step: 10100 Batch Loss: 2.194897 Tokens per Sec: 7569, Lr: 0.000300\n", "2020-02-20 13:07:04,902 Epoch 7 Step: 10200 Batch Loss: 2.210200 Tokens per Sec: 7497, Lr: 0.000300\n", "2020-02-20 13:07:33,669 Epoch 7 Step: 10300 Batch Loss: 1.960394 Tokens per Sec: 7572, Lr: 0.000300\n", "2020-02-20 13:08:02,569 Epoch 7 Step: 10400 Batch Loss: 2.159416 Tokens per Sec: 7490, Lr: 0.000300\n", "2020-02-20 13:08:31,622 Epoch 7 Step: 10500 Batch Loss: 2.274217 Tokens per Sec: 7653, Lr: 0.000300\n", "2020-02-20 13:09:00,132 Epoch 7 Step: 10600 Batch Loss: 2.411942 Tokens per Sec: 7477, Lr: 0.000300\n", "2020-02-20 13:09:13,505 Epoch 7: total training loss 3569.71\n", "2020-02-20 13:09:13,506 EPOCH 8\n", "2020-02-20 13:09:28,869 Epoch 8 Step: 10700 Batch Loss: 2.155770 Tokens per Sec: 7371, Lr: 0.000300\n", "2020-02-20 13:09:57,690 Epoch 8 Step: 10800 Batch Loss: 1.774834 Tokens per Sec: 7595, Lr: 0.000300\n", "2020-02-20 13:10:26,338 Epoch 8 Step: 10900 Batch Loss: 2.219854 Tokens per Sec: 7511, Lr: 0.000300\n", "2020-02-20 13:10:54,983 Epoch 8 Step: 11000 Batch Loss: 2.169687 Tokens per Sec: 7522, Lr: 0.000300\n", "2020-02-20 13:12:23,148 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:12:23,148 Saving new checkpoint.\n", "2020-02-20 13:12:23,367 Example #0\n", "2020-02-20 13:12:23,368 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:12:23,368 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:12:23,368 \tHypothesis: Giikore luwo gima Jehova dwaro ni mondo omi dhano obed gi paro makare .\n", "2020-02-20 13:12:23,368 Example #1\n", "2020-02-20 13:12:23,368 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:12:23,368 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:12:23,368 \tHypothesis: Bende , ahero miyo wuonwa luor kendo e wi Nyasaye . ”\n", "2020-02-20 13:12:23,368 Example #2\n", "2020-02-20 13:12:23,369 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:12:23,369 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:12:23,369 \tHypothesis: Bedo gi ler e Ndiko e yo matut , nyalo bedo ni iwuoyo kuom gik ma nyalo miyo ng’ato obed gi teko e chunye .\n", "2020-02-20 13:12:23,369 Example #3\n", "2020-02-20 13:12:23,369 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:12:23,369 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:12:23,369 \tHypothesis: Ne ondiko ni “ gik moko duto ma Nyasaye notiyogo e kinde ma ne en rawera . ”\n", "2020-02-20 13:12:23,369 Validation result at epoch 8, step 11000: bleu: 15.06, loss: 53643.9844, ppl: 8.8145, duration: 88.3858s\n", "2020-02-20 13:12:52,436 Epoch 8 Step: 11100 Batch Loss: 2.169235 Tokens per Sec: 7743, Lr: 0.000300\n", "2020-02-20 13:13:21,121 Epoch 8 Step: 11200 Batch Loss: 2.335445 Tokens per Sec: 7505, Lr: 0.000300\n", "2020-02-20 13:13:50,408 Epoch 8 Step: 11300 Batch Loss: 2.389638 Tokens per Sec: 7737, Lr: 0.000300\n", "2020-02-20 13:14:19,395 Epoch 8 Step: 11400 Batch Loss: 2.325415 Tokens per Sec: 7615, Lr: 0.000300\n", "2020-02-20 13:14:48,605 Epoch 8 Step: 11500 Batch Loss: 2.009950 Tokens per Sec: 7586, Lr: 0.000300\n", "2020-02-20 13:15:17,295 Epoch 8 Step: 11600 Batch Loss: 2.069922 Tokens per Sec: 7589, Lr: 0.000300\n", "2020-02-20 13:15:46,339 Epoch 8 Step: 11700 Batch Loss: 2.321386 Tokens per Sec: 7531, Lr: 0.000300\n", "2020-02-20 13:16:14,865 Epoch 8 Step: 11800 Batch Loss: 2.287055 Tokens per Sec: 7496, Lr: 0.000300\n", "2020-02-20 13:16:43,994 Epoch 8 Step: 11900 Batch Loss: 2.203277 Tokens per Sec: 7641, Lr: 0.000300\n", "2020-02-20 13:17:12,778 Epoch 8 Step: 12000 Batch Loss: 2.154271 Tokens per Sec: 7618, Lr: 0.000300\n", "2020-02-20 13:18:40,903 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:18:40,903 Saving new checkpoint.\n", "2020-02-20 13:18:41,120 Example #0\n", "2020-02-20 13:18:41,120 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:18:41,120 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:18:41,120 \tHypothesis: Giikore luwo chike ma Jehova dwaro ni mondo omi giluw chike mag Nyasaye .\n", "2020-02-20 13:18:41,120 Example #1\n", "2020-02-20 13:18:41,120 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:18:41,121 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:18:41,121 \tHypothesis: Bende , omiya luor momedore mar bedo wuon - gi kendo tiyo ne Nyasaye . ”\n", "2020-02-20 13:18:41,121 Example #2\n", "2020-02-20 13:18:41,121 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:18:41,121 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:18:41,121 \tHypothesis: Ket pachwa kuom gik ma Ndiko wacho nyalo bedo ni ok wanyal ng’eyo ni ok wanyal ng’eyo kaka chunywa dwaro .\n", "2020-02-20 13:18:41,121 Example #3\n", "2020-02-20 13:18:41,121 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:18:41,121 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:18:41,121 \tHypothesis: Ne ondiko weche ma ne okor e wi “ gik moko duto ma Nyasaye nowacho kokalo kuom janabi maler mar roho maler . ”\n", "2020-02-20 13:18:41,122 Validation result at epoch 8, step 12000: bleu: 15.88, loss: 52487.8945, ppl: 8.4107, duration: 88.3429s\n", "2020-02-20 13:19:09,691 Epoch 8 Step: 12100 Batch Loss: 2.467495 Tokens per Sec: 7483, Lr: 0.000300\n", "2020-02-20 13:19:28,266 Epoch 8: total training loss 3424.48\n", "2020-02-20 13:19:28,267 EPOCH 9\n", "2020-02-20 13:19:38,825 Epoch 9 Step: 12200 Batch Loss: 2.104245 Tokens per Sec: 7420, Lr: 0.000300\n", "2020-02-20 13:20:07,670 Epoch 9 Step: 12300 Batch Loss: 2.073637 Tokens per Sec: 7726, Lr: 0.000300\n", "2020-02-20 13:20:36,639 Epoch 9 Step: 12400 Batch Loss: 2.399733 Tokens per Sec: 7617, Lr: 0.000300\n", "2020-02-20 13:21:05,343 Epoch 9 Step: 12500 Batch Loss: 2.354544 Tokens per Sec: 7494, Lr: 0.000300\n", "2020-02-20 13:21:34,006 Epoch 9 Step: 12600 Batch Loss: 1.874124 Tokens per Sec: 7479, Lr: 0.000300\n", "2020-02-20 13:22:02,608 Epoch 9 Step: 12700 Batch Loss: 2.101764 Tokens per Sec: 7585, Lr: 0.000300\n", "2020-02-20 13:22:31,258 Epoch 9 Step: 12800 Batch Loss: 1.753699 Tokens per Sec: 7707, Lr: 0.000300\n", "2020-02-20 13:23:00,161 Epoch 9 Step: 12900 Batch Loss: 2.401218 Tokens per Sec: 7578, Lr: 0.000300\n", "2020-02-20 13:23:29,053 Epoch 9 Step: 13000 Batch Loss: 1.602911 Tokens per Sec: 7646, Lr: 0.000300\n", "2020-02-20 13:24:57,106 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:24:57,106 Saving new checkpoint.\n", "2020-02-20 13:24:57,320 Example #0\n", "2020-02-20 13:24:57,320 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:24:57,320 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:24:57,320 \tHypothesis: Luwo chikene ne Jehova , dwaro ni mondo omi Jehova obed gi paro mowinjore .\n", "2020-02-20 13:24:57,320 Example #1\n", "2020-02-20 13:24:57,321 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:24:57,321 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:24:57,321 \tHypothesis: Bende , amiyo abedo gi luor kaka wuoro kendo e wi winjruokna gi Nyasaye . ”\n", "2020-02-20 13:24:57,321 Example #2\n", "2020-02-20 13:24:57,321 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:24:57,321 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:24:57,321 \tHypothesis: Ket pachi kuom gik ma Ndiko wacho e wi Muma , nyalo bedo gima nyiso ni en gi teko mar ng’eyo gik ma nyalo miyo ng’ato otim .\n", "2020-02-20 13:24:57,321 Example #3\n", "2020-02-20 13:24:57,322 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:24:57,322 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:24:57,322 \tHypothesis: Ne ondiko e wi “ gik moko duto ma Nyasaye nowacho kokalo kuom janabi maler , ma ne en janabi maler . ”\n", "2020-02-20 13:24:57,322 Validation result at epoch 9, step 13000: bleu: 16.27, loss: 51491.7305, ppl: 8.0775, duration: 88.2683s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 13:25:26,199 Epoch 9 Step: 13100 Batch Loss: 1.966934 Tokens per Sec: 7477, Lr: 0.000300\n", "2020-02-20 13:25:54,608 Epoch 9 Step: 13200 Batch Loss: 2.030701 Tokens per Sec: 7424, Lr: 0.000300\n", "2020-02-20 13:26:23,230 Epoch 9 Step: 13300 Batch Loss: 2.384551 Tokens per Sec: 7609, Lr: 0.000300\n", "2020-02-20 13:26:52,018 Epoch 9 Step: 13400 Batch Loss: 2.075567 Tokens per Sec: 7556, Lr: 0.000300\n", "2020-02-20 13:27:20,908 Epoch 9 Step: 13500 Batch Loss: 2.342129 Tokens per Sec: 7744, Lr: 0.000300\n", "2020-02-20 13:27:50,110 Epoch 9 Step: 13600 Batch Loss: 2.523449 Tokens per Sec: 7784, Lr: 0.000300\n", "2020-02-20 13:28:13,814 Epoch 9: total training loss 3318.59\n", "2020-02-20 13:28:13,814 EPOCH 10\n", "2020-02-20 13:28:19,276 Epoch 10 Step: 13700 Batch Loss: 2.016206 Tokens per Sec: 7363, Lr: 0.000300\n", "2020-02-20 13:28:48,178 Epoch 10 Step: 13800 Batch Loss: 2.267726 Tokens per Sec: 7665, Lr: 0.000300\n", "2020-02-20 13:29:16,816 Epoch 10 Step: 13900 Batch Loss: 1.974838 Tokens per Sec: 7502, Lr: 0.000300\n", "2020-02-20 13:29:45,897 Epoch 10 Step: 14000 Batch Loss: 2.123787 Tokens per Sec: 7670, Lr: 0.000300\n", "2020-02-20 13:31:14,015 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:31:14,015 Saving new checkpoint.\n", "2020-02-20 13:31:14,234 Example #0\n", "2020-02-20 13:31:14,234 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:31:14,234 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:31:14,234 \tHypothesis: Ka giikore winjo wach Jehova , onego onyis ni loch mar Nyasaye ok nyal timo gik moko .\n", "2020-02-20 13:31:14,234 Example #1\n", "2020-02-20 13:31:14,235 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:31:14,235 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:31:14,235 \tHypothesis: Bende , amiyo amiye luor kaka wuonwa koda e wi Nyasaye . ”\n", "2020-02-20 13:31:14,235 Example #2\n", "2020-02-20 13:31:14,235 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:31:14,235 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:31:14,235 \tHypothesis: Bedo gi ler e yo matut kuom gima Ndiko wacho nyalo bedo gi teko e fwenyo ni chuny ng’ato nyalo fwenyo .\n", "2020-02-20 13:31:14,235 Example #3\n", "2020-02-20 13:31:14,235 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:31:14,236 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:31:14,236 \tHypothesis: Ne ondiko e wi “ gik moko duto ma Nyasaye ne wuoyo kuome e wi janabi maler . ”\n", "2020-02-20 13:31:14,236 Validation result at epoch 10, step 14000: bleu: 17.06, loss: 50492.8320, ppl: 7.7567, duration: 88.3380s\n", "2020-02-20 13:31:42,985 Epoch 10 Step: 14100 Batch Loss: 2.337510 Tokens per Sec: 7557, Lr: 0.000300\n", "2020-02-20 13:32:11,735 Epoch 10 Step: 14200 Batch Loss: 2.451478 Tokens per Sec: 7623, Lr: 0.000300\n", "2020-02-20 13:32:40,354 Epoch 10 Step: 14300 Batch Loss: 2.118098 Tokens per Sec: 7528, Lr: 0.000300\n", "2020-02-20 13:33:09,327 Epoch 10 Step: 14400 Batch Loss: 2.292750 Tokens per Sec: 7625, Lr: 0.000300\n", "2020-02-20 13:33:38,332 Epoch 10 Step: 14500 Batch Loss: 2.262671 Tokens per Sec: 7602, Lr: 0.000300\n", "2020-02-20 13:34:07,505 Epoch 10 Step: 14600 Batch Loss: 1.570102 Tokens per Sec: 7724, Lr: 0.000300\n", "2020-02-20 13:34:36,435 Epoch 10 Step: 14700 Batch Loss: 2.276570 Tokens per Sec: 7625, Lr: 0.000300\n", "2020-02-20 13:35:05,400 Epoch 10 Step: 14800 Batch Loss: 2.175345 Tokens per Sec: 7543, Lr: 0.000300\n", "2020-02-20 13:35:34,127 Epoch 10 Step: 14900 Batch Loss: 2.063216 Tokens per Sec: 7544, Lr: 0.000300\n", "2020-02-20 13:36:02,679 Epoch 10 Step: 15000 Batch Loss: 2.254591 Tokens per Sec: 7480, Lr: 0.000300\n", "2020-02-20 13:37:30,781 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:37:30,781 Saving new checkpoint.\n", "2020-02-20 13:37:30,997 Example #0\n", "2020-02-20 13:37:30,998 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:37:30,998 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:37:30,998 \tHypothesis: Chiwruok margi giwegi gi winjo gima Jehova dwaro ni watim , onego onyis ni loch mar Nyasaye en gima owinjore .\n", "2020-02-20 13:37:30,998 Example #1\n", "2020-02-20 13:37:30,998 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:37:30,998 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:37:30,998 \tHypothesis: Bende , amiyo awinj ka an gi luor , kendo ni an gi winjruok maber gi Nyasaye . ”\n", "2020-02-20 13:37:30,998 Example #2\n", "2020-02-20 13:37:30,998 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:37:30,999 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:37:30,999 \tHypothesis: Ng’eyo ni wan wawegi warieyo gik ma Ndiko wacho nyalo bedo gima duong ’ ahinya e fwenyo kaka chuny ng’ato nyalo bedo gi chuny mokuwe .\n", "2020-02-20 13:37:30,999 Example #3\n", "2020-02-20 13:37:30,999 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:37:30,999 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:37:30,999 \tHypothesis: Nondiko weche ma ne okor e wi “ gik moko duto ma Nyasaye ne wuoyo kuome kokalo kuom janabi maler . ”\n", "2020-02-20 13:37:30,999 Validation result at epoch 10, step 15000: bleu: 18.30, loss: 49596.8750, ppl: 7.4798, duration: 88.3195s\n", "2020-02-20 13:37:59,772 Epoch 10 Step: 15100 Batch Loss: 2.018469 Tokens per Sec: 7621, Lr: 0.000300\n", "2020-02-20 13:38:27,994 Epoch 10: total training loss 3217.65\n", "2020-02-20 13:38:27,994 EPOCH 11\n", "2020-02-20 13:38:28,817 Epoch 11 Step: 15200 Batch Loss: 2.301722 Tokens per Sec: 6638, Lr: 0.000300\n", "2020-02-20 13:38:57,385 Epoch 11 Step: 15300 Batch Loss: 1.495556 Tokens per Sec: 7584, Lr: 0.000300\n", "2020-02-20 13:39:25,909 Epoch 11 Step: 15400 Batch Loss: 1.303177 Tokens per Sec: 7512, Lr: 0.000300\n", "2020-02-20 13:39:54,449 Epoch 11 Step: 15500 Batch Loss: 1.359732 Tokens per Sec: 7428, Lr: 0.000300\n", "2020-02-20 13:40:23,046 Epoch 11 Step: 15600 Batch Loss: 2.006906 Tokens per Sec: 7684, Lr: 0.000300\n", "2020-02-20 13:40:52,076 Epoch 11 Step: 15700 Batch Loss: 1.985547 Tokens per Sec: 7695, Lr: 0.000300\n", "2020-02-20 13:41:20,618 Epoch 11 Step: 15800 Batch Loss: 2.046363 Tokens per Sec: 7562, Lr: 0.000300\n", "2020-02-20 13:41:49,285 Epoch 11 Step: 15900 Batch Loss: 1.693688 Tokens per Sec: 7541, Lr: 0.000300\n", "2020-02-20 13:42:17,959 Epoch 11 Step: 16000 Batch Loss: 1.819900 Tokens per Sec: 7676, Lr: 0.000300\n", "2020-02-20 13:43:46,033 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:43:46,034 Saving new checkpoint.\n", "2020-02-20 13:43:46,251 Example #0\n", "2020-02-20 13:43:46,251 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:43:46,251 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:43:46,251 \tHypothesis: Chiwruok margi giwegi gi gik ma Jehova dwaro ni mondo onyis ni loch mar Nyasaye en gima owinjore .\n", "2020-02-20 13:43:46,252 Example #1\n", "2020-02-20 13:43:46,252 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:43:46,252 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:43:46,252 \tHypothesis: Bende , amiyo amiye luor kaka wuonwa koda winjruokna gi Nyasaye . ”\n", "2020-02-20 13:43:46,252 Example #2\n", "2020-02-20 13:43:46,252 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:43:46,252 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:43:46,252 \tHypothesis: Bedo gi ler e ler mar Ndiko , nyalo bedo gima nyiso ni ng’ato nigi teko mar ng’eyo chuny .\n", "2020-02-20 13:43:46,252 Example #3\n", "2020-02-20 13:43:46,253 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:43:46,253 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:43:46,253 \tHypothesis: Ne ondiko e wi “ gik moko duto ma Nyasaye ne wuoyo kuome e dhoge mag ruodhe , ” ma ne ondik e wi “ chiemo duto ma Nyasaye ne tiyogo e dhoge . ”\n", "2020-02-20 13:43:46,253 Validation result at epoch 11, step 16000: bleu: 17.64, loss: 49577.4414, ppl: 7.4739, duration: 88.2932s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 13:44:15,485 Epoch 11 Step: 16100 Batch Loss: 1.779898 Tokens per Sec: 7723, Lr: 0.000300\n", "2020-02-20 13:44:44,008 Epoch 11 Step: 16200 Batch Loss: 1.693069 Tokens per Sec: 7414, Lr: 0.000300\n", "2020-02-20 13:45:12,976 Epoch 11 Step: 16300 Batch Loss: 2.256828 Tokens per Sec: 7699, Lr: 0.000300\n", "2020-02-20 13:45:41,342 Epoch 11 Step: 16400 Batch Loss: 2.473014 Tokens per Sec: 7479, Lr: 0.000300\n", "2020-02-20 13:46:10,144 Epoch 11 Step: 16500 Batch Loss: 1.979323 Tokens per Sec: 7733, Lr: 0.000300\n", "2020-02-20 13:46:38,915 Epoch 11 Step: 16600 Batch Loss: 2.123430 Tokens per Sec: 7579, Lr: 0.000300\n", "2020-02-20 13:47:07,672 Epoch 11 Step: 16700 Batch Loss: 1.495558 Tokens per Sec: 7603, Lr: 0.000300\n", "2020-02-20 13:47:13,399 Epoch 11: total training loss 3163.59\n", "2020-02-20 13:47:13,400 EPOCH 12\n", "2020-02-20 13:47:36,731 Epoch 12 Step: 16800 Batch Loss: 1.909336 Tokens per Sec: 7616, Lr: 0.000300\n", "2020-02-20 13:48:05,468 Epoch 12 Step: 16900 Batch Loss: 2.172665 Tokens per Sec: 7641, Lr: 0.000300\n", "2020-02-20 13:48:34,176 Epoch 12 Step: 17000 Batch Loss: 1.714036 Tokens per Sec: 7586, Lr: 0.000300\n", "2020-02-20 13:50:02,251 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:50:02,251 Saving new checkpoint.\n", "2020-02-20 13:50:02,469 Example #0\n", "2020-02-20 13:50:02,470 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:50:02,470 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:50:02,470 \tHypothesis: Luwo chikene , dwaro ni Jehova ong’ad ni giluw chike Nyasaye .\n", "2020-02-20 13:50:02,470 Example #1\n", "2020-02-20 13:50:02,470 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:50:02,470 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:50:02,470 \tHypothesis: Bende , omiya luor momedore mar miye luor kaka wuonwa kod winjruok maber gi Nyasaye . ”\n", "2020-02-20 13:50:02,470 Example #2\n", "2020-02-20 13:50:02,471 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:50:02,471 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:50:02,471 \tHypothesis: Kuom adier , ng’eyo tiend weche manie Ndiko nyalo bedo gima nyiso ni chuny ng’ato nyalo bedo maber .\n", "2020-02-20 13:50:02,471 Example #3\n", "2020-02-20 13:50:02,471 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:50:02,471 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:50:02,471 \tHypothesis: Nondiko ni “ gik moko duto mag Nyasaye ma Nyasaye nowuoyo kuome e wi janabi maler , ma ne en janabi maler . ”\n", "2020-02-20 13:50:02,472 Validation result at epoch 12, step 17000: bleu: 17.78, loss: 48477.7344, ppl: 7.1478, duration: 88.2948s\n", "2020-02-20 13:50:31,124 Epoch 12 Step: 17100 Batch Loss: 1.820889 Tokens per Sec: 7478, Lr: 0.000300\n", "2020-02-20 13:51:00,244 Epoch 12 Step: 17200 Batch Loss: 2.028999 Tokens per Sec: 7758, Lr: 0.000300\n", "2020-02-20 13:51:29,167 Epoch 12 Step: 17300 Batch Loss: 1.793971 Tokens per Sec: 7674, Lr: 0.000300\n", "2020-02-20 13:51:58,054 Epoch 12 Step: 17400 Batch Loss: 1.918590 Tokens per Sec: 7503, Lr: 0.000300\n", "2020-02-20 13:52:26,828 Epoch 12 Step: 17500 Batch Loss: 2.225218 Tokens per Sec: 7679, Lr: 0.000300\n", "2020-02-20 13:52:55,479 Epoch 12 Step: 17600 Batch Loss: 2.235759 Tokens per Sec: 7542, Lr: 0.000300\n", "2020-02-20 13:53:23,813 Epoch 12 Step: 17700 Batch Loss: 1.580759 Tokens per Sec: 7519, Lr: 0.000300\n", "2020-02-20 13:53:52,179 Epoch 12 Step: 17800 Batch Loss: 2.150620 Tokens per Sec: 7562, Lr: 0.000300\n", "2020-02-20 13:54:21,419 Epoch 12 Step: 17900 Batch Loss: 2.210498 Tokens per Sec: 7535, Lr: 0.000300\n", "2020-02-20 13:54:50,259 Epoch 12 Step: 18000 Batch Loss: 1.910157 Tokens per Sec: 7538, Lr: 0.000300\n", "2020-02-20 13:56:18,355 Hooray! New best validation result [ppl]!\n", "2020-02-20 13:56:18,356 Saving new checkpoint.\n", "2020-02-20 13:56:18,572 Example #0\n", "2020-02-20 13:56:18,573 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 13:56:18,573 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 13:56:18,573 \tHypothesis: Luwo chikene , onego onyis ni Jehova ema dwaro ni ji obed gi paro mowinjore .\n", "2020-02-20 13:56:18,573 Example #1\n", "2020-02-20 13:56:18,573 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 13:56:18,573 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 13:56:18,573 \tHypothesis: Bende , omiya luor ahinya kuom miye luor kaka wuonwa koda wiye . ”\n", "2020-02-20 13:56:18,573 Example #2\n", "2020-02-20 13:56:18,574 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 13:56:18,574 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 13:56:18,574 \tHypothesis: Bedo gi chuny mar ler e weche manie Ndiko nyalo bedo gi teko mar ng’eyo gik ma chuny dwaro .\n", "2020-02-20 13:56:18,574 Example #3\n", "2020-02-20 13:56:18,574 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 13:56:18,574 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 13:56:18,574 \tHypothesis: Ne ondiko weche ma ne ondik e wi “ gik moko duto ma Nyasaye nowacho kokalo kuom janabi maler , ma ne en janabi maler . ”\n", "2020-02-20 13:56:18,574 Validation result at epoch 12, step 18000: bleu: 18.39, loss: 47651.5781, ppl: 6.9122, duration: 88.3147s\n", "2020-02-20 13:56:47,318 Epoch 12 Step: 18100 Batch Loss: 1.852726 Tokens per Sec: 7622, Lr: 0.000300\n", "2020-02-20 13:57:16,447 Epoch 12 Step: 18200 Batch Loss: 1.929286 Tokens per Sec: 7652, Lr: 0.000300\n", "2020-02-20 13:57:27,111 Epoch 12: total training loss 3082.26\n", "2020-02-20 13:57:27,112 EPOCH 13\n", "2020-02-20 13:57:45,315 Epoch 13 Step: 18300 Batch Loss: 1.983091 Tokens per Sec: 7587, Lr: 0.000300\n", "2020-02-20 13:58:13,985 Epoch 13 Step: 18400 Batch Loss: 1.967488 Tokens per Sec: 7543, Lr: 0.000300\n", "2020-02-20 13:58:42,869 Epoch 13 Step: 18500 Batch Loss: 1.554304 Tokens per Sec: 7553, Lr: 0.000300\n", "2020-02-20 13:59:11,558 Epoch 13 Step: 18600 Batch Loss: 2.260455 Tokens per Sec: 7531, Lr: 0.000300\n", "2020-02-20 13:59:40,090 Epoch 13 Step: 18700 Batch Loss: 2.003440 Tokens per Sec: 7649, Lr: 0.000300\n", "2020-02-20 14:00:08,627 Epoch 13 Step: 18800 Batch Loss: 2.120882 Tokens per Sec: 7553, Lr: 0.000300\n", "2020-02-20 14:00:37,738 Epoch 13 Step: 18900 Batch Loss: 2.058761 Tokens per Sec: 7696, Lr: 0.000300\n", "2020-02-20 14:01:06,400 Epoch 13 Step: 19000 Batch Loss: 2.203524 Tokens per Sec: 7554, Lr: 0.000300\n", "2020-02-20 14:02:34,245 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:02:34,246 Saving new checkpoint.\n", "2020-02-20 14:02:34,458 Example #0\n", "2020-02-20 14:02:34,459 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:02:34,459 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:02:34,459 \tHypothesis: Luwo chikene ne dwaro ni wanyis ni chike Nyasaye ema owinjore .\n", "2020-02-20 14:02:34,459 Example #1\n", "2020-02-20 14:02:34,459 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:02:34,459 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:02:34,459 \tHypothesis: Bende , omiya luor momedore e miyo amiye luor kaka wuoro kendo e wi Nyasaye . ”\n", "2020-02-20 14:02:34,459 Example #2\n", "2020-02-20 14:02:34,460 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:02:34,460 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:02:34,460 \tHypothesis: Ng’e ni mondo waler e ler mar Ndiko , wanyalo fwenyo ni chuny ng’ato nyalo fwenyo .\n", "2020-02-20 14:02:34,460 Example #3\n", "2020-02-20 14:02:34,460 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:02:34,460 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:02:34,460 \tHypothesis: Nondiko weche ma ne ondik e wi “ gik moko duto ma Nyasaye nowuoyo kuome e dho - Hibrania machon . ”\n", "2020-02-20 14:02:34,460 Validation result at epoch 13, step 19000: bleu: 18.75, loss: 47321.0898, ppl: 6.8201, duration: 88.0598s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 14:03:03,626 Epoch 13 Step: 19100 Batch Loss: 2.026862 Tokens per Sec: 7638, Lr: 0.000300\n", "2020-02-20 14:03:32,593 Epoch 13 Step: 19200 Batch Loss: 2.185729 Tokens per Sec: 7646, Lr: 0.000300\n", "2020-02-20 14:04:01,428 Epoch 13 Step: 19300 Batch Loss: 1.903270 Tokens per Sec: 7637, Lr: 0.000300\n", "2020-02-20 14:04:29,793 Epoch 13 Step: 19400 Batch Loss: 2.150892 Tokens per Sec: 7482, Lr: 0.000300\n", "2020-02-20 14:04:58,275 Epoch 13 Step: 19500 Batch Loss: 1.806008 Tokens per Sec: 7577, Lr: 0.000300\n", "2020-02-20 14:05:27,178 Epoch 13 Step: 19600 Batch Loss: 1.681479 Tokens per Sec: 7557, Lr: 0.000300\n", "2020-02-20 14:05:55,988 Epoch 13 Step: 19700 Batch Loss: 1.494733 Tokens per Sec: 7634, Lr: 0.000300\n", "2020-02-20 14:06:12,684 Epoch 13: total training loss 3037.68\n", "2020-02-20 14:06:12,684 EPOCH 14\n", "2020-02-20 14:06:24,718 Epoch 14 Step: 19800 Batch Loss: 1.871077 Tokens per Sec: 7269, Lr: 0.000300\n", "2020-02-20 14:06:53,463 Epoch 14 Step: 19900 Batch Loss: 1.997911 Tokens per Sec: 7417, Lr: 0.000300\n", "2020-02-20 14:07:22,548 Epoch 14 Step: 20000 Batch Loss: 2.252568 Tokens per Sec: 7592, Lr: 0.000300\n", "2020-02-20 14:08:50,622 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:08:50,623 Saving new checkpoint.\n", "2020-02-20 14:08:50,839 Example #0\n", "2020-02-20 14:08:50,839 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:08:50,840 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:08:50,840 \tHypothesis: Luwo chike Jehova , onego onyis ni chike Nyasaye ok owinjore .\n", "2020-02-20 14:08:50,840 Example #1\n", "2020-02-20 14:08:50,840 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:08:50,840 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:08:50,840 \tHypothesis: Bende , amiyo amiye luor ne wuonwa koda winjruok maber gi Nyasaye . ”\n", "2020-02-20 14:08:50,840 Example #2\n", "2020-02-20 14:08:50,840 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:08:50,840 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:08:50,840 \tHypothesis: Ng’eyo ni Ndiko wacho adier nyalo bedo gi teko mar ng’eyo chuny .\n", "2020-02-20 14:08:50,841 Example #3\n", "2020-02-20 14:08:50,841 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:08:50,841 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:08:50,841 \tHypothesis: Ne ondiko ni “ weche duto mag Nyasaye ma Nyasaye nowuoyo kuome e dhoge machon . ”\n", "2020-02-20 14:08:50,841 Validation result at epoch 14, step 20000: bleu: 19.04, loss: 46756.9531, ppl: 6.6658, duration: 88.2920s\n", "2020-02-20 14:09:19,810 Epoch 14 Step: 20100 Batch Loss: 2.208756 Tokens per Sec: 7672, Lr: 0.000300\n", "2020-02-20 14:09:48,733 Epoch 14 Step: 20200 Batch Loss: 2.163957 Tokens per Sec: 7615, Lr: 0.000300\n", "2020-02-20 14:10:17,489 Epoch 14 Step: 20300 Batch Loss: 2.115371 Tokens per Sec: 7475, Lr: 0.000300\n", "2020-02-20 14:10:46,010 Epoch 14 Step: 20400 Batch Loss: 1.851070 Tokens per Sec: 7497, Lr: 0.000300\n", "2020-02-20 14:11:15,011 Epoch 14 Step: 20500 Batch Loss: 1.853435 Tokens per Sec: 7626, Lr: 0.000300\n", "2020-02-20 14:11:43,895 Epoch 14 Step: 20600 Batch Loss: 1.965919 Tokens per Sec: 7583, Lr: 0.000300\n", "2020-02-20 14:12:12,501 Epoch 14 Step: 20700 Batch Loss: 2.373045 Tokens per Sec: 7564, Lr: 0.000300\n", "2020-02-20 14:12:41,520 Epoch 14 Step: 20800 Batch Loss: 2.145957 Tokens per Sec: 7713, Lr: 0.000300\n", "2020-02-20 14:13:10,329 Epoch 14 Step: 20900 Batch Loss: 2.221987 Tokens per Sec: 7523, Lr: 0.000300\n", "2020-02-20 14:13:39,002 Epoch 14 Step: 21000 Batch Loss: 2.147277 Tokens per Sec: 7573, Lr: 0.000300\n", "2020-02-20 14:15:07,091 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:15:07,092 Saving new checkpoint.\n", "2020-02-20 14:15:07,309 Example #0\n", "2020-02-20 14:15:07,310 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:15:07,310 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:15:07,310 \tHypothesis: Chiwruok margi giwegi gi winjo wach ma Jehova dwaro ni mondo onyis ni chike mag Nyasaye ok owinjore .\n", "2020-02-20 14:15:07,310 Example #1\n", "2020-02-20 14:15:07,310 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:15:07,310 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:15:07,310 \tHypothesis: Bende , amiyo amiye luor ne wuonwa koda e wi Nyasaye . ”\n", "2020-02-20 14:15:07,310 Example #2\n", "2020-02-20 14:15:07,311 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:15:07,311 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:15:07,311 \tHypothesis: Ng’eyo ni Muma wacho adier nyalo bedo gima duong ’ ahinya e chuny ng’ato .\n", "2020-02-20 14:15:07,311 Example #3\n", "2020-02-20 14:15:07,311 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:15:07,311 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:15:07,311 \tHypothesis: Nondike e wi “ weche duto mag Nyasaye ma Nyasaye nowuoyo kuome e dhoge machon . ”\n", "2020-02-20 14:15:07,311 Validation result at epoch 14, step 21000: bleu: 19.73, loss: 46212.3164, ppl: 6.5201, duration: 88.3087s\n", "2020-02-20 14:15:35,732 Epoch 14 Step: 21100 Batch Loss: 1.614484 Tokens per Sec: 7533, Lr: 0.000300\n", "2020-02-20 14:16:04,341 Epoch 14 Step: 21200 Batch Loss: 1.654272 Tokens per Sec: 7611, Lr: 0.000300\n", "2020-02-20 14:16:27,989 Epoch 14: total training loss 2986.49\n", "2020-02-20 14:16:27,989 EPOCH 15\n", "2020-02-20 14:16:33,131 Epoch 15 Step: 21300 Batch Loss: 1.952488 Tokens per Sec: 7229, Lr: 0.000300\n", "2020-02-20 14:17:01,804 Epoch 15 Step: 21400 Batch Loss: 1.934405 Tokens per Sec: 7552, Lr: 0.000300\n", "2020-02-20 14:17:30,627 Epoch 15 Step: 21500 Batch Loss: 1.955800 Tokens per Sec: 7703, Lr: 0.000300\n", "2020-02-20 14:17:59,378 Epoch 15 Step: 21600 Batch Loss: 2.198103 Tokens per Sec: 7598, Lr: 0.000300\n", "2020-02-20 14:18:28,247 Epoch 15 Step: 21700 Batch Loss: 2.100889 Tokens per Sec: 7703, Lr: 0.000300\n", "2020-02-20 14:18:57,290 Epoch 15 Step: 21800 Batch Loss: 2.066184 Tokens per Sec: 7612, Lr: 0.000300\n", "2020-02-20 14:19:25,967 Epoch 15 Step: 21900 Batch Loss: 1.609360 Tokens per Sec: 7526, Lr: 0.000300\n", "2020-02-20 14:19:54,940 Epoch 15 Step: 22000 Batch Loss: 1.831302 Tokens per Sec: 7512, Lr: 0.000300\n", "2020-02-20 14:21:23,047 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:21:23,048 Saving new checkpoint.\n", "2020-02-20 14:21:23,266 Example #0\n", "2020-02-20 14:21:23,267 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:21:23,267 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:21:23,267 \tHypothesis: Luwo chik ma Jehova dwaro ni mondo giluw , onego onyis ni loch mar Nyasaye ok nyal timo .\n", "2020-02-20 14:21:23,267 Example #1\n", "2020-02-20 14:21:23,267 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:21:23,267 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:21:23,267 \tHypothesis: Bende , mano miyo amiye luor kaka wuonwa kendo e wi Nyasaye . ”\n", "2020-02-20 14:21:23,268 Example #2\n", "2020-02-20 14:21:23,268 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:21:23,268 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:21:23,268 \tHypothesis: Ng’eyo ni Muma wacho adier nyalo bedo gima duong ’ ahinya e wang ’ chuny .\n", "2020-02-20 14:21:23,268 Example #3\n", "2020-02-20 14:21:23,268 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:21:23,268 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:21:23,268 \tHypothesis: Ne ondiko e wi “ weche duto mag Nyasaye ma Nyasaye ne wuoyo kuome e dhoge machon . ”\n", "2020-02-20 14:21:23,268 Validation result at epoch 15, step 22000: bleu: 19.07, loss: 46018.4766, ppl: 6.4690, duration: 88.3280s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 14:21:52,230 Epoch 15 Step: 22100 Batch Loss: 1.829914 Tokens per Sec: 7648, Lr: 0.000300\n", "2020-02-20 14:22:21,127 Epoch 15 Step: 22200 Batch Loss: 2.466875 Tokens per Sec: 7643, Lr: 0.000300\n", "2020-02-20 14:22:50,008 Epoch 15 Step: 22300 Batch Loss: 1.900896 Tokens per Sec: 7610, Lr: 0.000300\n", "2020-02-20 14:23:18,907 Epoch 15 Step: 22400 Batch Loss: 1.941167 Tokens per Sec: 7638, Lr: 0.000300\n", "2020-02-20 14:23:47,500 Epoch 15 Step: 22500 Batch Loss: 2.090586 Tokens per Sec: 7550, Lr: 0.000300\n", "2020-02-20 14:24:16,126 Epoch 15 Step: 22600 Batch Loss: 2.041277 Tokens per Sec: 7500, Lr: 0.000300\n", "2020-02-20 14:24:44,754 Epoch 15 Step: 22700 Batch Loss: 2.085501 Tokens per Sec: 7495, Lr: 0.000300\n", "2020-02-20 14:25:13,195 Epoch 15 Step: 22800 Batch Loss: 2.269855 Tokens per Sec: 7553, Lr: 0.000300\n", "2020-02-20 14:25:13,805 Epoch 15: total training loss 2935.06\n", "2020-02-20 14:25:13,806 EPOCH 16\n", "2020-02-20 14:25:42,081 Epoch 16 Step: 22900 Batch Loss: 1.973166 Tokens per Sec: 7561, Lr: 0.000300\n", "2020-02-20 14:26:10,852 Epoch 16 Step: 23000 Batch Loss: 2.282125 Tokens per Sec: 7458, Lr: 0.000300\n", "2020-02-20 14:27:39,022 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:27:39,022 Saving new checkpoint.\n", "2020-02-20 14:27:39,236 Example #0\n", "2020-02-20 14:27:39,236 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:27:39,237 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:27:39,237 \tHypothesis: Luwo chikene , onego onyis ni chike Nyasaye ema owinjore .\n", "2020-02-20 14:27:39,237 Example #1\n", "2020-02-20 14:27:39,237 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:27:39,237 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:27:39,237 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuonwa kod winjruok maber gi Nyasaye . ”\n", "2020-02-20 14:27:39,237 Example #2\n", "2020-02-20 14:27:39,237 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:27:39,237 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:27:39,238 \tHypothesis: Bedo gi chuny mar ng’eyo kaka Muma wacho nyalo bedo gima duong ’ ahinya e fwenyo kaka chuny ng’ato nyalo bedo motegno .\n", "2020-02-20 14:27:39,238 Example #3\n", "2020-02-20 14:27:39,238 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:27:39,238 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:27:39,238 \tHypothesis: Noyudo osendiko e wi “ weche duto mag Nyasaye ma ne wuoyo kuom jonabi maler , ” ma ne en “ koth ndalo machon . ”\n", "2020-02-20 14:27:39,238 Validation result at epoch 16, step 23000: bleu: 19.68, loss: 45625.0352, ppl: 6.3666, duration: 88.3852s\n", "2020-02-20 14:28:07,895 Epoch 16 Step: 23100 Batch Loss: 1.623996 Tokens per Sec: 7526, Lr: 0.000300\n", "2020-02-20 14:28:36,968 Epoch 16 Step: 23200 Batch Loss: 2.171220 Tokens per Sec: 7538, Lr: 0.000300\n", "2020-02-20 14:29:06,002 Epoch 16 Step: 23300 Batch Loss: 1.812124 Tokens per Sec: 7634, Lr: 0.000300\n", "2020-02-20 14:29:34,646 Epoch 16 Step: 23400 Batch Loss: 1.823475 Tokens per Sec: 7604, Lr: 0.000300\n", "2020-02-20 14:30:03,539 Epoch 16 Step: 23500 Batch Loss: 2.066353 Tokens per Sec: 7595, Lr: 0.000300\n", "2020-02-20 14:30:32,291 Epoch 16 Step: 23600 Batch Loss: 1.938859 Tokens per Sec: 7612, Lr: 0.000300\n", "2020-02-20 14:31:00,714 Epoch 16 Step: 23700 Batch Loss: 1.713047 Tokens per Sec: 7437, Lr: 0.000300\n", "2020-02-20 14:31:29,704 Epoch 16 Step: 23800 Batch Loss: 1.987713 Tokens per Sec: 7574, Lr: 0.000300\n", "2020-02-20 14:31:58,520 Epoch 16 Step: 23900 Batch Loss: 1.617335 Tokens per Sec: 7662, Lr: 0.000300\n", "2020-02-20 14:32:27,342 Epoch 16 Step: 24000 Batch Loss: 2.072048 Tokens per Sec: 7589, Lr: 0.000300\n", "2020-02-20 14:33:55,514 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:33:55,514 Saving new checkpoint.\n", "2020-02-20 14:33:55,735 Example #0\n", "2020-02-20 14:33:55,735 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:33:55,735 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:33:55,735 \tHypothesis: Luwo chikene ne Jehova dwaro ni onyis ni chike Nyasaye owinjore .\n", "2020-02-20 14:33:55,736 Example #1\n", "2020-02-20 14:33:55,736 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:33:55,736 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:33:55,736 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuonwa kod winjruok maber gi Nyasaye . ”\n", "2020-02-20 14:33:55,736 Example #2\n", "2020-02-20 14:33:55,736 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:33:55,736 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:33:55,736 \tHypothesis: Ng’eyo ni Muma wacho adier nyalo bedo gi teko mar ng’eyo gik manie chuny .\n", "2020-02-20 14:33:55,736 Example #3\n", "2020-02-20 14:33:55,737 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:33:55,737 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:33:55,737 \tHypothesis: Nondike e wi “ weche duto mag Nyasaye ma ne wuoyo kuom jonabi maler , ” ma ne wuoyo kuom jonabi maler .\n", "2020-02-20 14:33:55,737 Validation result at epoch 16, step 24000: bleu: 20.11, loss: 45200.4805, ppl: 6.2579, duration: 88.3943s\n", "2020-02-20 14:34:24,395 Epoch 16 Step: 24100 Batch Loss: 2.104906 Tokens per Sec: 7481, Lr: 0.000300\n", "2020-02-20 14:34:53,164 Epoch 16 Step: 24200 Batch Loss: 1.972242 Tokens per Sec: 7663, Lr: 0.000300\n", "2020-02-20 14:35:21,945 Epoch 16 Step: 24300 Batch Loss: 2.164508 Tokens per Sec: 7640, Lr: 0.000300\n", "2020-02-20 14:35:28,695 Epoch 16: total training loss 2886.69\n", "2020-02-20 14:35:28,695 EPOCH 17\n", "2020-02-20 14:35:51,053 Epoch 17 Step: 24400 Batch Loss: 1.791167 Tokens per Sec: 7492, Lr: 0.000300\n", "2020-02-20 14:36:19,860 Epoch 17 Step: 24500 Batch Loss: 1.838502 Tokens per Sec: 7565, Lr: 0.000300\n", "2020-02-20 14:36:48,940 Epoch 17 Step: 24600 Batch Loss: 1.565369 Tokens per Sec: 7661, Lr: 0.000300\n", "2020-02-20 14:37:17,795 Epoch 17 Step: 24700 Batch Loss: 1.729771 Tokens per Sec: 7598, Lr: 0.000300\n", "2020-02-20 14:37:46,643 Epoch 17 Step: 24800 Batch Loss: 2.019106 Tokens per Sec: 7539, Lr: 0.000300\n", "2020-02-20 14:38:15,421 Epoch 17 Step: 24900 Batch Loss: 2.044064 Tokens per Sec: 7574, Lr: 0.000300\n", "2020-02-20 14:38:44,177 Epoch 17 Step: 25000 Batch Loss: 2.007406 Tokens per Sec: 7568, Lr: 0.000300\n", "2020-02-20 14:40:12,390 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:40:12,390 Saving new checkpoint.\n", "2020-02-20 14:40:12,612 Example #0\n", "2020-02-20 14:40:12,612 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:40:12,612 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:40:12,612 \tHypothesis: Luwo chikene ne gik ma Jehova dwaro ni mondo onyis ni chike Nyasaye ema owinjore .\n", "2020-02-20 14:40:12,613 Example #1\n", "2020-02-20 14:40:12,613 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:40:12,613 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:40:12,613 \tHypothesis: Bende , amiyo luor kaka wuonwa kod winjruok manie kindwa kode . ”\n", "2020-02-20 14:40:12,613 Example #2\n", "2020-02-20 14:40:12,613 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:40:12,613 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:40:12,613 \tHypothesis: Ng’eyo ni Muma wacho adier nyalo bedo gima duong ’ ahinya e fwenyo gik manie chunywa .\n", "2020-02-20 14:40:12,613 Example #3\n", "2020-02-20 14:40:12,614 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:40:12,614 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:40:12,614 \tHypothesis: Noyudo osendiko weche ma Nyasaye nowacho e wi “ weche duto ma Nyasaye nowuoyo kuom janabi maler , ma ne en janabi maler . ”\n", "2020-02-20 14:40:12,614 Validation result at epoch 17, step 25000: bleu: 19.91, loss: 44906.8984, ppl: 6.1838, duration: 88.4365s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 14:40:41,844 Epoch 17 Step: 25100 Batch Loss: 1.930991 Tokens per Sec: 7626, Lr: 0.000300\n", "2020-02-20 14:41:10,728 Epoch 17 Step: 25200 Batch Loss: 1.790509 Tokens per Sec: 7552, Lr: 0.000300\n", "2020-02-20 14:41:39,542 Epoch 17 Step: 25300 Batch Loss: 1.659317 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 14:42:08,450 Epoch 17 Step: 25400 Batch Loss: 1.727902 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 14:42:37,179 Epoch 17 Step: 25500 Batch Loss: 2.255066 Tokens per Sec: 7499, Lr: 0.000300\n", "2020-02-20 14:43:06,080 Epoch 17 Step: 25600 Batch Loss: 1.129977 Tokens per Sec: 7543, Lr: 0.000300\n", "2020-02-20 14:43:35,073 Epoch 17 Step: 25700 Batch Loss: 1.938497 Tokens per Sec: 7640, Lr: 0.000300\n", "2020-02-20 14:44:04,141 Epoch 17 Step: 25800 Batch Loss: 1.842105 Tokens per Sec: 7640, Lr: 0.000300\n", "2020-02-20 14:44:15,739 Epoch 17: total training loss 2840.30\n", "2020-02-20 14:44:15,740 EPOCH 18\n", "2020-02-20 14:44:32,707 Epoch 18 Step: 25900 Batch Loss: 1.698621 Tokens per Sec: 7253, Lr: 0.000300\n", "2020-02-20 14:45:01,407 Epoch 18 Step: 26000 Batch Loss: 1.913930 Tokens per Sec: 7689, Lr: 0.000300\n", "2020-02-20 14:46:29,459 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:46:29,459 Saving new checkpoint.\n", "2020-02-20 14:46:29,674 Example #0\n", "2020-02-20 14:46:29,675 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:46:29,675 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:46:29,675 \tHypothesis: Chiwruok ma Jehova dwaro ni mondo okonygi ng’eyo ni chike Nyasaye ema owinjore .\n", "2020-02-20 14:46:29,675 Example #1\n", "2020-02-20 14:46:29,675 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:46:29,675 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:46:29,675 \tHypothesis: Bende , omiya thuolo mar miyo luor kaka wuonwa kod winjruok maber gi Nyasaye . ”\n", "2020-02-20 14:46:29,676 Example #2\n", "2020-02-20 14:46:29,676 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:46:29,676 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:46:29,676 \tHypothesis: Ng’eyo ni Muma wacho adier nyalo konyowa ng’eyo kaka chuny ng’ato nyalo bedo gi teko .\n", "2020-02-20 14:46:29,676 Example #3\n", "2020-02-20 14:46:29,676 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:46:29,676 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:46:29,676 \tHypothesis: Nondike e wi “ weche duto mag Nyasaye ma Nyasaye nowuoyo kuome e dhoge machon . ”\n", "2020-02-20 14:46:29,676 Validation result at epoch 18, step 26000: bleu: 19.35, loss: 44817.0664, ppl: 6.1613, duration: 88.2689s\n", "2020-02-20 14:46:58,596 Epoch 18 Step: 26100 Batch Loss: 2.105821 Tokens per Sec: 7544, Lr: 0.000300\n", "2020-02-20 14:47:27,182 Epoch 18 Step: 26200 Batch Loss: 2.101782 Tokens per Sec: 7616, Lr: 0.000300\n", "2020-02-20 14:47:55,984 Epoch 18 Step: 26300 Batch Loss: 1.876869 Tokens per Sec: 7510, Lr: 0.000300\n", "2020-02-20 14:48:24,363 Epoch 18 Step: 26400 Batch Loss: 1.818757 Tokens per Sec: 7522, Lr: 0.000300\n", "2020-02-20 14:48:53,156 Epoch 18 Step: 26500 Batch Loss: 2.156519 Tokens per Sec: 7487, Lr: 0.000300\n", "2020-02-20 14:49:21,762 Epoch 18 Step: 26600 Batch Loss: 1.930380 Tokens per Sec: 7538, Lr: 0.000300\n", "2020-02-20 14:49:50,597 Epoch 18 Step: 26700 Batch Loss: 1.636875 Tokens per Sec: 7634, Lr: 0.000300\n", "2020-02-20 14:50:19,454 Epoch 18 Step: 26800 Batch Loss: 2.019928 Tokens per Sec: 7613, Lr: 0.000300\n", "2020-02-20 14:50:48,100 Epoch 18 Step: 26900 Batch Loss: 2.064134 Tokens per Sec: 7606, Lr: 0.000300\n", "2020-02-20 14:51:17,265 Epoch 18 Step: 27000 Batch Loss: 2.183713 Tokens per Sec: 7635, Lr: 0.000300\n", "2020-02-20 14:52:45,420 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:52:45,420 Saving new checkpoint.\n", "2020-02-20 14:52:45,638 Example #0\n", "2020-02-20 14:52:45,638 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:52:45,639 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:52:45,639 \tHypothesis: Luwo chikene , dwaro ni Jehova ochik gik ma odwaro , en ni chike mag Nyasaye ema owinjore .\n", "2020-02-20 14:52:45,639 Example #1\n", "2020-02-20 14:52:45,639 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:52:45,639 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:52:45,639 \tHypothesis: Bende , en gima miyo amiye luor kaka wuonwa koda wiye . ”\n", "2020-02-20 14:52:45,639 Example #2\n", "2020-02-20 14:52:45,639 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:52:45,639 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:52:45,639 \tHypothesis: Ng’eyo ni wan wawegi e ler ma Ndiko wacho nyalo bedo gi teko mar fwenyo chuny mar chuny .\n", "2020-02-20 14:52:45,640 Example #3\n", "2020-02-20 14:52:45,640 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:52:45,640 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:52:45,640 \tHypothesis: Ne ondiko e wi “ gik moko duto ma Nyasaye nowuoyo kuome kokalo kuom jonabi maler , ma ne en janabi maler . ”\n", "2020-02-20 14:52:45,640 Validation result at epoch 18, step 27000: bleu: 20.44, loss: 44218.1758, ppl: 6.0134, duration: 88.3739s\n", "2020-02-20 14:53:14,199 Epoch 18 Step: 27100 Batch Loss: 1.837035 Tokens per Sec: 7515, Lr: 0.000300\n", "2020-02-20 14:53:43,290 Epoch 18 Step: 27200 Batch Loss: 1.722504 Tokens per Sec: 7614, Lr: 0.000300\n", "2020-02-20 14:54:12,258 Epoch 18 Step: 27300 Batch Loss: 1.830405 Tokens per Sec: 7600, Lr: 0.000300\n", "2020-02-20 14:54:30,877 Epoch 18: total training loss 2821.93\n", "2020-02-20 14:54:30,877 EPOCH 19\n", "2020-02-20 14:54:41,573 Epoch 19 Step: 27400 Batch Loss: 1.818916 Tokens per Sec: 7436, Lr: 0.000300\n", "2020-02-20 14:55:10,496 Epoch 19 Step: 27500 Batch Loss: 1.644370 Tokens per Sec: 7690, Lr: 0.000300\n", "2020-02-20 14:55:39,376 Epoch 19 Step: 27600 Batch Loss: 1.980896 Tokens per Sec: 7614, Lr: 0.000300\n", "2020-02-20 14:56:07,923 Epoch 19 Step: 27700 Batch Loss: 1.704031 Tokens per Sec: 7469, Lr: 0.000300\n", "2020-02-20 14:56:36,653 Epoch 19 Step: 27800 Batch Loss: 1.831275 Tokens per Sec: 7393, Lr: 0.000300\n", "2020-02-20 14:57:05,844 Epoch 19 Step: 27900 Batch Loss: 2.040896 Tokens per Sec: 7743, Lr: 0.000300\n", "2020-02-20 14:57:34,599 Epoch 19 Step: 28000 Batch Loss: 1.963816 Tokens per Sec: 7609, Lr: 0.000300\n", "2020-02-20 14:59:02,671 Hooray! New best validation result [ppl]!\n", "2020-02-20 14:59:02,671 Saving new checkpoint.\n", "2020-02-20 14:59:02,895 Example #0\n", "2020-02-20 14:59:02,896 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 14:59:02,896 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 14:59:02,896 \tHypothesis: Luwo chike Jehova e yo makare , onego onyis ni chike Nyasaye nigi paro mowinjore .\n", "2020-02-20 14:59:02,896 Example #1\n", "2020-02-20 14:59:02,896 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 14:59:02,896 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 14:59:02,896 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuoro kod wiye . ”\n", "2020-02-20 14:59:02,896 Example #2\n", "2020-02-20 14:59:02,897 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 14:59:02,897 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 14:59:02,897 \tHypothesis: Nono adierago ei Ndiko ma wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 14:59:02,897 Example #3\n", "2020-02-20 14:59:02,897 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 14:59:02,897 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 14:59:02,897 \tHypothesis: Nondike e wi “ gik moko duto ma Nyasaye nowuoyo kuome e dhoge machon . ”\n", "2020-02-20 14:59:02,897 Validation result at epoch 19, step 28000: bleu: 20.51, loss: 44157.7266, ppl: 5.9986, duration: 88.2978s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 14:59:31,948 Epoch 19 Step: 28100 Batch Loss: 1.910940 Tokens per Sec: 7711, Lr: 0.000300\n", "2020-02-20 15:00:00,620 Epoch 19 Step: 28200 Batch Loss: 1.857561 Tokens per Sec: 7624, Lr: 0.000300\n", "2020-02-20 15:00:29,051 Epoch 19 Step: 28300 Batch Loss: 1.970976 Tokens per Sec: 7407, Lr: 0.000300\n", "2020-02-20 15:00:58,128 Epoch 19 Step: 28400 Batch Loss: 1.691468 Tokens per Sec: 7570, Lr: 0.000300\n", "2020-02-20 15:01:26,796 Epoch 19 Step: 28500 Batch Loss: 1.886670 Tokens per Sec: 7515, Lr: 0.000300\n", "2020-02-20 15:01:55,426 Epoch 19 Step: 28600 Batch Loss: 2.225640 Tokens per Sec: 7532, Lr: 0.000300\n", "2020-02-20 15:02:24,377 Epoch 19 Step: 28700 Batch Loss: 1.406204 Tokens per Sec: 7593, Lr: 0.000300\n", "2020-02-20 15:02:53,014 Epoch 19 Step: 28800 Batch Loss: 1.695791 Tokens per Sec: 7448, Lr: 0.000300\n", "2020-02-20 15:03:18,084 Epoch 19: total training loss 2790.81\n", "2020-02-20 15:03:18,085 EPOCH 20\n", "2020-02-20 15:03:22,124 Epoch 20 Step: 28900 Batch Loss: 1.790626 Tokens per Sec: 7397, Lr: 0.000300\n", "2020-02-20 15:03:50,648 Epoch 20 Step: 29000 Batch Loss: 1.866627 Tokens per Sec: 7535, Lr: 0.000300\n", "2020-02-20 15:05:18,692 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:05:18,692 Saving new checkpoint.\n", "2020-02-20 15:05:18,911 Example #0\n", "2020-02-20 15:05:18,911 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:05:18,911 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:05:18,911 \tHypothesis: Luwo chik ma Jehova dwaro ni mondo onyis ni chike mag Nyasaye owinjore .\n", "2020-02-20 15:05:18,911 Example #1\n", "2020-02-20 15:05:18,912 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:05:18,912 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:05:18,912 \tHypothesis: Bende , omiya thuolo mar miyo amiye luor kaka wuonwa koda wiye . ”\n", "2020-02-20 15:05:18,912 Example #2\n", "2020-02-20 15:05:18,912 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:05:18,912 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:05:18,912 \tHypothesis: Ng’eyo gik ma Ndiko wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 15:05:18,912 Example #3\n", "2020-02-20 15:05:18,912 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:05:18,913 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:05:18,913 \tHypothesis: Ne ondiko e wi “ weche duto mag Nyasaye ma ne wuoyo kuome e dho janabi maler , ma ne en janabi maler . ”\n", "2020-02-20 15:05:18,913 Validation result at epoch 20, step 29000: bleu: 20.97, loss: 43801.0352, ppl: 5.9125, duration: 88.2640s\n", "2020-02-20 15:05:47,655 Epoch 20 Step: 29100 Batch Loss: 1.752785 Tokens per Sec: 7613, Lr: 0.000300\n", "2020-02-20 15:06:16,751 Epoch 20 Step: 29200 Batch Loss: 1.508315 Tokens per Sec: 7679, Lr: 0.000300\n", "2020-02-20 15:06:45,926 Epoch 20 Step: 29300 Batch Loss: 1.816944 Tokens per Sec: 7820, Lr: 0.000300\n", "2020-02-20 15:07:14,827 Epoch 20 Step: 29400 Batch Loss: 1.822889 Tokens per Sec: 7560, Lr: 0.000300\n", "2020-02-20 15:07:43,516 Epoch 20 Step: 29500 Batch Loss: 2.019667 Tokens per Sec: 7581, Lr: 0.000300\n", "2020-02-20 15:08:12,460 Epoch 20 Step: 29600 Batch Loss: 1.640045 Tokens per Sec: 7638, Lr: 0.000300\n", "2020-02-20 15:08:40,786 Epoch 20 Step: 29700 Batch Loss: 2.031919 Tokens per Sec: 7375, Lr: 0.000300\n", "2020-02-20 15:09:09,955 Epoch 20 Step: 29800 Batch Loss: 1.888292 Tokens per Sec: 7733, Lr: 0.000300\n", "2020-02-20 15:09:38,439 Epoch 20 Step: 29900 Batch Loss: 1.080145 Tokens per Sec: 7422, Lr: 0.000300\n", "2020-02-20 15:10:07,055 Epoch 20 Step: 30000 Batch Loss: 2.215950 Tokens per Sec: 7469, Lr: 0.000300\n", "2020-02-20 15:11:35,124 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:11:35,124 Saving new checkpoint.\n", "2020-02-20 15:11:35,345 Example #0\n", "2020-02-20 15:11:35,345 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:11:35,345 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:11:35,345 \tHypothesis: Luwo chike Jehova e yo ma dwaro ni wanyis ni chike Nyasaye ok owinjore .\n", "2020-02-20 15:11:35,346 Example #1\n", "2020-02-20 15:11:35,346 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:11:35,346 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:11:35,346 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuonwa koda wiye . ”\n", "2020-02-20 15:11:35,346 Example #2\n", "2020-02-20 15:11:35,347 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:11:35,347 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:11:35,347 \tHypothesis: Ka wafwenyo ni wan wawegi warito ler mar Ndiko , mano nyalo miyo wafweny matut gik manie chunywa .\n", "2020-02-20 15:11:35,347 Example #3\n", "2020-02-20 15:11:35,347 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:11:35,347 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:11:35,347 \tHypothesis: Noyudo osendiko e wi “ weche duto mag Nyasaye ma Nyasaye nowuoyo kuome kokalo kuom jonabi maler . ”\n", "2020-02-20 15:11:35,348 Validation result at epoch 20, step 30000: bleu: 21.01, loss: 43687.1055, ppl: 5.8852, duration: 88.2914s\n", "2020-02-20 15:12:03,930 Epoch 20 Step: 30100 Batch Loss: 1.640167 Tokens per Sec: 7534, Lr: 0.000300\n", "2020-02-20 15:12:32,293 Epoch 20 Step: 30200 Batch Loss: 1.806264 Tokens per Sec: 7464, Lr: 0.000300\n", "2020-02-20 15:13:01,063 Epoch 20 Step: 30300 Batch Loss: 1.790595 Tokens per Sec: 7671, Lr: 0.000300\n", "2020-02-20 15:13:29,516 Epoch 20 Step: 30400 Batch Loss: 1.406210 Tokens per Sec: 7558, Lr: 0.000300\n", "2020-02-20 15:13:32,776 Epoch 20: total training loss 2756.82\n", "2020-02-20 15:13:32,776 EPOCH 21\n", "2020-02-20 15:13:58,627 Epoch 21 Step: 30500 Batch Loss: 1.913261 Tokens per Sec: 7420, Lr: 0.000300\n", "2020-02-20 15:14:27,334 Epoch 21 Step: 30600 Batch Loss: 1.737304 Tokens per Sec: 7541, Lr: 0.000300\n", "2020-02-20 15:14:56,442 Epoch 21 Step: 30700 Batch Loss: 1.684234 Tokens per Sec: 7634, Lr: 0.000300\n", "2020-02-20 15:15:25,375 Epoch 21 Step: 30800 Batch Loss: 2.049634 Tokens per Sec: 7664, Lr: 0.000300\n", "2020-02-20 15:15:54,106 Epoch 21 Step: 30900 Batch Loss: 1.679987 Tokens per Sec: 7571, Lr: 0.000300\n", "2020-02-20 15:16:22,846 Epoch 21 Step: 31000 Batch Loss: 1.611002 Tokens per Sec: 7605, Lr: 0.000300\n", "2020-02-20 15:17:50,653 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:17:50,654 Saving new checkpoint.\n", "2020-02-20 15:17:50,869 Example #0\n", "2020-02-20 15:17:50,870 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:17:50,870 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:17:50,870 \tHypothesis: Luwo chike Jehova e yo ma Jehova dwaro ni mondo okonywa bedo gi paro mowinjore .\n", "2020-02-20 15:17:50,870 Example #1\n", "2020-02-20 15:17:50,870 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:17:50,870 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:17:50,870 \tHypothesis: Bende , omiya rieko mar miye luor kaka wuonwa gi wiye . ”\n", "2020-02-20 15:17:50,871 Example #2\n", "2020-02-20 15:17:50,871 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:17:50,871 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:17:50,871 \tHypothesis: Ka wafwenyo ler mar weche manie Ndiko , wanyalo fwenyo ni chunywa nigi teko ahinya .\n", "2020-02-20 15:17:50,871 Example #3\n", "2020-02-20 15:17:50,871 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:17:50,871 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:17:50,871 \tHypothesis: Noyudo osendiko e wi “ weche duto mag Nyasaye ma Nyasaye nowuoyo kuome e dho jonabi maler kuom kinde machon . ”\n", "2020-02-20 15:17:50,871 Validation result at epoch 21, step 31000: bleu: 20.83, loss: 43309.3750, ppl: 5.7957, duration: 88.0246s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 15:18:19,622 Epoch 21 Step: 31100 Batch Loss: 1.819182 Tokens per Sec: 7524, Lr: 0.000300\n", "2020-02-20 15:18:48,484 Epoch 21 Step: 31200 Batch Loss: 1.671183 Tokens per Sec: 7697, Lr: 0.000300\n", "2020-02-20 15:19:17,193 Epoch 21 Step: 31300 Batch Loss: 1.073250 Tokens per Sec: 7488, Lr: 0.000300\n", "2020-02-20 15:19:45,994 Epoch 21 Step: 31400 Batch Loss: 1.645528 Tokens per Sec: 7604, Lr: 0.000300\n", "2020-02-20 15:20:14,749 Epoch 21 Step: 31500 Batch Loss: 1.817031 Tokens per Sec: 7573, Lr: 0.000300\n", "2020-02-20 15:20:43,542 Epoch 21 Step: 31600 Batch Loss: 1.894629 Tokens per Sec: 7575, Lr: 0.000300\n", "2020-02-20 15:21:12,418 Epoch 21 Step: 31700 Batch Loss: 1.885237 Tokens per Sec: 7576, Lr: 0.000300\n", "2020-02-20 15:21:41,376 Epoch 21 Step: 31800 Batch Loss: 1.353623 Tokens per Sec: 7636, Lr: 0.000300\n", "2020-02-20 15:22:10,286 Epoch 21 Step: 31900 Batch Loss: 1.591935 Tokens per Sec: 7582, Lr: 0.000300\n", "2020-02-20 15:22:18,431 Epoch 21: total training loss 2717.05\n", "2020-02-20 15:22:18,431 EPOCH 22\n", "2020-02-20 15:22:39,288 Epoch 22 Step: 32000 Batch Loss: 1.760571 Tokens per Sec: 7427, Lr: 0.000300\n", "2020-02-20 15:24:07,383 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:24:07,384 Saving new checkpoint.\n", "2020-02-20 15:24:07,606 Example #0\n", "2020-02-20 15:24:07,606 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:24:07,606 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:24:07,606 \tHypothesis: Luwo chikene makare , onego onyis ni chike Nyasaye ok nyal winjo tiende .\n", "2020-02-20 15:24:07,606 Example #1\n", "2020-02-20 15:24:07,607 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:24:07,607 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:24:07,607 \tHypothesis: Bende , omiya thuolo momedore mar miyo Nyasaye luor kendo tayo winjruokna kode . ”\n", "2020-02-20 15:24:07,607 Example #2\n", "2020-02-20 15:24:07,607 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:24:07,607 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:24:07,607 \tHypothesis: Ng’eyo ni Muma wacho adier nyalo bedo gi teko mar ng’eyo chuny ng’ato .\n", "2020-02-20 15:24:07,607 Example #3\n", "2020-02-20 15:24:07,607 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:24:07,607 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:24:07,608 \tHypothesis: Noyudo osendiko e wi “ gik moko duto ma Nyasaye nowuoyo kuome kokalo kuom jonabi maler . ”\n", "2020-02-20 15:24:07,608 Validation result at epoch 22, step 32000: bleu: 21.35, loss: 43100.0508, ppl: 5.7467, duration: 88.3187s\n", "2020-02-20 15:24:36,222 Epoch 22 Step: 32100 Batch Loss: 1.414382 Tokens per Sec: 7497, Lr: 0.000300\n", "2020-02-20 15:25:04,712 Epoch 22 Step: 32200 Batch Loss: 1.545235 Tokens per Sec: 7444, Lr: 0.000300\n", "2020-02-20 15:25:33,666 Epoch 22 Step: 32300 Batch Loss: 2.022698 Tokens per Sec: 7642, Lr: 0.000300\n", "2020-02-20 15:26:02,242 Epoch 22 Step: 32400 Batch Loss: 2.208554 Tokens per Sec: 7568, Lr: 0.000300\n", "2020-02-20 15:26:31,015 Epoch 22 Step: 32500 Batch Loss: 1.741829 Tokens per Sec: 7576, Lr: 0.000300\n", "2020-02-20 15:26:59,857 Epoch 22 Step: 32600 Batch Loss: 1.308279 Tokens per Sec: 7590, Lr: 0.000300\n", "2020-02-20 15:27:28,962 Epoch 22 Step: 32700 Batch Loss: 1.890848 Tokens per Sec: 7652, Lr: 0.000300\n", "2020-02-20 15:27:57,781 Epoch 22 Step: 32800 Batch Loss: 1.702807 Tokens per Sec: 7751, Lr: 0.000300\n", "2020-02-20 15:28:26,449 Epoch 22 Step: 32900 Batch Loss: 2.175448 Tokens per Sec: 7541, Lr: 0.000300\n", "2020-02-20 15:28:55,299 Epoch 22 Step: 33000 Batch Loss: 1.553985 Tokens per Sec: 7687, Lr: 0.000300\n", "2020-02-20 15:30:23,371 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:30:23,371 Saving new checkpoint.\n", "2020-02-20 15:30:23,589 Example #0\n", "2020-02-20 15:30:23,589 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:30:23,589 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:30:23,589 \tHypothesis: Luwo chike Jehova en gima dwaro ni mondo giluw chikene .\n", "2020-02-20 15:30:23,590 Example #1\n", "2020-02-20 15:30:23,590 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:30:23,590 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:30:23,590 \tHypothesis: Bende , omiya thuolo momedore mar miye luor kaka wuonwa gi wiye . ”\n", "2020-02-20 15:30:23,590 Example #2\n", "2020-02-20 15:30:23,590 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:30:23,590 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:30:23,590 \tHypothesis: Nono gik ma Ndiko wacho nyalo konyo ahinya e fwenyo gik ma chunywa dwaro .\n", "2020-02-20 15:30:23,590 Example #3\n", "2020-02-20 15:30:23,591 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:30:23,591 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:30:23,591 \tHypothesis: Ne ondiko e wi “ gik moko duto ma Nyasaye nowuoyo kuome kokalo kuom dho jonabi maler , ma ne en jonabi machon . ”\n", "2020-02-20 15:30:23,591 Validation result at epoch 22, step 33000: bleu: 20.99, loss: 42939.6875, ppl: 5.7094, duration: 88.2914s\n", "2020-02-20 15:30:52,353 Epoch 22 Step: 33100 Batch Loss: 2.253767 Tokens per Sec: 7501, Lr: 0.000300\n", "2020-02-20 15:31:21,146 Epoch 22 Step: 33200 Batch Loss: 2.039063 Tokens per Sec: 7529, Lr: 0.000300\n", "2020-02-20 15:31:49,986 Epoch 22 Step: 33300 Batch Loss: 1.518212 Tokens per Sec: 7588, Lr: 0.000300\n", "2020-02-20 15:32:18,995 Epoch 22 Step: 33400 Batch Loss: 1.767002 Tokens per Sec: 7592, Lr: 0.000300\n", "2020-02-20 15:32:33,246 Epoch 22: total training loss 2699.78\n", "2020-02-20 15:32:33,246 EPOCH 23\n", "2020-02-20 15:32:47,958 Epoch 23 Step: 33500 Batch Loss: 1.491392 Tokens per Sec: 7456, Lr: 0.000300\n", "2020-02-20 15:33:16,992 Epoch 23 Step: 33600 Batch Loss: 1.746894 Tokens per Sec: 7591, Lr: 0.000300\n", "2020-02-20 15:33:45,691 Epoch 23 Step: 33700 Batch Loss: 1.184824 Tokens per Sec: 7602, Lr: 0.000300\n", "2020-02-20 15:34:14,468 Epoch 23 Step: 33800 Batch Loss: 1.476453 Tokens per Sec: 7481, Lr: 0.000300\n", "2020-02-20 15:34:42,872 Epoch 23 Step: 33900 Batch Loss: 1.657290 Tokens per Sec: 7496, Lr: 0.000300\n", "2020-02-20 15:35:11,671 Epoch 23 Step: 34000 Batch Loss: 1.806342 Tokens per Sec: 7676, Lr: 0.000300\n", "2020-02-20 15:36:39,685 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:36:39,685 Saving new checkpoint.\n", "2020-02-20 15:36:39,899 Example #0\n", "2020-02-20 15:36:39,899 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:36:39,899 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:36:39,900 \tHypothesis: Luwo chike Jehova dwaro ni ginyis ni chike Nyasaye owinjore .\n", "2020-02-20 15:36:39,900 Example #1\n", "2020-02-20 15:36:39,900 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:36:39,900 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:36:39,900 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuonwa koda wiye . ”\n", "2020-02-20 15:36:39,900 Example #2\n", "2020-02-20 15:36:39,900 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:36:39,900 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:36:39,900 \tHypothesis: Nono gik ma Ndiko wacho nyalo konyowa ng’eyo kaka chuny ng’ato nyalo bedo gi teko .\n", "2020-02-20 15:36:39,900 Example #3\n", "2020-02-20 15:36:39,901 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:36:39,901 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:36:39,901 \tHypothesis: Noyudo osendiko e wi “ gik moko duto ma Nyasaye nowuoyo kuom jonabi maler kuom kinde machon . ”\n", "2020-02-20 15:36:39,901 Validation result at epoch 23, step 34000: bleu: 21.52, loss: 42751.1289, ppl: 5.6659, duration: 88.2290s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 15:37:08,695 Epoch 23 Step: 34100 Batch Loss: 1.914464 Tokens per Sec: 7584, Lr: 0.000300\n", "2020-02-20 15:37:37,404 Epoch 23 Step: 34200 Batch Loss: 1.760642 Tokens per Sec: 7584, Lr: 0.000300\n", "2020-02-20 15:38:06,019 Epoch 23 Step: 34300 Batch Loss: 1.812148 Tokens per Sec: 7586, Lr: 0.000300\n", "2020-02-20 15:38:34,760 Epoch 23 Step: 34400 Batch Loss: 1.241874 Tokens per Sec: 7601, Lr: 0.000300\n", "2020-02-20 15:39:03,539 Epoch 23 Step: 34500 Batch Loss: 1.844906 Tokens per Sec: 7624, Lr: 0.000300\n", "2020-02-20 15:39:32,884 Epoch 23 Step: 34600 Batch Loss: 1.787987 Tokens per Sec: 7863, Lr: 0.000300\n", "2020-02-20 15:40:01,826 Epoch 23 Step: 34700 Batch Loss: 1.775381 Tokens per Sec: 7588, Lr: 0.000300\n", "2020-02-20 15:40:30,810 Epoch 23 Step: 34800 Batch Loss: 1.828186 Tokens per Sec: 7622, Lr: 0.000300\n", "2020-02-20 15:40:59,724 Epoch 23 Step: 34900 Batch Loss: 1.916007 Tokens per Sec: 7516, Lr: 0.000300\n", "2020-02-20 15:41:19,055 Epoch 23: total training loss 2670.62\n", "2020-02-20 15:41:19,055 EPOCH 24\n", "2020-02-20 15:41:28,576 Epoch 24 Step: 35000 Batch Loss: 1.632923 Tokens per Sec: 7289, Lr: 0.000300\n", "2020-02-20 15:42:56,702 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:42:56,702 Saving new checkpoint.\n", "2020-02-20 15:42:56,920 Example #0\n", "2020-02-20 15:42:56,920 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:42:56,920 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:42:56,920 \tHypothesis: Luwo chikene , dwaro ni wanyis ni chike Nyasaye nigi paro mowinjore .\n", "2020-02-20 15:42:56,921 Example #1\n", "2020-02-20 15:42:56,921 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:42:56,921 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:42:56,921 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuonwa koda winjruok maber gi Nyasaye . ”\n", "2020-02-20 15:42:56,921 Example #2\n", "2020-02-20 15:42:56,921 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:42:56,921 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:42:56,921 \tHypothesis: Nono gik ma Ndiko wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 15:42:56,921 Example #3\n", "2020-02-20 15:42:56,922 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:42:56,922 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:42:56,922 \tHypothesis: Noyudo osendiko e wi “ gik moko duto ma Nyasaye nowuoyo kuome e dho jonabi maler . ”\n", "2020-02-20 15:42:56,922 Validation result at epoch 24, step 35000: bleu: 21.60, loss: 42456.4180, ppl: 5.5986, duration: 88.3449s\n", "2020-02-20 15:43:25,825 Epoch 24 Step: 35100 Batch Loss: 2.230939 Tokens per Sec: 7603, Lr: 0.000300\n", "2020-02-20 15:43:54,911 Epoch 24 Step: 35200 Batch Loss: 1.771439 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 15:44:23,615 Epoch 24 Step: 35300 Batch Loss: 2.057425 Tokens per Sec: 7470, Lr: 0.000300\n", "2020-02-20 15:44:52,359 Epoch 24 Step: 35400 Batch Loss: 2.026919 Tokens per Sec: 7631, Lr: 0.000300\n", "2020-02-20 15:45:20,898 Epoch 24 Step: 35500 Batch Loss: 1.822903 Tokens per Sec: 7536, Lr: 0.000300\n", "2020-02-20 15:45:49,847 Epoch 24 Step: 35600 Batch Loss: 1.721725 Tokens per Sec: 7618, Lr: 0.000300\n", "2020-02-20 15:46:18,354 Epoch 24 Step: 35700 Batch Loss: 1.844272 Tokens per Sec: 7455, Lr: 0.000300\n", "2020-02-20 15:46:46,754 Epoch 24 Step: 35800 Batch Loss: 1.998473 Tokens per Sec: 7488, Lr: 0.000300\n", "2020-02-20 15:47:16,004 Epoch 24 Step: 35900 Batch Loss: 1.597854 Tokens per Sec: 7708, Lr: 0.000300\n", "2020-02-20 15:47:44,855 Epoch 24 Step: 36000 Batch Loss: 1.540648 Tokens per Sec: 7538, Lr: 0.000300\n", "2020-02-20 15:49:13,031 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:49:13,031 Saving new checkpoint.\n", "2020-02-20 15:49:13,250 Example #0\n", "2020-02-20 15:49:13,250 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:49:13,250 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:49:13,250 \tHypothesis: Luwo chikene , onego onyis ni chike Nyasaye ema owinjore .\n", "2020-02-20 15:49:13,250 Example #1\n", "2020-02-20 15:49:13,250 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:49:13,250 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:49:13,250 \tHypothesis: Bende , omiya thuolo mar miye luor kaka wuonwa kod wiye . ”\n", "2020-02-20 15:49:13,251 Example #2\n", "2020-02-20 15:49:13,251 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:49:13,251 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:49:13,251 \tHypothesis: Neno gik ma Ndiko wacho nyalo konyowa ng’eyo kaka chunywa chalo .\n", "2020-02-20 15:49:13,251 Example #3\n", "2020-02-20 15:49:13,251 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 15:49:13,251 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:49:13,251 \tHypothesis: Noyudo osendiko kuom “ weche duto ma Nyasaye nowuoyo kuome kokalo kuom jonabi maler ma ne en jonabi maler . ”\n", "2020-02-20 15:49:13,251 Validation result at epoch 24, step 36000: bleu: 21.19, loss: 42349.5820, ppl: 5.5743, duration: 88.3960s\n", "2020-02-20 15:49:42,256 Epoch 24 Step: 36100 Batch Loss: 1.857429 Tokens per Sec: 7654, Lr: 0.000300\n", "2020-02-20 15:50:11,323 Epoch 24 Step: 36200 Batch Loss: 1.692087 Tokens per Sec: 7579, Lr: 0.000300\n", "2020-02-20 15:50:40,178 Epoch 24 Step: 36300 Batch Loss: 1.871893 Tokens per Sec: 7658, Lr: 0.000300\n", "2020-02-20 15:51:09,207 Epoch 24 Step: 36400 Batch Loss: 2.020421 Tokens per Sec: 7760, Lr: 0.000300\n", "2020-02-20 15:51:33,421 Epoch 24: total training loss 2640.82\n", "2020-02-20 15:51:33,421 EPOCH 25\n", "2020-02-20 15:51:38,282 Epoch 25 Step: 36500 Batch Loss: 1.380916 Tokens per Sec: 7233, Lr: 0.000300\n", "2020-02-20 15:52:07,143 Epoch 25 Step: 36600 Batch Loss: 1.699091 Tokens per Sec: 7551, Lr: 0.000300\n", "2020-02-20 15:52:35,780 Epoch 25 Step: 36700 Batch Loss: 1.813512 Tokens per Sec: 7590, Lr: 0.000300\n", "2020-02-20 15:53:04,754 Epoch 25 Step: 36800 Batch Loss: 1.879889 Tokens per Sec: 7619, Lr: 0.000300\n", "2020-02-20 15:53:33,617 Epoch 25 Step: 36900 Batch Loss: 1.529651 Tokens per Sec: 7609, Lr: 0.000300\n", "2020-02-20 15:54:02,465 Epoch 25 Step: 37000 Batch Loss: 1.654283 Tokens per Sec: 7575, Lr: 0.000300\n", "2020-02-20 15:55:30,642 Hooray! New best validation result [ppl]!\n", "2020-02-20 15:55:30,642 Saving new checkpoint.\n", "2020-02-20 15:55:30,861 Example #0\n", "2020-02-20 15:55:30,862 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 15:55:30,862 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 15:55:30,862 \tHypothesis: Luwo chike Jehova e yo mowinjore , onego onyis ni chike Nyasaye ok owinjore .\n", "2020-02-20 15:55:30,862 Example #1\n", "2020-02-20 15:55:30,862 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 15:55:30,862 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 15:55:30,862 \tHypothesis: Bende , omiya luor matut ne wuonwa koda wiye . ”\n", "2020-02-20 15:55:30,862 Example #2\n", "2020-02-20 15:55:30,862 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 15:55:30,863 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 15:55:30,863 \tHypothesis: Nono gik ma Ndiko wacho nyalo konyo ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 15:55:30,863 Example #3\n", "2020-02-20 15:55:30,863 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 15:55:30,863 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 15:55:30,863 \tHypothesis: Noyudo osendiko e wi “ gik moko duto ma Nyasaye nowuoyo kuom dho jonabi machon . ”\n", "2020-02-20 15:55:30,863 Validation result at epoch 25, step 37000: bleu: 21.67, loss: 42305.6289, ppl: 5.5644, duration: 88.3971s\n", "2020-02-20 15:55:59,189 Epoch 25 Step: 37100 Batch Loss: 1.815181 Tokens per Sec: 7414, Lr: 0.000300\n", "2020-02-20 15:56:28,284 Epoch 25 Step: 37200 Batch Loss: 1.838013 Tokens per Sec: 7600, Lr: 0.000300\n", "2020-02-20 15:56:56,996 Epoch 25 Step: 37300 Batch Loss: 1.826828 Tokens per Sec: 7536, Lr: 0.000300\n", "2020-02-20 15:57:25,629 Epoch 25 Step: 37400 Batch Loss: 1.523698 Tokens per Sec: 7541, Lr: 0.000300\n", "2020-02-20 15:57:54,631 Epoch 25 Step: 37500 Batch Loss: 1.658947 Tokens per Sec: 7609, Lr: 0.000300\n", "2020-02-20 15:58:23,496 Epoch 25 Step: 37600 Batch Loss: 1.775190 Tokens per Sec: 7494, Lr: 0.000300\n", "2020-02-20 15:58:52,272 Epoch 25 Step: 37700 Batch Loss: 1.083689 Tokens per Sec: 7649, Lr: 0.000300\n", "2020-02-20 15:59:21,261 Epoch 25 Step: 37800 Batch Loss: 1.861390 Tokens per Sec: 7653, Lr: 0.000300\n", "2020-02-20 15:59:50,236 Epoch 25 Step: 37900 Batch Loss: 1.760268 Tokens per Sec: 7633, Lr: 0.000300\n", "2020-02-20 16:00:19,228 Epoch 25 Step: 38000 Batch Loss: 2.033724 Tokens per Sec: 7649, Lr: 0.000300\n", "2020-02-20 16:01:47,336 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:01:47,336 Saving new checkpoint.\n", "2020-02-20 16:01:47,557 Example #0\n", "2020-02-20 16:01:47,557 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:01:47,557 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:01:47,557 \tHypothesis: Luwo chike Jehova dwaro ni mondo onyis ni chike Nyasaye owinjore .\n", "2020-02-20 16:01:47,557 Example #1\n", "2020-02-20 16:01:47,557 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:01:47,557 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:01:47,558 \tHypothesis: Bende , en gima miyo amiye luor kaka wuonwa koda wiye . ”\n", "2020-02-20 16:01:47,558 Example #2\n", "2020-02-20 16:01:47,558 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:01:47,558 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:01:47,558 \tHypothesis: Nono kaka Muma wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:01:47,558 Example #3\n", "2020-02-20 16:01:47,558 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:01:47,558 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:01:47,558 \tHypothesis: Noyudo osendiko weche duto ma Nyasaye ne wuoyo kuome e dho jonabi maler kuom kinde machon . ”\n", "2020-02-20 16:01:47,558 Validation result at epoch 25, step 38000: bleu: 21.64, loss: 42005.7266, ppl: 5.4971, duration: 88.3296s\n", "2020-02-20 16:01:48,159 Epoch 25: total training loss 2618.74\n", "2020-02-20 16:01:48,159 EPOCH 26\n", "2020-02-20 16:02:16,329 Epoch 26 Step: 38100 Batch Loss: 1.761483 Tokens per Sec: 7532, Lr: 0.000300\n", "2020-02-20 16:02:45,058 Epoch 26 Step: 38200 Batch Loss: 1.693808 Tokens per Sec: 7657, Lr: 0.000300\n", "2020-02-20 16:03:13,803 Epoch 26 Step: 38300 Batch Loss: 1.515896 Tokens per Sec: 7502, Lr: 0.000300\n", "2020-02-20 16:03:42,617 Epoch 26 Step: 38400 Batch Loss: 1.760625 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 16:04:11,016 Epoch 26 Step: 38500 Batch Loss: 0.938905 Tokens per Sec: 7395, Lr: 0.000300\n", "2020-02-20 16:04:39,951 Epoch 26 Step: 38600 Batch Loss: 1.577976 Tokens per Sec: 7537, Lr: 0.000300\n", "2020-02-20 16:05:08,704 Epoch 26 Step: 38700 Batch Loss: 1.703813 Tokens per Sec: 7629, Lr: 0.000300\n", "2020-02-20 16:05:37,617 Epoch 26 Step: 38800 Batch Loss: 1.682674 Tokens per Sec: 7560, Lr: 0.000300\n", "2020-02-20 16:06:06,358 Epoch 26 Step: 38900 Batch Loss: 1.234267 Tokens per Sec: 7645, Lr: 0.000300\n", "2020-02-20 16:06:35,344 Epoch 26 Step: 39000 Batch Loss: 1.931374 Tokens per Sec: 7689, Lr: 0.000300\n", "2020-02-20 16:08:03,511 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:08:03,512 Saving new checkpoint.\n", "2020-02-20 16:08:03,727 Example #0\n", "2020-02-20 16:08:03,728 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:08:03,728 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:08:03,728 \tHypothesis: Ka giikore winjo dwond Jehova , onego onyis ni chike Nyasaye ok owinjore .\n", "2020-02-20 16:08:03,728 Example #1\n", "2020-02-20 16:08:03,728 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:08:03,728 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:08:03,728 \tHypothesis: Bende , omiya luor matut ne en kaka wuonwa kod wiye . ”\n", "2020-02-20 16:08:03,728 Example #2\n", "2020-02-20 16:08:03,729 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:08:03,729 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:08:03,729 \tHypothesis: Nono ler mar gima Ndiko wacho nyalo konyo ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:08:03,729 Example #3\n", "2020-02-20 16:08:03,729 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:08:03,729 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:08:03,729 \tHypothesis: Noyudo osendiko e wi “ weche duto ma Nyasaye nowuoyo kuom dho jonabi maler mar ndalo machon . ”\n", "2020-02-20 16:08:03,729 Validation result at epoch 26, step 39000: bleu: 21.74, loss: 41955.6680, ppl: 5.4860, duration: 88.3847s\n", "2020-02-20 16:08:32,482 Epoch 26 Step: 39100 Batch Loss: 1.833119 Tokens per Sec: 7616, Lr: 0.000300\n", "2020-02-20 16:09:01,268 Epoch 26 Step: 39200 Batch Loss: 1.840668 Tokens per Sec: 7588, Lr: 0.000300\n", "2020-02-20 16:09:29,756 Epoch 26 Step: 39300 Batch Loss: 1.956726 Tokens per Sec: 7580, Lr: 0.000300\n", "2020-02-20 16:09:58,532 Epoch 26 Step: 39400 Batch Loss: 1.796959 Tokens per Sec: 7627, Lr: 0.000300\n", "2020-02-20 16:10:27,478 Epoch 26 Step: 39500 Batch Loss: 1.474928 Tokens per Sec: 7744, Lr: 0.000300\n", "2020-02-20 16:10:33,873 Epoch 26: total training loss 2605.51\n", "2020-02-20 16:10:33,873 EPOCH 27\n", "2020-02-20 16:10:56,508 Epoch 27 Step: 39600 Batch Loss: 1.582130 Tokens per Sec: 7580, Lr: 0.000300\n", "2020-02-20 16:11:25,268 Epoch 27 Step: 39700 Batch Loss: 1.310958 Tokens per Sec: 7513, Lr: 0.000300\n", "2020-02-20 16:11:54,123 Epoch 27 Step: 39800 Batch Loss: 1.736443 Tokens per Sec: 7625, Lr: 0.000300\n", "2020-02-20 16:12:23,131 Epoch 27 Step: 39900 Batch Loss: 1.639174 Tokens per Sec: 7540, Lr: 0.000300\n", "2020-02-20 16:12:52,007 Epoch 27 Step: 40000 Batch Loss: 1.594159 Tokens per Sec: 7664, Lr: 0.000300\n", "2020-02-20 16:14:20,170 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:14:20,170 Saving new checkpoint.\n", "2020-02-20 16:14:20,391 Example #0\n", "2020-02-20 16:14:20,391 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:14:20,391 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:14:20,391 \tHypothesis: Luwo chike Jehova e yo makare , onego onyis ni chike Nyasaye owinjore .\n", "2020-02-20 16:14:20,391 Example #1\n", "2020-02-20 16:14:20,392 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:14:20,392 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:14:20,392 \tHypothesis: Bende , omiya luor matut kuom miye luor kaka wuonwa koda wiwa . ”\n", "2020-02-20 16:14:20,392 Example #2\n", "2020-02-20 16:14:20,392 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:14:20,392 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:14:20,392 \tHypothesis: Neno kaka Muma wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:14:20,392 Example #3\n", "2020-02-20 16:14:20,393 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:14:20,393 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:14:20,393 \tHypothesis: Noyudo osendiko e wi “ weche duto ma Nyasaye nowuoyo kuome e dho jonabi maler kuom kinde machon . ”\n", "2020-02-20 16:14:20,393 Validation result at epoch 27, step 40000: bleu: 21.32, loss: 41715.1719, ppl: 5.4327, duration: 88.3852s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 16:14:49,104 Epoch 27 Step: 40100 Batch Loss: 1.829319 Tokens per Sec: 7649, Lr: 0.000300\n", "2020-02-20 16:15:17,967 Epoch 27 Step: 40200 Batch Loss: 1.796776 Tokens per Sec: 7594, Lr: 0.000300\n", "2020-02-20 16:15:47,029 Epoch 27 Step: 40300 Batch Loss: 1.575827 Tokens per Sec: 7586, Lr: 0.000300\n", "2020-02-20 16:16:15,906 Epoch 27 Step: 40400 Batch Loss: 1.830190 Tokens per Sec: 7541, Lr: 0.000300\n", "2020-02-20 16:16:44,727 Epoch 27 Step: 40500 Batch Loss: 1.579970 Tokens per Sec: 7617, Lr: 0.000300\n", "2020-02-20 16:17:13,546 Epoch 27 Step: 40600 Batch Loss: 1.676199 Tokens per Sec: 7521, Lr: 0.000300\n", "2020-02-20 16:17:42,689 Epoch 27 Step: 40700 Batch Loss: 1.726659 Tokens per Sec: 7699, Lr: 0.000300\n", "2020-02-20 16:18:11,558 Epoch 27 Step: 40800 Batch Loss: 1.675419 Tokens per Sec: 7484, Lr: 0.000300\n", "2020-02-20 16:18:40,394 Epoch 27 Step: 40900 Batch Loss: 1.705130 Tokens per Sec: 7408, Lr: 0.000300\n", "2020-02-20 16:19:09,545 Epoch 27 Step: 41000 Batch Loss: 1.187504 Tokens per Sec: 7602, Lr: 0.000300\n", "2020-02-20 16:20:37,641 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:20:37,641 Saving new checkpoint.\n", "2020-02-20 16:20:37,860 Example #0\n", "2020-02-20 16:20:37,860 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:20:37,860 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:20:37,860 \tHypothesis: Luwo chik ma Jehova dwaro ni mondo ginyis ni chike Nyasaye owinjore .\n", "2020-02-20 16:20:37,860 Example #1\n", "2020-02-20 16:20:37,861 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:20:37,861 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:20:37,861 \tHypothesis: Omiya luor bende miyo amiye luor kaka wuonwa gi Nyasaye . ”\n", "2020-02-20 16:20:37,861 Example #2\n", "2020-02-20 16:20:37,861 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:20:37,861 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:20:37,861 \tHypothesis: Neno kaka Muma wacho nyalo konyo ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:20:37,861 Example #3\n", "2020-02-20 16:20:37,862 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:20:37,862 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:20:37,862 \tHypothesis: Noyudo osendiko e wi “ liet mar gik moko duto ma Nyasaye nowuoyo kuome kokalo kuom jonabi maler ma ne tiyo . ”\n", "2020-02-20 16:20:37,862 Validation result at epoch 27, step 41000: bleu: 22.04, loss: 41422.9102, ppl: 5.3687, duration: 88.3159s\n", "2020-02-20 16:20:48,781 Epoch 27: total training loss 2575.73\n", "2020-02-20 16:20:48,782 EPOCH 28\n", "2020-02-20 16:21:06,622 Epoch 28 Step: 41100 Batch Loss: 1.798172 Tokens per Sec: 7491, Lr: 0.000300\n", "2020-02-20 16:21:35,147 Epoch 28 Step: 41200 Batch Loss: 1.775519 Tokens per Sec: 7441, Lr: 0.000300\n", "2020-02-20 16:22:03,839 Epoch 28 Step: 41300 Batch Loss: 1.444540 Tokens per Sec: 7504, Lr: 0.000300\n", "2020-02-20 16:22:32,533 Epoch 28 Step: 41400 Batch Loss: 1.862024 Tokens per Sec: 7554, Lr: 0.000300\n", "2020-02-20 16:23:01,616 Epoch 28 Step: 41500 Batch Loss: 1.405001 Tokens per Sec: 7729, Lr: 0.000300\n", "2020-02-20 16:23:30,363 Epoch 28 Step: 41600 Batch Loss: 1.625368 Tokens per Sec: 7482, Lr: 0.000300\n", "2020-02-20 16:23:59,021 Epoch 28 Step: 41700 Batch Loss: 1.713350 Tokens per Sec: 7432, Lr: 0.000300\n", "2020-02-20 16:24:27,949 Epoch 28 Step: 41800 Batch Loss: 1.636419 Tokens per Sec: 7589, Lr: 0.000300\n", "2020-02-20 16:24:56,914 Epoch 28 Step: 41900 Batch Loss: 1.398697 Tokens per Sec: 7678, Lr: 0.000300\n", "2020-02-20 16:25:25,811 Epoch 28 Step: 42000 Batch Loss: 1.813509 Tokens per Sec: 7645, Lr: 0.000300\n", "2020-02-20 16:26:53,604 Example #0\n", "2020-02-20 16:26:53,604 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:26:53,604 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:26:53,605 \tHypothesis: Luwo chike Jehova dwaro ni mondo onyis ni chike Nyasaye ok owinjore .\n", "2020-02-20 16:26:53,605 Example #1\n", "2020-02-20 16:26:53,605 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:26:53,605 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:26:53,605 \tHypothesis: Bende , omiya thuolo mar miyo luor kaka wuonwa kod wiye otegno . ”\n", "2020-02-20 16:26:53,605 Example #2\n", "2020-02-20 16:26:53,605 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:26:53,605 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:26:53,605 \tHypothesis: Neno kaka Muma wacho nyalo konyo ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:26:53,605 Example #3\n", "2020-02-20 16:26:53,606 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:26:53,606 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:26:53,606 \tHypothesis: Noyudo osendiko e wi “ weche duto ma Nyasaye nowuoyo kuome kokalo kuom jonabi maler mag higini machon . ”\n", "2020-02-20 16:26:53,606 Validation result at epoch 28, step 42000: bleu: 22.21, loss: 41490.4336, ppl: 5.3834, duration: 87.7938s\n", "2020-02-20 16:27:22,751 Epoch 28 Step: 42100 Batch Loss: 1.777676 Tokens per Sec: 7661, Lr: 0.000300\n", "2020-02-20 16:27:51,485 Epoch 28 Step: 42200 Batch Loss: 1.654832 Tokens per Sec: 7571, Lr: 0.000300\n", "2020-02-20 16:28:20,580 Epoch 28 Step: 42300 Batch Loss: 1.638294 Tokens per Sec: 7781, Lr: 0.000300\n", "2020-02-20 16:28:49,416 Epoch 28 Step: 42400 Batch Loss: 1.761941 Tokens per Sec: 7631, Lr: 0.000300\n", "2020-02-20 16:29:18,212 Epoch 28 Step: 42500 Batch Loss: 1.954315 Tokens per Sec: 7629, Lr: 0.000300\n", "2020-02-20 16:29:34,278 Epoch 28: total training loss 2562.86\n", "2020-02-20 16:29:34,279 EPOCH 29\n", "2020-02-20 16:29:46,915 Epoch 29 Step: 42600 Batch Loss: 1.826073 Tokens per Sec: 7389, Lr: 0.000300\n", "2020-02-20 16:30:15,572 Epoch 29 Step: 42700 Batch Loss: 1.895255 Tokens per Sec: 7517, Lr: 0.000300\n", "2020-02-20 16:30:44,511 Epoch 29 Step: 42800 Batch Loss: 1.770706 Tokens per Sec: 7666, Lr: 0.000300\n", "2020-02-20 16:31:13,390 Epoch 29 Step: 42900 Batch Loss: 1.765434 Tokens per Sec: 7636, Lr: 0.000300\n", "2020-02-20 16:31:42,077 Epoch 29 Step: 43000 Batch Loss: 1.890643 Tokens per Sec: 7591, Lr: 0.000300\n", "2020-02-20 16:33:10,168 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:33:10,168 Saving new checkpoint.\n", "2020-02-20 16:33:10,395 Example #0\n", "2020-02-20 16:33:10,396 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:33:10,396 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:33:10,396 \tHypothesis: Luwo chike Jehova dwaro ni ginyis ni chike Nyasaye owinjore .\n", "2020-02-20 16:33:10,396 Example #1\n", "2020-02-20 16:33:10,396 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:33:10,396 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:33:10,396 \tHypothesis: Bende , omiya luor matut kaka wuonwa kod wiye . ”\n", "2020-02-20 16:33:10,396 Example #2\n", "2020-02-20 16:33:10,397 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:33:10,397 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:33:10,397 \tHypothesis: Nono gik ma Ndiko wacho nyalo konyowa ng’eyo kaka chunywa dwaro .\n", "2020-02-20 16:33:10,397 Example #3\n", "2020-02-20 16:33:10,397 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:33:10,397 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:33:10,397 \tHypothesis: Noyudo osendiko e wi “ weche duto ma Nyasaye nowacho kokalo kuom jonabi maler mag higini machon . ”\n", "2020-02-20 16:33:10,397 Validation result at epoch 29, step 43000: bleu: 22.42, loss: 41402.2656, ppl: 5.3642, duration: 88.3194s\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "2020-02-20 16:33:39,037 Epoch 29 Step: 43100 Batch Loss: 1.562337 Tokens per Sec: 7495, Lr: 0.000300\n", "2020-02-20 16:34:07,539 Epoch 29 Step: 43200 Batch Loss: 1.512897 Tokens per Sec: 7540, Lr: 0.000300\n", "2020-02-20 16:34:36,579 Epoch 29 Step: 43300 Batch Loss: 1.912865 Tokens per Sec: 7742, Lr: 0.000300\n", "2020-02-20 16:35:05,179 Epoch 29 Step: 43400 Batch Loss: 1.795569 Tokens per Sec: 7543, Lr: 0.000300\n", "2020-02-20 16:35:34,011 Epoch 29 Step: 43500 Batch Loss: 1.239226 Tokens per Sec: 7626, Lr: 0.000300\n", "2020-02-20 16:36:02,734 Epoch 29 Step: 43600 Batch Loss: 1.947909 Tokens per Sec: 7432, Lr: 0.000300\n", "2020-02-20 16:36:31,516 Epoch 29 Step: 43700 Batch Loss: 1.863791 Tokens per Sec: 7587, Lr: 0.000300\n", "2020-02-20 16:37:00,585 Epoch 29 Step: 43800 Batch Loss: 1.787845 Tokens per Sec: 7634, Lr: 0.000300\n", "2020-02-20 16:37:29,387 Epoch 29 Step: 43900 Batch Loss: 1.920231 Tokens per Sec: 7677, Lr: 0.000300\n", "2020-02-20 16:37:57,968 Epoch 29 Step: 44000 Batch Loss: 1.613283 Tokens per Sec: 7532, Lr: 0.000300\n", "2020-02-20 16:39:25,983 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:39:25,983 Saving new checkpoint.\n", "2020-02-20 16:39:26,198 Example #0\n", "2020-02-20 16:39:26,199 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:39:26,199 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:39:26,199 \tHypothesis: Luwo chike Jehova dwaro ni mondo ginyis ni chike Nyasaye owinjore .\n", "2020-02-20 16:39:26,199 Example #1\n", "2020-02-20 16:39:26,199 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:39:26,199 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:39:26,199 \tHypothesis: Omiya thuolo mar miye luor kaka wuonwa kendo tayo winjruokna gi Nyasaye . ”\n", "2020-02-20 16:39:26,200 Example #2\n", "2020-02-20 16:39:26,200 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:39:26,200 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:39:26,200 \tHypothesis: Neno kaka Muma wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:39:26,200 Example #3\n", "2020-02-20 16:39:26,200 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:39:26,200 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:39:26,200 \tHypothesis: Nondiko e wi “ weche duto ma Nyasaye nowacho kokalo kuom jonabi maler kuom kinde machon . ”\n", "2020-02-20 16:39:26,200 Validation result at epoch 29, step 44000: bleu: 21.92, loss: 41178.2109, ppl: 5.3156, duration: 88.2323s\n", "2020-02-20 16:39:48,441 Epoch 29: total training loss 2546.09\n", "2020-02-20 16:39:48,441 EPOCH 30\n", "2020-02-20 16:39:55,621 Epoch 30 Step: 44100 Batch Loss: 1.842117 Tokens per Sec: 7399, Lr: 0.000300\n", "2020-02-20 16:40:24,504 Epoch 30 Step: 44200 Batch Loss: 1.342121 Tokens per Sec: 7623, Lr: 0.000300\n", "2020-02-20 16:40:53,230 Epoch 30 Step: 44300 Batch Loss: 1.576577 Tokens per Sec: 7583, Lr: 0.000300\n", "2020-02-20 16:41:22,240 Epoch 30 Step: 44400 Batch Loss: 1.735350 Tokens per Sec: 7642, Lr: 0.000300\n", "2020-02-20 16:41:51,012 Epoch 30 Step: 44500 Batch Loss: 1.906591 Tokens per Sec: 7496, Lr: 0.000300\n", "2020-02-20 16:42:19,766 Epoch 30 Step: 44600 Batch Loss: 1.563841 Tokens per Sec: 7569, Lr: 0.000300\n", "2020-02-20 16:42:48,494 Epoch 30 Step: 44700 Batch Loss: 1.597712 Tokens per Sec: 7567, Lr: 0.000300\n", "2020-02-20 16:43:17,344 Epoch 30 Step: 44800 Batch Loss: 1.467246 Tokens per Sec: 7521, Lr: 0.000300\n", "2020-02-20 16:43:45,937 Epoch 30 Step: 44900 Batch Loss: 1.723284 Tokens per Sec: 7495, Lr: 0.000300\n", "2020-02-20 16:44:14,470 Epoch 30 Step: 45000 Batch Loss: 1.647184 Tokens per Sec: 7650, Lr: 0.000300\n", "2020-02-20 16:45:42,572 Hooray! New best validation result [ppl]!\n", "2020-02-20 16:45:42,572 Saving new checkpoint.\n", "2020-02-20 16:45:42,798 Example #0\n", "2020-02-20 16:45:42,798 \tSource: Their own willing obedience to what Jehovah requires should demonstrate that God’s rules are reasonable .\n", "2020-02-20 16:45:42,798 \tReference: Ka gin giwegi giikore luwo gima Jehova dwaro , mano biro nyiso ni chike Nyasaye gin chike mowinjore kendo inyalo makgi .\n", "2020-02-20 16:45:42,798 \tHypothesis: Luwo chike Jehova e yo ma Jehova dwaro ni mondo onyis ni chike Nyasaye owinjore .\n", "2020-02-20 16:45:42,798 Example #1\n", "2020-02-20 16:45:42,798 \tSource: It also gives me more reason to respect him as my father and spiritual head . ”\n", "2020-02-20 16:45:42,799 \tReference: Mano bende miyo amedo miye luor kaka wuonwa kendo kaka ng’at matayowa e weche mag lamo . ”\n", "2020-02-20 16:45:42,799 \tHypothesis: Bende , omiyo amiyo luor kaka wuonwa kod wiye . ”\n", "2020-02-20 16:45:42,799 Example #2\n", "2020-02-20 16:45:42,799 \tSource: Seeing ourselves in the light of what the Scriptures say can indeed be a powerful aid in identifying the intentions of the heart .\n", "2020-02-20 16:45:42,799 \tReference: Ka wanonore wawegi kaluwore gi gima Ndiko wacho , mano nyalo konyowa ng’eyo paro manie chunywa .\n", "2020-02-20 16:45:42,799 \tHypothesis: Neno kaka Muma wacho nyalo bedo gima duong ’ ahinya e fwenyo chuny ng’ato .\n", "2020-02-20 16:45:42,799 Example #3\n", "2020-02-20 16:45:42,799 \tSource: He had written about the “ restoration of all things of which God spoke through the mouth of his holy prophets of old time . ”\n", "2020-02-20 16:45:42,799 \tReference: Noyudo Dunn osendiko wach “ loso gik moko duto odok makare , kaka Nyasaye nowacho gi dho jonabi duto maler nyaka a chakruok piny . ”\n", "2020-02-20 16:45:42,799 \tHypothesis: Noyudo osendiko e wi “ weche duto ma Nyasaye ne wuoyo kuome kokalo kuom jonabi maler ma ne okoro chon . ”\n", "2020-02-20 16:45:42,800 Validation result at epoch 30, step 45000: bleu: 22.35, loss: 41098.9102, ppl: 5.2986, duration: 88.3286s\n", "2020-02-20 16:46:11,424 Epoch 30 Step: 45100 Batch Loss: 1.743886 Tokens per Sec: 7400, Lr: 0.000300\n", "2020-02-20 16:46:40,731 Epoch 30 Step: 45200 Batch Loss: 1.809998 Tokens per Sec: 7796, Lr: 0.000300\n", "2020-02-20 16:47:09,258 Epoch 30 Step: 45300 Batch Loss: 1.818204 Tokens per Sec: 7525, Lr: 0.000300\n", "2020-02-20 16:47:38,422 Epoch 30 Step: 45400 Batch Loss: 1.736148 Tokens per Sec: 7673, Lr: 0.000300\n", "2020-02-20 16:48:07,190 Epoch 30 Step: 45500 Batch Loss: 1.416873 Tokens per Sec: 7698, Lr: 0.000300\n", "2020-02-20 16:48:34,725 Epoch 30: total training loss 2530.09\n", "2020-02-20 16:48:34,725 Training ended after 30 epochs.\n", "2020-02-20 16:48:34,725 Best validation result at step 45000: 5.30 ppl.\n", "2020-02-20 16:49:17,978 dev bleu: 23.22 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-02-20 16:49:17,979 Translations saved to: models/enluo_transformer/00045000.hyps.dev\n", "2020-02-20 16:50:37,458 test bleu: 32.64 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-02-20 16:50:37,460 Translations saved to: models/enluo_transformer/00045000.hyps.test\n" ] } ], "source": [ "# Train the model\n", "# You can press Ctrl-C to stop. And then run the next cell to save your checkpoints! \n", "! cd ../../joeynmt; python3 -m joeynmt train configs/transformer_$src$tgt.yaml" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "collapsed": true, "id": "MBoDS09JM807" }, "outputs": [], "source": [ "# Copy the created models from the notebook storage to google drive for persistant storage \n", "!cp -r joeynmt/models/${src}${tgt}_transformer/* \"$gdrive_path/models/${src}${tgt}_transformer/\"" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "colab": {}, "colab_type": "code", "id": "n94wlrCjVc17" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Steps: 1000\tLoss: 98175.97656\tPPL: 53.68433\tbleu: 1.00511\tLR: 0.00030000\t*\r\n", "Steps: 2000\tLoss: 83177.78906\tPPL: 29.21336\tbleu: 2.74320\tLR: 0.00030000\t*\r\n", "Steps: 3000\tLoss: 75435.21094\tPPL: 21.33823\tbleu: 4.85840\tLR: 0.00030000\t*\r\n", "Steps: 4000\tLoss: 69503.42969\tPPL: 16.77417\tbleu: 7.70259\tLR: 0.00030000\t*\r\n", "Steps: 5000\tLoss: 65853.02344\tPPL: 14.46510\tbleu: 9.55595\tLR: 0.00030000\t*\r\n", "Steps: 6000\tLoss: 63031.21875\tPPL: 12.90035\tbleu: 10.99594\tLR: 0.00030000\t*\r\n", "Steps: 7000\tLoss: 60268.74609\tPPL: 11.53260\tbleu: 11.95757\tLR: 0.00030000\t*\r\n", "Steps: 8000\tLoss: 58475.81250\tPPL: 10.72349\tbleu: 12.97758\tLR: 0.00030000\t*\r\n", "Steps: 9000\tLoss: 57032.02734\tPPL: 10.11339\tbleu: 13.53204\tLR: 0.00030000\t*\r\n", "Steps: 10000\tLoss: 54667.55078\tPPL: 9.18829\tbleu: 15.21627\tLR: 0.00030000\t*\r\n", "Steps: 11000\tLoss: 53643.98438\tPPL: 8.81454\tbleu: 15.05572\tLR: 0.00030000\t*\r\n", "Steps: 12000\tLoss: 52487.89453\tPPL: 8.41065\tbleu: 15.88412\tLR: 0.00030000\t*\r\n", "Steps: 13000\tLoss: 51491.73047\tPPL: 8.07751\tbleu: 16.26519\tLR: 0.00030000\t*\r\n", "Steps: 14000\tLoss: 50492.83203\tPPL: 7.75670\tbleu: 17.06413\tLR: 0.00030000\t*\r\n", "Steps: 15000\tLoss: 49596.87500\tPPL: 7.47981\tbleu: 18.29802\tLR: 0.00030000\t*\r\n", "Steps: 16000\tLoss: 49577.44141\tPPL: 7.47391\tbleu: 17.63687\tLR: 0.00030000\t*\r\n", "Steps: 17000\tLoss: 48477.73438\tPPL: 7.14778\tbleu: 17.78224\tLR: 0.00030000\t*\r\n", "Steps: 18000\tLoss: 47651.57812\tPPL: 6.91217\tbleu: 18.38615\tLR: 0.00030000\t*\r\n", "Steps: 19000\tLoss: 47321.08984\tPPL: 6.82011\tbleu: 18.75187\tLR: 0.00030000\t*\r\n", "Steps: 20000\tLoss: 46756.95312\tPPL: 6.66578\tbleu: 19.03500\tLR: 0.00030000\t*\r\n", "Steps: 21000\tLoss: 46212.31641\tPPL: 6.52011\tbleu: 19.73430\tLR: 0.00030000\t*\r\n", "Steps: 22000\tLoss: 46018.47656\tPPL: 6.46903\tbleu: 19.07405\tLR: 0.00030000\t*\r\n", "Steps: 23000\tLoss: 45625.03516\tPPL: 6.36659\tbleu: 19.67849\tLR: 0.00030000\t*\r\n", "Steps: 24000\tLoss: 45200.48047\tPPL: 6.25787\tbleu: 20.11382\tLR: 0.00030000\t*\r\n", "Steps: 25000\tLoss: 44906.89844\tPPL: 6.18377\tbleu: 19.90823\tLR: 0.00030000\t*\r\n", "Steps: 26000\tLoss: 44817.06641\tPPL: 6.16128\tbleu: 19.35355\tLR: 0.00030000\t*\r\n", "Steps: 27000\tLoss: 44218.17578\tPPL: 6.01338\tbleu: 20.43616\tLR: 0.00030000\t*\r\n", "Steps: 28000\tLoss: 44157.72656\tPPL: 5.99865\tbleu: 20.51112\tLR: 0.00030000\t*\r\n", "Steps: 29000\tLoss: 43801.03516\tPPL: 5.91246\tbleu: 20.96693\tLR: 0.00030000\t*\r\n", "Steps: 30000\tLoss: 43687.10547\tPPL: 5.88520\tbleu: 21.00739\tLR: 0.00030000\t*\r\n", "Steps: 31000\tLoss: 43309.37500\tPPL: 5.79569\tbleu: 20.82572\tLR: 0.00030000\t*\r\n", "Steps: 32000\tLoss: 43100.05078\tPPL: 5.74668\tbleu: 21.35273\tLR: 0.00030000\t*\r\n", "Steps: 33000\tLoss: 42939.68750\tPPL: 5.70941\tbleu: 20.98994\tLR: 0.00030000\t*\r\n", "Steps: 34000\tLoss: 42751.12891\tPPL: 5.66590\tbleu: 21.51582\tLR: 0.00030000\t*\r\n", "Steps: 35000\tLoss: 42456.41797\tPPL: 5.59856\tbleu: 21.60048\tLR: 0.00030000\t*\r\n", "Steps: 36000\tLoss: 42349.58203\tPPL: 5.57435\tbleu: 21.19294\tLR: 0.00030000\t*\r\n", "Steps: 37000\tLoss: 42305.62891\tPPL: 5.56441\tbleu: 21.66766\tLR: 0.00030000\t*\r\n", "Steps: 38000\tLoss: 42005.72656\tPPL: 5.49712\tbleu: 21.63990\tLR: 0.00030000\t*\r\n", "Steps: 39000\tLoss: 41955.66797\tPPL: 5.48597\tbleu: 21.74473\tLR: 0.00030000\t*\r\n", "Steps: 40000\tLoss: 41715.17188\tPPL: 5.43270\tbleu: 21.32302\tLR: 0.00030000\t*\r\n", "Steps: 41000\tLoss: 41422.91016\tPPL: 5.36866\tbleu: 22.04028\tLR: 0.00030000\t*\r\n", "Steps: 42000\tLoss: 41490.43359\tPPL: 5.38339\tbleu: 22.21160\tLR: 0.00030000\t\r\n", "Steps: 43000\tLoss: 41402.26562\tPPL: 5.36417\tbleu: 22.42439\tLR: 0.00030000\t*\r\n", "Steps: 44000\tLoss: 41178.21094\tPPL: 5.31563\tbleu: 21.92493\tLR: 0.00030000\t*\r\n", "Steps: 45000\tLoss: 41098.91016\tPPL: 5.29855\tbleu: 22.34637\tLR: 0.00030000\t*\r\n" ] } ], "source": [ "# Output our validation accuracy\n", "! cat \"../../joeynmt/models/${src}${tgt}_transformer/validations.txt\"" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "colab": {}, "colab_type": "code", "id": "66WhRE9lIhoD" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n", "/home/espoir_mur_gmail_com/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n", "2020-02-20 16:51:25,414 - dev bleu: 23.22 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-02-20 16:52:45,303 - test bleu: 32.64 [Beam search decoding with beam size = 5 and alpha = 1.0]\n" ] } ], "source": [ "# Test our model\n", "! cd ../../joeynmt; python3 -m joeynmt test \"models/${src}${tgt}_transformer/config.yaml\"" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [] } ], "metadata": { "accelerator": "GPU", "colab": { "collapsed_sections": [], "name": "starter_notebook.ipynb", "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 1 }