{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "accelerator": "GPU", "colab": { "name": "starter_notebook.ipynb", "provenance": [], "collapsed_sections": [], "toc_visible": true, "include_colab_link": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.6" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Igc5itf-xMGj" }, "source": [ "# Masakhane - Machine Translation for African Languages (Using JoeyNMT)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "x4fXCKCf36IK" }, "source": [ "## Note before beginning:\n", "### - The idea is that you should be able to make minimal changes to this in order to get SOME result for your own translation corpus. \n", "\n", "### - The tl;dr: Go to the **\"TODO\"** comments which will tell you what to update to get up and running\n", "\n", "### - If you actually want to have a clue what you're doing, read the text and peek at the links\n", "\n", "### - With 100 epochs, it should take around 7 hours to run in Google Colab\n", "\n", "### - Once you've gotten a result for your language, please attach and email your notebook that generated it to masakhanetranslation@gmail.com\n", "\n", "### - If you care enough and get a chance, doing a brief background on your language would be amazing. See examples in [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "l929HimrxS0a" }, "source": [ "## Retrieve your data & make a parallel corpus\n", "\n", "If you are wanting to use the JW300 data referenced on the Masakhane website or in our GitHub repo, you can use `opus-tools` to convert the data into a convenient format. `opus_read` from that package provides a convenient tool for reading the native aligned XML files and to convert them to TMX format. The tool can also be used to fetch relevant files from OPUS on the fly and to filter the data as necessary. [Read the documentation](https://pypi.org/project/opustools-pkg/) for more details.\n", "\n", "Once you have your corpus files in TMX format (an xml structure which will include the sentences in your target language and your source language in a single file), we recommend reading them into a pandas dataframe. Thankfully, Jade wrote a silly `tmx2dataframe` package which converts your tmx file to a pandas dataframe. " ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "oGRmDELn7Az0", "outputId": "93358349-d2c5-4f33-dce0-baf8f2bdc5ea", "colab": { "base_uri": "https://localhost:8080/", "height": 122 } }, "source": [ "from google.colab import drive\n", "drive.mount('/content/drive')" ], "execution_count": 1, "outputs": [ { "output_type": "stream", "text": [ "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n", "\n", "Enter your authorization code:\n", "··········\n", "Mounted at /content/drive\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "Cn3tgQLzUxwn", "colab": {} }, "source": [ "# TODO: Set your source and target languages. Keep in mind, these traditionally use language codes as found here:\n", "# These will also become the suffix's of all vocab and corpus files used throughout\n", "import os\n", "source_language = \"en\"\n", "target_language = \"iso\" \n", "lc = False # If True, lowercase the data.\n", "seed = 42 # Random seed for shuffling.\n", "tag = \"baseline\" # Give a unique name to your folder - this is to ensure you don't rewrite any models you've already submitted\n", "\n", "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n", "os.environ[\"tgt\"] = target_language\n", "os.environ[\"tag\"] = tag\n", "\n", "# This will save it to a folder in our gdrive instead!\n", "!mkdir -p \"/content/drive/My Drive/masakhane/$src-$tgt-$tag\"\n", "os.environ[\"gdrive_path\"] = \"/content/drive/My Drive/masakhane/%s-%s-%s\" % (source_language, target_language, tag)" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "kBSgJHEw7Nvx", "outputId": "11f957be-66db-4077-9fe6-e21cf6d63315", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "source": [ "!echo $gdrive_path" ], "execution_count": 6, "outputs": [ { "output_type": "stream", "text": [ "/content/drive/My Drive/masakhane/en-iso-baseline\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "gA75Fs9ys8Y9", "outputId": "85f1a7de-5ab8-45b9-cdb0-9ae3199df91b", "colab": { "base_uri": "https://localhost:8080/", "height": 102 } }, "source": [ "# Install opus-tools\n", "! pip install opustools-pkg" ], "execution_count": 7, "outputs": [ { "output_type": "stream", "text": [ "Collecting opustools-pkg\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/6c/9f/e829a0cceccc603450cd18e1ff80807b6237a88d9a8df2c0bb320796e900/opustools_pkg-0.0.52-py3-none-any.whl (80kB)\n", "\r\u001b[K |████ | 10kB 23.8MB/s eta 0:00:01\r\u001b[K |████████ | 20kB 1.7MB/s eta 0:00:01\r\u001b[K |████████████▏ | 30kB 2.5MB/s eta 0:00:01\r\u001b[K |████████████████▏ | 40kB 1.7MB/s eta 0:00:01\r\u001b[K |████████████████████▎ | 51kB 2.1MB/s eta 0:00:01\r\u001b[K |████████████████████████▎ | 61kB 2.5MB/s eta 0:00:01\r\u001b[K |████████████████████████████▎ | 71kB 2.9MB/s eta 0:00:01\r\u001b[K |████████████████████████████████| 81kB 2.5MB/s \n", "\u001b[?25hInstalling collected packages: opustools-pkg\n", "Successfully installed opustools-pkg-0.0.52\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "xq-tDZVks7ZD", "outputId": "9d17c9e1-b6a7-43f4-ee1e-3ea9a8559c27", "colab": { "base_uri": "https://localhost:8080/", "height": 204 } }, "source": [ "# Downloading our corpus\n", "! opus_read -d JW300 -s $src -t $tgt -wm moses -w jw300.$src jw300.$tgt -q\n", "\n", "# extract the corpus file\n", "! gunzip JW300_latest_xml_$src-$tgt.xml.gz" ], "execution_count": 8, "outputs": [ { "output_type": "stream", "text": [ "\n", "Alignment file /proj/nlpl/data/OPUS/JW300/latest/xml/en-iso.xml.gz not found. The following files are available for downloading:\n", "\n", " 2 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en-iso.xml.gz\n", " 263 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en.zip\n", " 26 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/iso.zip\n", "\n", " 291 MB Total size\n", "./JW300_latest_xml_en-iso.xml.gz ... 100% of 2 MB\n", "./JW300_latest_xml_en.zip ... 100% of 263 MB\n", "./JW300_latest_xml_iso.zip ... 100% of 26 MB\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "n48GDRnP8y2G", "colab_type": "code", "outputId": "8949fb1b-854c-4644-a03c-43fdb68dbc42", "colab": { "base_uri": "https://localhost:8080/", "height": 578 } }, "source": [ "# Download the global test set.\n", "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n", " \n", "# And the specific test set for this language pair.\n", "os.environ[\"trg\"] = target_language \n", "os.environ[\"src\"] = source_language \n", "\n", "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.en \n", "! mv test.en-$trg.en test.en\n", "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.$trg \n", "! mv test.en-$trg.$trg test.$trg" ], "execution_count": 9, "outputs": [ { "output_type": "stream", "text": [ "--2020-01-17 04:59:50-- https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 277791 (271K) [text/plain]\n", "Saving to: ‘test.en-any.en’\n", "\n", "\rtest.en-any.en 0%[ ] 0 --.-KB/s \rtest.en-any.en 100%[===================>] 271.28K --.-KB/s in 0.05s \n", "\n", "2020-01-17 04:59:51 (5.15 MB/s) - ‘test.en-any.en’ saved [277791/277791]\n", "\n", "--2020-01-17 04:59:52-- https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-iso.en\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 205488 (201K) [text/plain]\n", "Saving to: ‘test.en-iso.en’\n", "\n", "test.en-iso.en 100%[===================>] 200.67K --.-KB/s in 0.04s \n", "\n", "2020-01-17 04:59:52 (5.30 MB/s) - ‘test.en-iso.en’ saved [205488/205488]\n", "\n", "--2020-01-17 04:59:54-- https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-iso.iso\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 251918 (246K) [text/plain]\n", "Saving to: ‘test.en-iso.iso’\n", "\n", "test.en-iso.iso 100%[===================>] 246.01K --.-KB/s in 0.05s \n", "\n", "2020-01-17 04:59:55 (4.68 MB/s) - ‘test.en-iso.iso’ saved [251918/251918]\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "NqDG-CI28y2L", "colab_type": "code", "outputId": "44ef9271-b0fc-418d-dc5f-04c90e4e38cd", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "source": [ "# Read the test data to filter from train and dev splits.\n", "# Store english portion in set for quick filtering checks.\n", "en_test_sents = set()\n", "filter_test_sents = \"test.en-any.en\"\n", "j = 0\n", "with open(filter_test_sents) as f:\n", " for line in f:\n", " en_test_sents.add(line.strip())\n", " j += 1\n", "print('Loaded {} global test sentences to filter from the training/dev data.'.format(j))" ], "execution_count": 10, "outputs": [ { "output_type": "stream", "text": [ "Loaded 3571 global test sentences to filter from the training/dev data.\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "3CNdwLBCfSIl", "outputId": "fcf09810-26de-481b-8ba0-39853c4378b0", "colab": { "base_uri": "https://localhost:8080/", "height": 159 } }, "source": [ "import pandas as pd\n", "\n", "# TMX file to dataframe\n", "source_file = 'jw300.' + source_language\n", "target_file = 'jw300.' + target_language\n", "\n", "source = []\n", "target = []\n", "skip_lines = [] # Collect the line numbers of the source portion to skip the same lines for the target portion.\n", "with open(source_file) as f:\n", " for i, line in enumerate(f):\n", " # Skip sentences that are contained in the test set.\n", " if line.strip() not in en_test_sents:\n", " source.append(line.strip())\n", " else:\n", " skip_lines.append(i) \n", "with open(target_file) as f:\n", " for j, line in enumerate(f):\n", " # Only add to corpus if corresponding source was not skipped.\n", " if j not in skip_lines:\n", " target.append(line.strip())\n", " \n", "print('Loaded data and skipped {}/{} lines since contained in test set.'.format(len(skip_lines), i))\n", " \n", "df = pd.DataFrame(zip(source, target), columns=['source_sentence', 'target_sentence'])\n", "# if you get TypeError: data argument can't be an iterator is because of your zip version run this below\n", "#df = pd.DataFrame(list(zip(source, target)), columns=['source_sentence', 'target_sentence'])\n", "df.head(3)" ], "execution_count": 11, "outputs": [ { "output_type": "stream", "text": [ "Loaded data and skipped 5685/243487 lines since contained in test set.\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
source_sentencetarget_sentence
0“ Only One of the Many Lives That You Touched ”“ Omọvo Ahwo Buobu nọ Who Duobọ te Uzuazọ Riẹ ”
1BACK in January 1996 , Carol was sick with a b...EVAỌ January 1996 , Carol ọ jẹ mọ ẹyao ẹvori .
2She was in her 60 ’ s and until then had alway...Ọ kpako te ikpe 60 no yọ oke yena kpobi ọ jọ a...
\n", "
" ], "text/plain": [ " source_sentence target_sentence\n", "0 “ Only One of the Many Lives That You Touched ” “ Omọvo Ahwo Buobu nọ Who Duobọ te Uzuazọ Riẹ ”\n", "1 BACK in January 1996 , Carol was sick with a b... EVAỌ January 1996 , Carol ọ jẹ mọ ẹyao ẹvori .\n", "2 She was in her 60 ’ s and until then had alway... Ọ kpako te ikpe 60 no yọ oke yena kpobi ọ jọ a..." ] }, "metadata": { "tags": [] }, "execution_count": 11 } ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "YkuK3B4p2AkN" }, "source": [ "## Pre-processing and export\n", "\n", "It is generally a good idea to remove duplicate translations and conflicting translations from the corpus. In practice, these public corpora include some number of these that need to be cleaned.\n", "\n", "In addition we will split our data into dev/test/train and export to the filesystem." ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "M_2ouEOH1_1q", "outputId": "b109ef42-7557-4c8c-8952-874ac33ed667", "colab": { "base_uri": "https://localhost:8080/", "height": 187 } }, "source": [ "# drop duplicate translations\n", "df_pp = df.drop_duplicates()\n", "\n", "# drop conflicting translations\n", "# (this is optional and something that you might want to comment out \n", "# depending on the size of your corpus)\n", "df_pp.drop_duplicates(subset='source_sentence', inplace=True)\n", "df_pp.drop_duplicates(subset='target_sentence', inplace=True)\n", "\n", "# Shuffle the data to remove bias in dev set selection.\n", "df_pp = df_pp.sample(frac=1, random_state=seed).reset_index(drop=True)" ], "execution_count": 12, "outputs": [ { "output_type": "stream", "text": [ "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:6: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " \n", "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:7: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " import sys\n" ], "name": "stderr" } ] }, { "cell_type": "code", "metadata": { "id": "Z_1BwAApEtMk", "colab_type": "code", "outputId": "175b8c38-2c43-47f7-bccc-0cffe0336959", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Install fuzzy wuzzy to remove \"almost duplicate\" sentences in the\n", "# test and training sets.\n", "! pip install fuzzywuzzy\n", "! pip install python-Levenshtein\n", "import time\n", "from fuzzywuzzy import process\n", "import numpy as np\n", "\n", "# reset the index of the training set after previous filtering\n", "df_pp.reset_index(drop=False, inplace=True)\n", "\n", "# Remove samples from the training data set if they \"almost overlap\" with the\n", "# samples in the test set.\n", "\n", "# Filtering function. Adjust pad to narrow down the candidate matches to\n", "# within a certain length of characters of the given sample.\n", "def fuzzfilter(sample, candidates, pad):\n", " candidates = [x for x in candidates if len(x) <= len(sample)+pad and len(x) >= len(sample)-pad] \n", " if len(candidates) > 0:\n", " return process.extractOne(sample, candidates)[1]\n", " else:\n", " return np.nan\n", "\n", "# NOTE - This might run slow depending on the size of your training set. We are\n", "# printing some information to help you track how long it would take. \n", "scores = []\n", "start_time = time.time()\n", "for idx, row in df_pp.iterrows():\n", " scores.append(fuzzfilter(row['source_sentence'], list(en_test_sents), 5))\n", " if idx % 1000 == 0:\n", " hours, rem = divmod(time.time() - start_time, 3600)\n", " minutes, seconds = divmod(rem, 60)\n", " print(\"{:0>2}:{:0>2}:{:05.2f}\".format(int(hours),int(minutes),seconds), \"%0.2f percent complete\" % (100.0*float(idx)/float(len(df_pp))))\n", "\n", "# Filter out \"almost overlapping samples\"\n", "df_pp['scores'] = scores\n", "df_pp = df_pp[df_pp['scores'] < 95]" ], "execution_count": 13, "outputs": [ { "output_type": "stream", "text": [ "Collecting fuzzywuzzy\n", " Downloading https://files.pythonhosted.org/packages/d8/f1/5a267addb30ab7eaa1beab2b9323073815da4551076554ecc890a3595ec9/fuzzywuzzy-0.17.0-py2.py3-none-any.whl\n", "Installing collected packages: fuzzywuzzy\n", "Successfully installed fuzzywuzzy-0.17.0\n", "Collecting python-Levenshtein\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/42/a9/d1785c85ebf9b7dfacd08938dd028209c34a0ea3b1bcdb895208bd40a67d/python-Levenshtein-0.12.0.tar.gz (48kB)\n", "\u001b[K |████████████████████████████████| 51kB 1.7MB/s \n", "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from python-Levenshtein) (42.0.2)\n", "Building wheels for collected packages: python-Levenshtein\n", " Building wheel for python-Levenshtein (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for python-Levenshtein: filename=python_Levenshtein-0.12.0-cp36-cp36m-linux_x86_64.whl size=144672 sha256=27cc679925e9ea7d499147fe08f2564ad8e4295a570a1a075b46139a669d35bc\n", " Stored in directory: /root/.cache/pip/wheels/de/c2/93/660fd5f7559049268ad2dc6d81c4e39e9e36518766eaf7e342\n", "Successfully built python-Levenshtein\n", "Installing collected packages: python-Levenshtein\n", "Successfully installed python-Levenshtein-0.12.0\n", "00:00:00.10 0.00 percent complete\n", "00:00:23.78 0.46 percent complete\n" ], "name": "stdout" }, { "output_type": "stream", "text": [ "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '']\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "00:00:47.40 0.92 percent complete\n", "00:01:10.57 1.39 percent complete\n" ], "name": "stdout" }, { "output_type": "stream", "text": [ "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '*']\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "00:01:33.67 1.85 percent complete\n", "00:01:56.94 2.31 percent complete\n", "00:02:20.72 2.77 percent complete\n", "00:02:43.97 3.23 percent complete\n", "00:03:07.38 3.70 percent complete\n", "00:03:31.62 4.16 percent complete\n", "00:03:56.02 4.62 percent complete\n", "00:04:19.06 5.08 percent complete\n", "00:04:42.39 5.54 percent complete\n", "00:05:06.25 6.00 percent complete\n", "00:05:30.46 6.47 percent complete\n", "00:05:53.63 6.93 percent complete\n", "00:06:17.23 7.39 percent complete\n", "00:06:41.67 7.85 percent complete\n", "00:07:05.22 8.31 percent complete\n", "00:07:29.08 8.78 percent complete\n", "00:07:52.78 9.24 percent complete\n", "00:08:16.19 9.70 percent complete\n", "00:08:39.44 10.16 percent complete\n", "00:09:03.31 10.62 percent complete\n", "00:09:26.40 11.09 percent complete\n", "00:09:51.25 11.55 percent complete\n", "00:10:15.14 12.01 percent complete\n", "00:10:38.52 12.47 percent complete\n", "00:11:02.41 12.93 percent complete\n", "00:11:26.41 13.40 percent complete\n", "00:11:50.05 13.86 percent complete\n", "00:12:13.38 14.32 percent complete\n" ], "name": "stdout" }, { "output_type": "stream", "text": [ "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '․ ․ ․ ․ ․']\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "00:12:37.46 14.78 percent complete\n", "00:13:01.81 15.24 percent complete\n", "00:13:25.69 15.70 percent complete\n", "00:13:49.63 16.17 percent complete\n", "00:14:13.48 16.63 percent complete\n", "00:14:37.29 17.09 percent complete\n", "00:15:01.50 17.55 percent complete\n", "00:15:25.57 18.01 percent complete\n", "00:15:49.18 18.48 percent complete\n", "00:16:13.51 18.94 percent complete\n", "00:16:37.01 19.40 percent complete\n", "00:17:00.10 19.86 percent complete\n", "00:17:23.41 20.32 percent complete\n", "00:17:47.00 20.79 percent complete\n", "00:18:10.96 21.25 percent complete\n", "00:18:34.67 21.71 percent complete\n", "00:18:59.32 22.17 percent complete\n", "00:19:23.61 22.63 percent complete\n", "00:19:47.81 23.10 percent complete\n", "00:20:12.35 23.56 percent complete\n", "00:20:36.05 24.02 percent complete\n", "00:20:59.50 24.48 percent complete\n", "00:21:23.22 24.94 percent complete\n", "00:21:47.62 25.40 percent complete\n", "00:22:11.51 25.87 percent complete\n", "00:22:35.28 26.33 percent complete\n", "00:22:58.71 26.79 percent complete\n", "00:23:22.48 27.25 percent complete\n", "00:23:46.81 27.71 percent complete\n" ], "name": "stdout" }, { "output_type": "stream", "text": [ "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '․ ․ ․ ․ ․ ․ ․ ․']\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "00:24:10.94 28.18 percent complete\n", "00:24:34.16 28.64 percent complete\n", "00:24:58.34 29.10 percent complete\n", "00:25:22.14 29.56 percent complete\n", "00:25:46.52 30.02 percent complete\n", "00:26:09.90 30.49 percent complete\n", "00:26:34.10 30.95 percent complete\n", "00:26:58.06 31.41 percent complete\n", "00:27:21.26 31.87 percent complete\n", "00:27:45.50 32.33 percent complete\n", "00:28:09.00 32.79 percent complete\n", "00:28:32.87 33.26 percent complete\n", "00:28:57.11 33.72 percent complete\n", "00:29:21.37 34.18 percent complete\n", "00:29:45.41 34.64 percent complete\n", "00:30:08.93 35.10 percent complete\n", "00:30:32.97 35.57 percent complete\n", "00:30:56.46 36.03 percent complete\n", "00:31:19.93 36.49 percent complete\n", "00:31:44.33 36.95 percent complete\n", "00:32:07.88 37.41 percent complete\n", "00:32:32.08 37.88 percent complete\n", "00:32:56.34 38.34 percent complete\n", "00:33:20.79 38.80 percent complete\n", "00:33:44.59 39.26 percent complete\n", "00:34:08.98 39.72 percent complete\n", "00:34:32.93 40.19 percent complete\n", "00:34:56.72 40.65 percent complete\n", "00:35:21.12 41.11 percent complete\n", "00:35:45.38 41.57 percent complete\n", "00:36:09.10 42.03 percent complete\n", "00:36:33.30 42.49 percent complete\n", "00:36:57.68 42.96 percent complete\n", "00:37:21.90 43.42 percent complete\n", "00:37:45.21 43.88 percent complete\n", "00:38:09.01 44.34 percent complete\n", "00:38:33.14 44.80 percent complete\n", "00:38:57.65 45.27 percent complete\n", "00:39:21.34 45.73 percent complete\n", "00:39:45.27 46.19 percent complete\n", "00:40:09.16 46.65 percent complete\n" ], "name": "stdout" }, { "output_type": "stream", "text": [ "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '․ ․']\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "00:40:32.51 47.11 percent complete\n", "00:40:56.64 47.58 percent complete\n", "00:41:20.36 48.04 percent complete\n", "00:41:44.61 48.50 percent complete\n", "00:42:08.32 48.96 percent complete\n", "00:42:31.93 49.42 percent complete\n", "00:42:56.28 49.89 percent complete\n", "00:43:20.24 50.35 percent complete\n", "00:43:44.62 50.81 percent complete\n", "00:44:08.91 51.27 percent complete\n", "00:44:32.54 51.73 percent complete\n", "00:44:56.09 52.19 percent complete\n", "00:45:20.95 52.66 percent complete\n", "00:45:44.71 53.12 percent complete\n", "00:46:08.30 53.58 percent complete\n", "00:46:32.61 54.04 percent complete\n", "00:46:56.86 54.50 percent complete\n", "00:47:20.82 54.97 percent complete\n", "00:47:44.99 55.43 percent complete\n", "00:48:08.38 55.89 percent complete\n", "00:48:32.48 56.35 percent complete\n", "00:48:56.67 56.81 percent complete\n", "00:49:20.35 57.28 percent complete\n", "00:49:44.61 57.74 percent complete\n", "00:50:08.07 58.20 percent complete\n", "00:50:32.40 58.66 percent complete\n", "00:50:56.51 59.12 percent complete\n", "00:51:20.65 59.59 percent complete\n", "00:51:44.56 60.05 percent complete\n", "00:52:08.95 60.51 percent complete\n", "00:52:32.77 60.97 percent complete\n", "00:52:56.32 61.43 percent complete\n", "00:53:20.96 61.89 percent complete\n", "00:53:45.01 62.36 percent complete\n", "00:54:08.62 62.82 percent complete\n", "00:54:32.80 63.28 percent complete\n", "00:54:56.52 63.74 percent complete\n", "00:55:20.42 64.20 percent complete\n", "00:55:44.48 64.67 percent complete\n", "00:56:08.38 65.13 percent complete\n", "00:56:32.50 65.59 percent complete\n", "00:56:56.41 66.05 percent complete\n", "00:57:20.00 66.51 percent complete\n", "00:57:43.52 66.98 percent complete\n", "00:58:07.51 67.44 percent complete\n", "00:58:31.43 67.90 percent complete\n", "00:58:55.35 68.36 percent complete\n", "00:59:19.06 68.82 percent complete\n", "00:59:42.85 69.29 percent complete\n", "01:00:07.00 69.75 percent complete\n", "01:00:30.53 70.21 percent complete\n", "01:00:54.45 70.67 percent complete\n", "01:01:18.35 71.13 percent complete\n", "01:01:42.37 71.59 percent complete\n", "01:02:06.05 72.06 percent complete\n", "01:02:29.54 72.52 percent complete\n", "01:02:53.79 72.98 percent complete\n", "01:03:17.95 73.44 percent complete\n", "01:03:41.78 73.90 percent complete\n", "01:04:05.36 74.37 percent complete\n", "01:04:29.49 74.83 percent complete\n", "01:04:54.40 75.29 percent complete\n", "01:05:18.19 75.75 percent complete\n", "01:05:41.97 76.21 percent complete\n", "01:06:05.42 76.68 percent complete\n", "01:06:29.65 77.14 percent complete\n", "01:06:53.24 77.60 percent complete\n", "01:07:16.62 78.06 percent complete\n", "01:07:40.89 78.52 percent complete\n", "01:08:05.22 78.98 percent complete\n", "01:08:28.49 79.45 percent complete\n", "01:08:52.56 79.91 percent complete\n", "01:09:16.44 80.37 percent complete\n", "01:09:40.34 80.83 percent complete\n", "01:10:04.15 81.29 percent complete\n", "01:10:28.28 81.76 percent complete\n", "01:10:51.83 82.22 percent complete\n", "01:11:15.87 82.68 percent complete\n", "01:11:39.95 83.14 percent complete\n", "01:12:03.72 83.60 percent complete\n", "01:12:26.92 84.07 percent complete\n", "01:12:51.37 84.53 percent complete\n" ], "name": "stdout" }, { "output_type": "stream", "text": [ "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '⇩']\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "01:13:15.72 84.99 percent complete\n", "01:13:39.67 85.45 percent complete\n", "01:14:03.39 85.91 percent complete\n", "01:14:27.32 86.38 percent complete\n", "01:14:51.76 86.84 percent complete\n", "01:15:15.58 87.30 percent complete\n", "01:15:39.33 87.76 percent complete\n", "01:16:03.85 88.22 percent complete\n", "01:16:27.71 88.68 percent complete\n", "01:16:51.58 89.15 percent complete\n", "01:17:15.41 89.61 percent complete\n", "01:17:39.83 90.07 percent complete\n", "01:18:04.13 90.53 percent complete\n", "01:18:28.67 90.99 percent complete\n", "01:18:53.01 91.46 percent complete\n", "01:19:16.73 91.92 percent complete\n", "01:19:41.87 92.38 percent complete\n", "01:20:05.67 92.84 percent complete\n", "01:20:29.44 93.30 percent complete\n", "01:20:53.86 93.77 percent complete\n", "01:21:18.33 94.23 percent complete\n", "01:21:42.36 94.69 percent complete\n", "01:22:06.26 95.15 percent complete\n", "01:22:29.82 95.61 percent complete\n", "01:22:53.83 96.08 percent complete\n", "01:23:17.51 96.54 percent complete\n", "01:23:41.70 97.00 percent complete\n", "01:24:05.69 97.46 percent complete\n", "01:24:30.08 97.92 percent complete\n", "01:24:53.94 98.38 percent complete\n", "01:25:17.57 98.85 percent complete\n", "01:25:41.43 99.31 percent complete\n", "01:26:05.41 99.77 percent complete\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "hxxBOCA-xXhy", "outputId": "c1d5d941-1524-480d-9371-b2abd5a19d0b", "colab": { "base_uri": "https://localhost:8080/", "height": 819 } }, "source": [ "# This section does the split between train/dev for the parallel corpora then saves them as separate files\n", "# We use 1000 dev test and the given test set.\n", "import csv\n", "\n", "# Do the split between dev/train and create parallel corpora\n", "num_dev_patterns = 1000\n", "\n", "# Optional: lower case the corpora - this will make it easier to generalize, but without proper casing.\n", "if lc: # Julia: making lowercasing optional\n", " df_pp[\"source_sentence\"] = df_pp[\"source_sentence\"].str.lower()\n", " df_pp[\"target_sentence\"] = df_pp[\"target_sentence\"].str.lower()\n", "\n", "# Julia: test sets are already generated\n", "dev = df_pp.tail(num_dev_patterns) # Herman: Error in original\n", "stripped = df_pp.drop(df_pp.tail(num_dev_patterns).index)\n", "\n", "with open(\"train.\"+source_language, \"w\") as src_file, open(\"train.\"+target_language, \"w\") as trg_file:\n", " for index, row in stripped.iterrows():\n", " src_file.write(row[\"source_sentence\"]+\"\\n\")\n", " trg_file.write(row[\"target_sentence\"]+\"\\n\")\n", " \n", "with open(\"dev.\"+source_language, \"w\") as src_file, open(\"dev.\"+target_language, \"w\") as trg_file:\n", " for index, row in dev.iterrows():\n", " src_file.write(row[\"source_sentence\"]+\"\\n\")\n", " trg_file.write(row[\"target_sentence\"]+\"\\n\")\n", "\n", "#stripped[[\"source_sentence\"]].to_csv(\"train.\"+source_language, header=False, index=False) # Herman: Added `header=False` everywhere\n", "#stripped[[\"target_sentence\"]].to_csv(\"train.\"+target_language, header=False, index=False) # Julia: Problematic handling of quotation marks.\n", "\n", "#dev[[\"source_sentence\"]].to_csv(\"dev.\"+source_language, header=False, index=False)\n", "#dev[[\"target_sentence\"]].to_csv(\"dev.\"+target_language, header=False, index=False)\n", "\n", "# Doublecheck the format below. There should be no extra quotation marks or weird characters.\n", "! head train.*\n", "! head dev.*" ], "execution_count": 14, "outputs": [ { "output_type": "stream", "text": [ "==> train.en <==\n", "Now in his 80 ’ s , John admits that at times he feels despondent .\n", "The Bible exhorts us to “ speak consolingly to the depressed souls , support the weak . ”\n", "Those words of 17th - century British historian Edward Herbert underscore one reason why we need to be forgiving toward others : Sooner or later , we may need to ask others to forgive us .\n", "So why not prove to yourself that what you have been taught from the Bible is indeed the truth ?\n", "Father , glorify your name . ’\n", "God’s Word foretold this development , saying : “ There will be a period of time when they [ people professing to serve God ] will not put up with the healthful teaching , but , in accord with their own desires , they will accumulate teachers for themselves to have their ears tickled . ”\n", "We get water from a well at the police station .\n", "Do we courageously identify ourselves as Jehovah’s Witnesses , even though doing so may mean persecution ?\n", "Tatiana , a full - time evangelizer in Kamchatka , a Russian peninsula located northeast of Japan , began saving for the trip a year in advance .\n", "Since 1939 , the cover of each issue of The Watchtower magazine has displayed the words “ Announcing Jehovah’s Kingdom . ”\n", "\n", "==> train.iso <==\n", "Enẹna Jọn ọ kpako te ikpe udhone gbọ no , ọ ta nọ ẹsejọ ọ be hai wo elọhoma .\n", "Ebaibol na e ta udu họ omai awọ nnọ “ wha ta udu họ enọ udu u re bro awọ , wha fiobọhọ kẹ enọ e ko . ”\n", "Eme yena nọ ogbiku yena ọ ta anwọ ikpe udhusoi akwa ane ( 400 ) nọ i kpemu na , i dhesẹ epanọ u wuzou te re ma rọ vrẹ amọfa , keme ma te siọ amọfa ba eruthọ ẹdẹjọ họ , nọ o te gwọlọ nọ a rọ vrẹ omai .\n", "Kiẹ kẹ oma ra re u mu owhẹ ẹro inọ eware nọ a wuhrẹ owhẹ no Ebaibol ze na ginọ uzẹme .\n", "Koyehọ uru jọ u te no eva ehru ze nọ , ‘ Mẹ kẹ riẹ oro uno , mẹ jẹ te wariẹ kẹ e oro . ’\n", "A jọ Ebaibol ruẹaro kpahe onana , inọ : “ Oke o be tha nọ ahwo [ enọ i se oma rai eg’Ọghẹnẹ ] a rẹ te rehọ uwuhrẹ nọ o gbunu hu , [ rekọ ] ezọ e rẹ te sae okpọ a ve ti koko iwuhrẹ nọ i re ti wuhrẹ ai onọ a guọlọ . ”\n", "Ozae nọ o rrọ ogba iporisi ma re kpohọ jo vo ame .\n", "Kọ ma be hae gbaudu dhesẹ oma wọhọ Isẹri Jihova , o tẹ make rọnọ ere oruo o rẹ sae wha ukpokpoma ze ?\n", "Tatiana , ọtausiuwoma oke - kpobi nọ o no Kamchatka ze , ẹwho Russia jọ nọ ọ rrọ ofẹ obọze ẹkpẹlobọ ovatha - ọre ọrọ Japan , o muọ ugho họ ekoko họ kẹ erẹ na ukpe soso taure oke na u te ti te .\n", "Anwọ ukpe 1939 ze , uzoẹme nọ o rrọ uke emagazini Uwou - Eroro Na họ “ Uwou - Eroro Na Nọ U Bi Whowho Uvie Jihova . ”\n", "==> dev.en <==\n", "We can even ask God to ‘ create in us a pure heart . ’\n", "This was followed by Kingdom News No .\n", "In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "Still , words of apology are a strong force toward making peace .\n", "Elijah brings the boy down to his mother and says : “ See , your son is alive . ”\n", "By means of his ministry , he gave a priceless gift to humankind ​ — a message that revealed the truth about God and His will .\n", "Comparably , Christian husbands assign their mates honor and praise them .\n", "How do we feel about all those who are making sacrifices for the Kingdom , and what should all of us consider ?\n", "Jesus too is a shepherd and a conquering king .\n", "These examples clearly establish that those who truly belong to Jehovah must firmly take their stand for righteousness and against wickedness .\n", "\n", "==> dev.iso <==\n", "Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "Elaeja ọ tẹ rehọ ọmọ na se oni riẹ jẹ ta nọ : “ Ri , ọmọ ra ọ zọe ” no .\n", "Ọ rehọ ẹkwoma odibọgba riẹ kẹ ahwo - akpọ okẹ jọ nọ o ghare thesiwa — ovuẹ nọ u dhesẹ uzẹme na via kpahe Ọghẹnẹ gbe oreva riẹ .\n", "Epọvo na re , ezae Ileleikristi a re tete eyae rai je jiri ai .\n", "Ẹvẹ u fo nọ ma re rri inievo nọ i bi si obọ no eware jọ re a sae rọ iruo Uvie na karo , kọ eme u fo nọ mai omomọvo o re ru ?\n", "Jesu omariẹ yọ othuru - igodẹ gbe ovie nọ o bi fi kparobọ .\n", "Iriruo nana i dhesẹ vevẹ nọ enọ e ginẹ rrọ erọ Jihova a rẹ gbaemu nọ a re ru eware nọ i dhesẹ nọ a kiẹrẹe je mukpahe oware uyoma .\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "epeCydmCyS8X" }, "source": [ "\n", "\n", "---\n", "\n", "\n", "## Installation of JoeyNMT\n", "\n", "JoeyNMT is a simple, minimalist NMT package which is useful for learning and teaching. Check out the documentation for JoeyNMT [here](https://joeynmt.readthedocs.io) " ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "iBRMm4kMxZ8L", "outputId": "bf2e98d0-e30f-444d-f565-6e5dafff7528", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Install JoeyNMT\n", "! git clone https://github.com/joeynmt/joeynmt.git\n", "! cd joeynmt; pip3 install ." ], "execution_count": 3, "outputs": [ { "output_type": "stream", "text": [ "Cloning into 'joeynmt'...\n", "remote: Enumerating objects: 20, done.\u001b[K\n", "remote: Counting objects: 100% (20/20), done.\u001b[K\n", "remote: Compressing objects: 100% (17/17), done.\u001b[K\n", "remote: Total 2204 (delta 8), reused 5 (delta 3), pack-reused 2184\u001b[K\n", "Receiving objects: 100% (2204/2204), 2.60 MiB | 16.26 MiB/s, done.\n", "Resolving deltas: 100% (1529/1529), done.\n", "Processing /content/joeynmt\n", "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.16.0)\n", "Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (6.2.2)\n", "Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.17.5)\n", "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (42.0.2)\n", "Requirement already satisfied: torch>=1.1 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.3.1)\n", "Requirement already satisfied: tensorflow>=1.14 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.15.0)\n", "Requirement already satisfied: torchtext in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.3.1)\n", "Collecting sacrebleu>=1.3.6\n", " Downloading https://files.pythonhosted.org/packages/45/31/1a135b964c169984b27fb2f7a50280fa7f8e6d9d404d8a9e596180487fd1/sacrebleu-1.4.3-py3-none-any.whl\n", "Collecting subword-nmt\n", " Downloading https://files.pythonhosted.org/packages/74/60/6600a7bc09e7ab38bc53a48a20d8cae49b837f93f5842a41fe513a694912/subword_nmt-0.3.7-py2.py3-none-any.whl\n", "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (3.1.2)\n", "Requirement already satisfied: seaborn in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.9.0)\n", "Collecting pyyaml>=5.1\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/3d/d9/ea9816aea31beeadccd03f1f8b625ecf8f645bd66744484d162d84803ce5/PyYAML-5.3.tar.gz (268kB)\n", "\u001b[K |████████████████████████████████| 276kB 8.2MB/s \n", "\u001b[?25hCollecting pylint\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl (302kB)\n", "\u001b[K |████████████████████████████████| 307kB 9.7MB/s \n", "\u001b[?25hRequirement already satisfied: six==1.12 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.12.0)\n", "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.1.0)\n", "Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.1)\n", "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.11.2)\n", "Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n", "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.10.0)\n", "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.0.8)\n", "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.33.6)\n", "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.8.1)\n", "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n", "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n", "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.2.2)\n", "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n", "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.1.8)\n", "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.9.0)\n", "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (4.28.1)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (2.21.0)\n", "Collecting portalocker\n", " Downloading https://files.pythonhosted.org/packages/91/db/7bc703c0760df726839e0699b7f78a4d8217fdc9c7fcb1b51b39c5a22a4e/portalocker-1.5.2-py2.py3-none-any.whl\n", "Requirement already satisfied: typing in /usr/local/lib/python3.6/dist-packages (from sacrebleu>=1.3.6->joeynmt==0.0.1) (3.6.6)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (0.10.0)\n", "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.4.6)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (1.1.0)\n", "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.6.1)\n", "Requirement already satisfied: scipy>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (1.4.1)\n", "Requirement already satisfied: pandas>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (0.25.3)\n", "Collecting mccabe<0.7,>=0.6\n", " Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl\n", "Collecting astroid<2.4,>=2.3.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/ad/ae/86734823047962e7b8c8529186a1ac4a7ca19aaf1aa0c7713c022ef593fd/astroid-2.3.3-py3-none-any.whl (205kB)\n", "\u001b[K |████████████████████████████████| 215kB 19.9MB/s \n", "\u001b[?25hCollecting isort<5,>=4.2.5\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl (42kB)\n", "\u001b[K |████████████████████████████████| 51kB 9.1MB/s \n", "\u001b[?25hRequirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (0.16.0)\n", "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (3.1.1)\n", "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow>=1.14->joeynmt==0.0.1) (2.8.0)\n", "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (1.24.3)\n", "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (3.0.4)\n", "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2.8)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2019.11.28)\n", "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.15.2->seaborn->joeynmt==0.0.1) (2018.9)\n", "Collecting lazy-object-proxy==1.4.*\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/0b/dd/b1e3407e9e6913cf178e506cd0dee818e58694d9a5cd1984e3f6a8b9a10f/lazy_object_proxy-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (55kB)\n", "\u001b[K |████████████████████████████████| 61kB 9.3MB/s \n", "\u001b[?25hCollecting typed-ast<1.5,>=1.4.0; implementation_name == \"cpython\" and python_version < \"3.8\"\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/90/ed/5459080d95eb87a02fe860d447197be63b6e2b5e9ff73c2b0a85622994f4/typed_ast-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (737kB)\n", "\u001b[K |████████████████████████████████| 747kB 19.9MB/s \n", "\u001b[?25hBuilding wheels for collected packages: joeynmt, pyyaml\n", " Building wheel for joeynmt (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for joeynmt: filename=joeynmt-0.0.1-cp36-none-any.whl size=72136 sha256=bbe4d9c5390f6074d9a813a1690ae719d37cd77bbc8e1865105668a1862dac9a\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-1h4i3ayp/wheels/db/01/db/751cc9f3e7f6faec127c43644ba250a3ea7ad200594aeda70a\n", " Building wheel for pyyaml (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for pyyaml: filename=PyYAML-5.3-cp36-cp36m-linux_x86_64.whl size=44229 sha256=36c938900abf13262c7d1bab10af4ffe976a172a0b9de62ad4a123c307bb7417\n", " Stored in directory: /root/.cache/pip/wheels/e4/76/4d/a95b8dd7b452b69e8ed4f68b69e1b55e12c9c9624dd962b191\n", "Successfully built joeynmt pyyaml\n", "Installing collected packages: portalocker, sacrebleu, subword-nmt, pyyaml, mccabe, lazy-object-proxy, typed-ast, astroid, isort, pylint, joeynmt\n", " Found existing installation: PyYAML 3.13\n", " Uninstalling PyYAML-3.13:\n", " Successfully uninstalled PyYAML-3.13\n", "Successfully installed astroid-2.3.3 isort-4.3.21 joeynmt-0.0.1 lazy-object-proxy-1.4.3 mccabe-0.6.1 portalocker-1.5.2 pylint-2.4.4 pyyaml-5.3 sacrebleu-1.4.3 subword-nmt-0.3.7 typed-ast-1.4.1\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AaE77Tcppex9" }, "source": [ "# Preprocessing the Data into Subword BPE Tokens\n", "\n", "- One of the most powerful improvements for agglutinative languages (a feature of most Bantu languages) is using BPE tokenization [ (Sennrich, 2015) ](https://arxiv.org/abs/1508.07909).\n", "\n", "- It was also shown that by optimizing the umber of BPE codes we significantly improve results for low-resourced languages [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021) [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)\n", "\n", "- Below we have the scripts for doing BPE tokenization of our data. We use 4000 tokens as recommended by [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021). You do not need to change anything. Simply running the below will be suitable. " ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "H-TyjtmXB1mL", "outputId": "291628f4-f823-4ead-d7ec-3a887e954089", "colab": { "base_uri": "https://localhost:8080/", "height": 408 } }, "source": [ "# One of the huge boosts in NMT performance was to use a different method of tokenizing. \n", "# Usually, NMT would tokenize by words. However, using a method called BPE gave amazing boosts to performance\n", "\n", "# Do subword NMT\n", "from os import path\n", "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n", "os.environ[\"tgt\"] = target_language\n", "\n", "# Learn BPEs on the training data.\n", "os.environ[\"data_path\"] = path.join(\"joeynmt\", \"data\", source_language + target_language) # Herman! \n", "! subword-nmt learn-joint-bpe-and-vocab --input train.$src train.$tgt -s 4000 -o bpe.codes.4000 --write-vocabulary vocab.$src vocab.$tgt\n", "\n", "# Apply BPE splits to the development and test data.\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < train.$src > train.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < train.$tgt > train.bpe.$tgt\n", "\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < dev.$src > dev.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < dev.$tgt > dev.bpe.$tgt\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < test.$src > test.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt\n", "\n", "# Create directory, move everyone we care about to the correct location\n", "! mkdir -p $data_path\n", "! cp train.* $data_path\n", "! cp test.* $data_path\n", "! cp dev.* $data_path\n", "! cp bpe.codes.4000 $data_path\n", "! ls $data_path\n", "\n", "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n", "! cp train.* \"$gdrive_path\"\n", "! cp test.* \"$gdrive_path\"\n", "! cp dev.* \"$gdrive_path\"\n", "! cp bpe.codes.4000 \"$gdrive_path\"\n", "! ls \"$gdrive_path\"\n", "\n", "# Create that vocab using build_vocab\n", "! sudo chmod 777 joeynmt/scripts/build_vocab.py\n", "! joeynmt/scripts/build_vocab.py joeynmt/data/$src$tgt/train.bpe.$src joeynmt/data/$src$tgt/train.bpe.$tgt --output_path joeynmt/data/$src$tgt/vocab.txt\n", "\n", "# Some output\n", "! echo \"BPE Isoko Sentences\"\n", "! tail -n 5 test.bpe.$tgt\n", "! echo \"Combined BPE Vocab\"\n", "! tail -n 10 joeynmt/data/$src$tgt/vocab.txt # Herman\n", "! cp joeynmt/data/$src$tgt/vocab.txt \"$gdrive_path\"" ], "execution_count": 16, "outputs": [ { "output_type": "stream", "text": [ "bpe.codes.4000\tdev.en\t test.bpe.iso test.iso\t train.en\n", "dev.bpe.en\tdev.iso test.en\t train.bpe.en train.iso\n", "dev.bpe.iso\ttest.bpe.en test.en-any.en train.bpe.iso\n", "bpe.codes.4000\tdev.en\t test.bpe.iso test.iso\t train.en\n", "dev.bpe.en\tdev.iso test.en\t train.bpe.en train.iso\n", "dev.bpe.iso\ttest.bpe.en test.en-any.en train.bpe.iso\n", "BPE Isoko Sentences\n", "Fikieme o rọ gwọlọ nọ H@@ us@@ hai o re wo udu re ọ sae tal@@ amu Ọghẹnẹ ?\n", "Fikieme o rọ gwọlọ nọ ma wo udu re ma sai yoẹme kẹ Jihova ?\n", "Mẹ tẹ lẹ se Ọghẹnẹ re ọ kẹ omẹ udu nọ me re ro ru lele iroro nọ mẹ jẹ na .\n", "Enẹna eva mẹ e gbẹ be d@@ hae he , yọ mẹ sai weze bru ai oke kpobi . ” — Se Itẹ 29 : 25 .\n", "[ 1 ] ( edhe - ẹme avọ 7 ) Ma nwene edẹ na jọ .\n", "Combined BPE Vocab\n", "ş\n", "⁄\n", "=\n", "↓\n", "rint\n", "nwan@@\n", "Ó@@\n", "especi@@\n", "Prover@@\n", "ā\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "IlMitUHR8Qy-", "outputId": "a1f4e86b-e790-4ad3-bd6f-ebcaaa356a7e", "colab": { "base_uri": "https://localhost:8080/", "height": 68 } }, "source": [ "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n", "! cp train.* \"$gdrive_path\"\n", "! cp test.* \"$gdrive_path\"\n", "! cp dev.* \"$gdrive_path\"\n", "! cp bpe.codes.4000 \"$gdrive_path\"\n", "! ls \"$gdrive_path\"" ], "execution_count": 17, "outputs": [ { "output_type": "stream", "text": [ "bpe.codes.4000\tdev.en\t test.bpe.iso test.iso\t train.en\n", "dev.bpe.en\tdev.iso test.en\t train.bpe.en train.iso\n", "dev.bpe.iso\ttest.bpe.en test.en-any.en train.bpe.iso vocab.txt\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Ixmzi60WsUZ8" }, "source": [ "# Creating the JoeyNMT Config\n", "\n", "JoeyNMT requires a yaml config. We provide a template below. We've also set a number of defaults with it, that you may play with!\n", "\n", "- We used Transformer architecture \n", "- We set our dropout to reasonably high: 0.3 (recommended in [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021))\n", "\n", "Things worth playing with:\n", "- The batch size (also recommended to change for low-resourced languages)\n", "- The number of epochs (we've set it at 30 just so it runs in about an hour, for testing purposes)\n", "- The decoder options (beam_size, alpha)\n", "- Evaluation metrics (BLEU versus Crhf4)" ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "PIs1lY2hxMsl", "colab": {} }, "source": [ "# This creates the config file for our JoeyNMT system. It might seem overwhelming so we've provided a couple of useful parameters you'll need to update\n", "# (You can of course play with all the parameters if you'd like!)\n", "\n", "name = '%s%s' % (source_language, target_language)\n", "gdrive_path = os.environ[\"gdrive_path\"]\n", "\n", "# Create the config\n", "config = \"\"\"\n", "name: \"{name}_transformer\"\n", "\n", "data:\n", " src: \"{source_language}\"\n", " trg: \"{target_language}\"\n", " train: \"{gdrive_path}/train.bpe\"\n", " dev: \"{gdrive_path}/dev.bpe\"\n", " test: \"{gdrive_path}/test.bpe\"\n", " level: \"bpe\"\n", " lowercase: False\n", " max_sent_length: 100\n", " src_vocab: \"{gdrive_path}/vocab.txt\"\n", " trg_vocab: \"{gdrive_path}/vocab.txt\"\n", "\n", "testing:\n", " beam_size: 5\n", " alpha: 1.0\n", "\n", "training:\n", " load_model: \"{gdrive_path}/models/{name}_transformer_orig/142000.ckpt\" # if uncommented, load a pre-trained model from this checkpoint\n", " random_seed: 42\n", " optimizer: \"adam\"\n", " normalization: \"tokens\"\n", " adam_betas: [0.9, 0.999] \n", " scheduling: \"plateau\" # TODO: try switching from plateau to Noam scheduling\n", " patience: 5 # For plateau: decrease learning rate by decrease_factor if validation score has not improved for this many validation rounds.\n", " learning_rate_factor: 0.5 # factor for Noam scheduler (used with Transformer)\n", " learning_rate_warmup: 1000 # warmup steps for Noam scheduler (used with Transformer)\n", " decrease_factor: 0.7\n", " loss: \"crossentropy\"\n", " learning_rate: 0.0003\n", " learning_rate_min: 0.00000001\n", " weight_decay: 0.0\n", " label_smoothing: 0.1\n", " batch_size: 4096\n", " batch_type: \"token\"\n", " eval_batch_size: 3600\n", " eval_batch_type: \"token\"\n", " batch_multiplier: 1\n", " early_stopping_metric: \"ppl\"\n", " epochs: 45 # TODO: Decrease for when playing around and checking of working. Around 30 is sufficient to check if its working at all\n", " validation_freq: 1000 # TODO: Set to at least once per epoch.\n", " logging_freq: 100\n", " eval_metric: \"bleu\"\n", " model_dir: \"{gdrive_path}/models/{name}_transformer\"\n", " overwrite: True # TODO: Set to True if you want to overwrite possibly existing models. \n", " shuffle: True\n", " use_cuda: True\n", " max_output_length: 100\n", " print_valid_sents: [0, 1, 2, 3]\n", " keep_last_ckpts: 3\n", "\n", "model:\n", " initializer: \"xavier\"\n", " bias_initializer: \"zeros\"\n", " init_gain: 1.0\n", " embed_initializer: \"xavier\"\n", " embed_init_gain: 1.0\n", " tied_embeddings: True\n", " tied_softmax: True\n", " encoder:\n", " type: \"transformer\"\n", " num_layers: 6\n", " num_heads: 4 # TODO: Increase to 8 for larger data.\n", " embeddings:\n", " embedding_dim: 256 # TODO: Increase to 512 for larger data.\n", " scale: True\n", " dropout: 0.2\n", " # typically ff_size = 4 x hidden_size\n", " hidden_size: 256 # TODO: Increase to 512 for larger data.\n", " ff_size: 1024 # TODO: Increase to 2048 for larger data.\n", " dropout: 0.3\n", " decoder:\n", " type: \"transformer\"\n", " num_layers: 6\n", " num_heads: 4 # TODO: Increase to 8 for larger data.\n", " embeddings:\n", " embedding_dim: 256 # TODO: Increase to 512 for larger data.\n", " scale: True\n", " dropout: 0.2\n", " # typically ff_size = 4 x hidden_size\n", " hidden_size: 256 # TODO: Increase to 512 for larger data.\n", " ff_size: 1024 # TODO: Increase to 2048 for larger data.\n", " dropout: 0.3\n", "\"\"\".format(name=name, gdrive_path=os.environ[\"gdrive_path\"], source_language=source_language, target_language=target_language)\n", "with open(\"joeynmt/configs/transformer_{name}.yaml\".format(name=name),'w') as f:\n", " f.write(config)\n", "\n", "! cp joeynmt/configs/transformer_$src$tgt.yaml \"$gdrive_path\"" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "pIifxE3Qzuvs" }, "source": [ "# Train the Model\n", "\n", "This single line of joeynmt runs the training using the config we made above" ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "6ZBPFwT94WpI", "outputId": "9928b074-4f6b-48d4-add2-47b127069c83", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Train the model\n", "# You can press Ctrl-C to stop. And then run the next cell to save your checkpoints! \n", "# !cd joeynmt; python3 -m joeynmt train configs/transformer_$src$tgt.yaml\n", "!python3 -m joeynmt train \"$gdrive_path/transformer_$src$tgt.yaml\"" ], "execution_count": 9, "outputs": [ { "output_type": "stream", "text": [ "2020-01-17 22:07:05,330 Hello! This is Joey-NMT.\n", "2020-01-17 22:07:06,516 Total params: 12149248\n", "2020-01-17 22:07:06,517 Trainable parameters: ['decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.dec_layer_norm.bias', 'decoder.layers.0.dec_layer_norm.weight', 'decoder.layers.0.feed_forward.layer_norm.bias', 'decoder.layers.0.feed_forward.layer_norm.weight', 'decoder.layers.0.feed_forward.pwff_layer.0.bias', 'decoder.layers.0.feed_forward.pwff_layer.0.weight', 'decoder.layers.0.feed_forward.pwff_layer.3.bias', 'decoder.layers.0.feed_forward.pwff_layer.3.weight', 'decoder.layers.0.src_trg_att.k_layer.bias', 'decoder.layers.0.src_trg_att.k_layer.weight', 'decoder.layers.0.src_trg_att.output_layer.bias', 'decoder.layers.0.src_trg_att.output_layer.weight', 'decoder.layers.0.src_trg_att.q_layer.bias', 'decoder.layers.0.src_trg_att.q_layer.weight', 'decoder.layers.0.src_trg_att.v_layer.bias', 'decoder.layers.0.src_trg_att.v_layer.weight', 'decoder.layers.0.trg_trg_att.k_layer.bias', 'decoder.layers.0.trg_trg_att.k_layer.weight', 'decoder.layers.0.trg_trg_att.output_layer.bias', 'decoder.layers.0.trg_trg_att.output_layer.weight', 'decoder.layers.0.trg_trg_att.q_layer.bias', 'decoder.layers.0.trg_trg_att.q_layer.weight', 'decoder.layers.0.trg_trg_att.v_layer.bias', 'decoder.layers.0.trg_trg_att.v_layer.weight', 'decoder.layers.0.x_layer_norm.bias', 'decoder.layers.0.x_layer_norm.weight', 'decoder.layers.1.dec_layer_norm.bias', 'decoder.layers.1.dec_layer_norm.weight', 'decoder.layers.1.feed_forward.layer_norm.bias', 'decoder.layers.1.feed_forward.layer_norm.weight', 'decoder.layers.1.feed_forward.pwff_layer.0.bias', 'decoder.layers.1.feed_forward.pwff_layer.0.weight', 'decoder.layers.1.feed_forward.pwff_layer.3.bias', 'decoder.layers.1.feed_forward.pwff_layer.3.weight', 'decoder.layers.1.src_trg_att.k_layer.bias', 'decoder.layers.1.src_trg_att.k_layer.weight', 'decoder.layers.1.src_trg_att.output_layer.bias', 'decoder.layers.1.src_trg_att.output_layer.weight', 'decoder.layers.1.src_trg_att.q_layer.bias', 'decoder.layers.1.src_trg_att.q_layer.weight', 'decoder.layers.1.src_trg_att.v_layer.bias', 'decoder.layers.1.src_trg_att.v_layer.weight', 'decoder.layers.1.trg_trg_att.k_layer.bias', 'decoder.layers.1.trg_trg_att.k_layer.weight', 'decoder.layers.1.trg_trg_att.output_layer.bias', 'decoder.layers.1.trg_trg_att.output_layer.weight', 'decoder.layers.1.trg_trg_att.q_layer.bias', 'decoder.layers.1.trg_trg_att.q_layer.weight', 'decoder.layers.1.trg_trg_att.v_layer.bias', 'decoder.layers.1.trg_trg_att.v_layer.weight', 'decoder.layers.1.x_layer_norm.bias', 'decoder.layers.1.x_layer_norm.weight', 'decoder.layers.2.dec_layer_norm.bias', 'decoder.layers.2.dec_layer_norm.weight', 'decoder.layers.2.feed_forward.layer_norm.bias', 'decoder.layers.2.feed_forward.layer_norm.weight', 'decoder.layers.2.feed_forward.pwff_layer.0.bias', 'decoder.layers.2.feed_forward.pwff_layer.0.weight', 'decoder.layers.2.feed_forward.pwff_layer.3.bias', 'decoder.layers.2.feed_forward.pwff_layer.3.weight', 'decoder.layers.2.src_trg_att.k_layer.bias', 'decoder.layers.2.src_trg_att.k_layer.weight', 'decoder.layers.2.src_trg_att.output_layer.bias', 'decoder.layers.2.src_trg_att.output_layer.weight', 'decoder.layers.2.src_trg_att.q_layer.bias', 'decoder.layers.2.src_trg_att.q_layer.weight', 'decoder.layers.2.src_trg_att.v_layer.bias', 'decoder.layers.2.src_trg_att.v_layer.weight', 'decoder.layers.2.trg_trg_att.k_layer.bias', 'decoder.layers.2.trg_trg_att.k_layer.weight', 'decoder.layers.2.trg_trg_att.output_layer.bias', 'decoder.layers.2.trg_trg_att.output_layer.weight', 'decoder.layers.2.trg_trg_att.q_layer.bias', 'decoder.layers.2.trg_trg_att.q_layer.weight', 'decoder.layers.2.trg_trg_att.v_layer.bias', 'decoder.layers.2.trg_trg_att.v_layer.weight', 'decoder.layers.2.x_layer_norm.bias', 'decoder.layers.2.x_layer_norm.weight', 'decoder.layers.3.dec_layer_norm.bias', 'decoder.layers.3.dec_layer_norm.weight', 'decoder.layers.3.feed_forward.layer_norm.bias', 'decoder.layers.3.feed_forward.layer_norm.weight', 'decoder.layers.3.feed_forward.pwff_layer.0.bias', 'decoder.layers.3.feed_forward.pwff_layer.0.weight', 'decoder.layers.3.feed_forward.pwff_layer.3.bias', 'decoder.layers.3.feed_forward.pwff_layer.3.weight', 'decoder.layers.3.src_trg_att.k_layer.bias', 'decoder.layers.3.src_trg_att.k_layer.weight', 'decoder.layers.3.src_trg_att.output_layer.bias', 'decoder.layers.3.src_trg_att.output_layer.weight', 'decoder.layers.3.src_trg_att.q_layer.bias', 'decoder.layers.3.src_trg_att.q_layer.weight', 'decoder.layers.3.src_trg_att.v_layer.bias', 'decoder.layers.3.src_trg_att.v_layer.weight', 'decoder.layers.3.trg_trg_att.k_layer.bias', 'decoder.layers.3.trg_trg_att.k_layer.weight', 'decoder.layers.3.trg_trg_att.output_layer.bias', 'decoder.layers.3.trg_trg_att.output_layer.weight', 'decoder.layers.3.trg_trg_att.q_layer.bias', 'decoder.layers.3.trg_trg_att.q_layer.weight', 'decoder.layers.3.trg_trg_att.v_layer.bias', 'decoder.layers.3.trg_trg_att.v_layer.weight', 'decoder.layers.3.x_layer_norm.bias', 'decoder.layers.3.x_layer_norm.weight', 'decoder.layers.4.dec_layer_norm.bias', 'decoder.layers.4.dec_layer_norm.weight', 'decoder.layers.4.feed_forward.layer_norm.bias', 'decoder.layers.4.feed_forward.layer_norm.weight', 'decoder.layers.4.feed_forward.pwff_layer.0.bias', 'decoder.layers.4.feed_forward.pwff_layer.0.weight', 'decoder.layers.4.feed_forward.pwff_layer.3.bias', 'decoder.layers.4.feed_forward.pwff_layer.3.weight', 'decoder.layers.4.src_trg_att.k_layer.bias', 'decoder.layers.4.src_trg_att.k_layer.weight', 'decoder.layers.4.src_trg_att.output_layer.bias', 'decoder.layers.4.src_trg_att.output_layer.weight', 'decoder.layers.4.src_trg_att.q_layer.bias', 'decoder.layers.4.src_trg_att.q_layer.weight', 'decoder.layers.4.src_trg_att.v_layer.bias', 'decoder.layers.4.src_trg_att.v_layer.weight', 'decoder.layers.4.trg_trg_att.k_layer.bias', 'decoder.layers.4.trg_trg_att.k_layer.weight', 'decoder.layers.4.trg_trg_att.output_layer.bias', 'decoder.layers.4.trg_trg_att.output_layer.weight', 'decoder.layers.4.trg_trg_att.q_layer.bias', 'decoder.layers.4.trg_trg_att.q_layer.weight', 'decoder.layers.4.trg_trg_att.v_layer.bias', 'decoder.layers.4.trg_trg_att.v_layer.weight', 'decoder.layers.4.x_layer_norm.bias', 'decoder.layers.4.x_layer_norm.weight', 'decoder.layers.5.dec_layer_norm.bias', 'decoder.layers.5.dec_layer_norm.weight', 'decoder.layers.5.feed_forward.layer_norm.bias', 'decoder.layers.5.feed_forward.layer_norm.weight', 'decoder.layers.5.feed_forward.pwff_layer.0.bias', 'decoder.layers.5.feed_forward.pwff_layer.0.weight', 'decoder.layers.5.feed_forward.pwff_layer.3.bias', 'decoder.layers.5.feed_forward.pwff_layer.3.weight', 'decoder.layers.5.src_trg_att.k_layer.bias', 'decoder.layers.5.src_trg_att.k_layer.weight', 'decoder.layers.5.src_trg_att.output_layer.bias', 'decoder.layers.5.src_trg_att.output_layer.weight', 'decoder.layers.5.src_trg_att.q_layer.bias', 'decoder.layers.5.src_trg_att.q_layer.weight', 'decoder.layers.5.src_trg_att.v_layer.bias', 'decoder.layers.5.src_trg_att.v_layer.weight', 'decoder.layers.5.trg_trg_att.k_layer.bias', 'decoder.layers.5.trg_trg_att.k_layer.weight', 'decoder.layers.5.trg_trg_att.output_layer.bias', 'decoder.layers.5.trg_trg_att.output_layer.weight', 'decoder.layers.5.trg_trg_att.q_layer.bias', 'decoder.layers.5.trg_trg_att.q_layer.weight', 'decoder.layers.5.trg_trg_att.v_layer.bias', 'decoder.layers.5.trg_trg_att.v_layer.weight', 'decoder.layers.5.x_layer_norm.bias', 'decoder.layers.5.x_layer_norm.weight', 'encoder.layer_norm.bias', 'encoder.layer_norm.weight', 'encoder.layers.0.feed_forward.layer_norm.bias', 'encoder.layers.0.feed_forward.layer_norm.weight', 'encoder.layers.0.feed_forward.pwff_layer.0.bias', 'encoder.layers.0.feed_forward.pwff_layer.0.weight', 'encoder.layers.0.feed_forward.pwff_layer.3.bias', 'encoder.layers.0.feed_forward.pwff_layer.3.weight', 'encoder.layers.0.layer_norm.bias', 'encoder.layers.0.layer_norm.weight', 'encoder.layers.0.src_src_att.k_layer.bias', 'encoder.layers.0.src_src_att.k_layer.weight', 'encoder.layers.0.src_src_att.output_layer.bias', 'encoder.layers.0.src_src_att.output_layer.weight', 'encoder.layers.0.src_src_att.q_layer.bias', 'encoder.layers.0.src_src_att.q_layer.weight', 'encoder.layers.0.src_src_att.v_layer.bias', 'encoder.layers.0.src_src_att.v_layer.weight', 'encoder.layers.1.feed_forward.layer_norm.bias', 'encoder.layers.1.feed_forward.layer_norm.weight', 'encoder.layers.1.feed_forward.pwff_layer.0.bias', 'encoder.layers.1.feed_forward.pwff_layer.0.weight', 'encoder.layers.1.feed_forward.pwff_layer.3.bias', 'encoder.layers.1.feed_forward.pwff_layer.3.weight', 'encoder.layers.1.layer_norm.bias', 'encoder.layers.1.layer_norm.weight', 'encoder.layers.1.src_src_att.k_layer.bias', 'encoder.layers.1.src_src_att.k_layer.weight', 'encoder.layers.1.src_src_att.output_layer.bias', 'encoder.layers.1.src_src_att.output_layer.weight', 'encoder.layers.1.src_src_att.q_layer.bias', 'encoder.layers.1.src_src_att.q_layer.weight', 'encoder.layers.1.src_src_att.v_layer.bias', 'encoder.layers.1.src_src_att.v_layer.weight', 'encoder.layers.2.feed_forward.layer_norm.bias', 'encoder.layers.2.feed_forward.layer_norm.weight', 'encoder.layers.2.feed_forward.pwff_layer.0.bias', 'encoder.layers.2.feed_forward.pwff_layer.0.weight', 'encoder.layers.2.feed_forward.pwff_layer.3.bias', 'encoder.layers.2.feed_forward.pwff_layer.3.weight', 'encoder.layers.2.layer_norm.bias', 'encoder.layers.2.layer_norm.weight', 'encoder.layers.2.src_src_att.k_layer.bias', 'encoder.layers.2.src_src_att.k_layer.weight', 'encoder.layers.2.src_src_att.output_layer.bias', 'encoder.layers.2.src_src_att.output_layer.weight', 'encoder.layers.2.src_src_att.q_layer.bias', 'encoder.layers.2.src_src_att.q_layer.weight', 'encoder.layers.2.src_src_att.v_layer.bias', 'encoder.layers.2.src_src_att.v_layer.weight', 'encoder.layers.3.feed_forward.layer_norm.bias', 'encoder.layers.3.feed_forward.layer_norm.weight', 'encoder.layers.3.feed_forward.pwff_layer.0.bias', 'encoder.layers.3.feed_forward.pwff_layer.0.weight', 'encoder.layers.3.feed_forward.pwff_layer.3.bias', 'encoder.layers.3.feed_forward.pwff_layer.3.weight', 'encoder.layers.3.layer_norm.bias', 'encoder.layers.3.layer_norm.weight', 'encoder.layers.3.src_src_att.k_layer.bias', 'encoder.layers.3.src_src_att.k_layer.weight', 'encoder.layers.3.src_src_att.output_layer.bias', 'encoder.layers.3.src_src_att.output_layer.weight', 'encoder.layers.3.src_src_att.q_layer.bias', 'encoder.layers.3.src_src_att.q_layer.weight', 'encoder.layers.3.src_src_att.v_layer.bias', 'encoder.layers.3.src_src_att.v_layer.weight', 'encoder.layers.4.feed_forward.layer_norm.bias', 'encoder.layers.4.feed_forward.layer_norm.weight', 'encoder.layers.4.feed_forward.pwff_layer.0.bias', 'encoder.layers.4.feed_forward.pwff_layer.0.weight', 'encoder.layers.4.feed_forward.pwff_layer.3.bias', 'encoder.layers.4.feed_forward.pwff_layer.3.weight', 'encoder.layers.4.layer_norm.bias', 'encoder.layers.4.layer_norm.weight', 'encoder.layers.4.src_src_att.k_layer.bias', 'encoder.layers.4.src_src_att.k_layer.weight', 'encoder.layers.4.src_src_att.output_layer.bias', 'encoder.layers.4.src_src_att.output_layer.weight', 'encoder.layers.4.src_src_att.q_layer.bias', 'encoder.layers.4.src_src_att.q_layer.weight', 'encoder.layers.4.src_src_att.v_layer.bias', 'encoder.layers.4.src_src_att.v_layer.weight', 'encoder.layers.5.feed_forward.layer_norm.bias', 'encoder.layers.5.feed_forward.layer_norm.weight', 'encoder.layers.5.feed_forward.pwff_layer.0.bias', 'encoder.layers.5.feed_forward.pwff_layer.0.weight', 'encoder.layers.5.feed_forward.pwff_layer.3.bias', 'encoder.layers.5.feed_forward.pwff_layer.3.weight', 'encoder.layers.5.layer_norm.bias', 'encoder.layers.5.layer_norm.weight', 'encoder.layers.5.src_src_att.k_layer.bias', 'encoder.layers.5.src_src_att.k_layer.weight', 'encoder.layers.5.src_src_att.output_layer.bias', 'encoder.layers.5.src_src_att.output_layer.weight', 'encoder.layers.5.src_src_att.q_layer.bias', 'encoder.layers.5.src_src_att.q_layer.weight', 'encoder.layers.5.src_src_att.v_layer.bias', 'encoder.layers.5.src_src_att.v_layer.weight', 'src_embed.lut.weight']\n", "2020-01-17 22:07:09,609 Loading model from /content/drive/My Drive/masakhane/en-iso-baseline/models/eniso_transformer_orig/142000.ckpt\n", "2020-01-17 22:07:09,921 cfg.name : eniso_transformer\n", "2020-01-17 22:07:09,921 cfg.data.src : en\n", "2020-01-17 22:07:09,922 cfg.data.trg : iso\n", "2020-01-17 22:07:09,922 cfg.data.train : /content/drive/My Drive/masakhane/en-iso-baseline/train.bpe\n", "2020-01-17 22:07:09,922 cfg.data.dev : /content/drive/My Drive/masakhane/en-iso-baseline/dev.bpe\n", "2020-01-17 22:07:09,922 cfg.data.test : /content/drive/My Drive/masakhane/en-iso-baseline/test.bpe\n", "2020-01-17 22:07:09,922 cfg.data.level : bpe\n", "2020-01-17 22:07:09,922 cfg.data.lowercase : False\n", "2020-01-17 22:07:09,922 cfg.data.max_sent_length : 100\n", "2020-01-17 22:07:09,923 cfg.data.src_vocab : /content/drive/My Drive/masakhane/en-iso-baseline/vocab.txt\n", "2020-01-17 22:07:09,923 cfg.data.trg_vocab : /content/drive/My Drive/masakhane/en-iso-baseline/vocab.txt\n", "2020-01-17 22:07:09,923 cfg.testing.beam_size : 5\n", "2020-01-17 22:07:09,923 cfg.testing.alpha : 1.0\n", "2020-01-17 22:07:09,923 cfg.training.load_model : /content/drive/My Drive/masakhane/en-iso-baseline/models/eniso_transformer_orig/142000.ckpt\n", "2020-01-17 22:07:09,923 cfg.training.random_seed : 42\n", "2020-01-17 22:07:09,923 cfg.training.optimizer : adam\n", "2020-01-17 22:07:09,923 cfg.training.normalization : tokens\n", "2020-01-17 22:07:09,924 cfg.training.adam_betas : [0.9, 0.999]\n", "2020-01-17 22:07:09,924 cfg.training.scheduling : plateau\n", "2020-01-17 22:07:09,924 cfg.training.patience : 5\n", "2020-01-17 22:07:09,924 cfg.training.learning_rate_factor : 0.5\n", "2020-01-17 22:07:09,924 cfg.training.learning_rate_warmup : 1000\n", "2020-01-17 22:07:09,924 cfg.training.decrease_factor : 0.7\n", "2020-01-17 22:07:09,924 cfg.training.loss : crossentropy\n", "2020-01-17 22:07:09,924 cfg.training.learning_rate : 0.0003\n", "2020-01-17 22:07:09,925 cfg.training.learning_rate_min : 1e-08\n", "2020-01-17 22:07:09,925 cfg.training.weight_decay : 0.0\n", "2020-01-17 22:07:09,925 cfg.training.label_smoothing : 0.1\n", "2020-01-17 22:07:09,925 cfg.training.batch_size : 4096\n", "2020-01-17 22:07:09,925 cfg.training.batch_type : token\n", "2020-01-17 22:07:09,925 cfg.training.eval_batch_size : 3600\n", "2020-01-17 22:07:09,925 cfg.training.eval_batch_type : token\n", "2020-01-17 22:07:09,925 cfg.training.batch_multiplier : 1\n", "2020-01-17 22:07:09,926 cfg.training.early_stopping_metric : ppl\n", "2020-01-17 22:07:09,926 cfg.training.epochs : 45\n", "2020-01-17 22:07:09,926 cfg.training.validation_freq : 1000\n", "2020-01-17 22:07:09,926 cfg.training.logging_freq : 100\n", "2020-01-17 22:07:09,926 cfg.training.eval_metric : bleu\n", "2020-01-17 22:07:09,926 cfg.training.model_dir : /content/drive/My Drive/masakhane/en-iso-baseline/models/eniso_transformer\n", "2020-01-17 22:07:09,926 cfg.training.overwrite : True\n", "2020-01-17 22:07:09,926 cfg.training.shuffle : True\n", "2020-01-17 22:07:09,927 cfg.training.use_cuda : True\n", "2020-01-17 22:07:09,927 cfg.training.max_output_length : 100\n", "2020-01-17 22:07:09,927 cfg.training.print_valid_sents : [0, 1, 2, 3]\n", "2020-01-17 22:07:09,927 cfg.training.keep_last_ckpts : 3\n", "2020-01-17 22:07:09,927 cfg.model.initializer : xavier\n", "2020-01-17 22:07:09,927 cfg.model.bias_initializer : zeros\n", "2020-01-17 22:07:09,927 cfg.model.init_gain : 1.0\n", "2020-01-17 22:07:09,927 cfg.model.embed_initializer : xavier\n", "2020-01-17 22:07:09,928 cfg.model.embed_init_gain : 1.0\n", "2020-01-17 22:07:09,928 cfg.model.tied_embeddings : True\n", "2020-01-17 22:07:09,928 cfg.model.tied_softmax : True\n", "2020-01-17 22:07:09,928 cfg.model.encoder.type : transformer\n", "2020-01-17 22:07:09,928 cfg.model.encoder.num_layers : 6\n", "2020-01-17 22:07:09,928 cfg.model.encoder.num_heads : 4\n", "2020-01-17 22:07:09,928 cfg.model.encoder.embeddings.embedding_dim : 256\n", "2020-01-17 22:07:09,928 cfg.model.encoder.embeddings.scale : True\n", "2020-01-17 22:07:09,929 cfg.model.encoder.embeddings.dropout : 0.2\n", "2020-01-17 22:07:09,929 cfg.model.encoder.hidden_size : 256\n", "2020-01-17 22:07:09,929 cfg.model.encoder.ff_size : 1024\n", "2020-01-17 22:07:09,929 cfg.model.encoder.dropout : 0.3\n", "2020-01-17 22:07:09,929 cfg.model.decoder.type : transformer\n", "2020-01-17 22:07:09,929 cfg.model.decoder.num_layers : 6\n", "2020-01-17 22:07:09,929 cfg.model.decoder.num_heads : 4\n", "2020-01-17 22:07:09,929 cfg.model.decoder.embeddings.embedding_dim : 256\n", "2020-01-17 22:07:09,930 cfg.model.decoder.embeddings.scale : True\n", "2020-01-17 22:07:09,930 cfg.model.decoder.embeddings.dropout : 0.2\n", "2020-01-17 22:07:09,930 cfg.model.decoder.hidden_size : 256\n", "2020-01-17 22:07:09,930 cfg.model.decoder.ff_size : 1024\n", "2020-01-17 22:07:09,930 cfg.model.decoder.dropout : 0.3\n", "2020-01-17 22:07:09,930 Data set sizes: \n", "\ttrain 214464,\n", "\tvalid 1000,\n", "\ttest 2709\n", "2020-01-17 22:07:09,930 First training example:\n", "\t[SRC] Now in his 8@@ 0 ’ s , John adm@@ its that at times he feel@@ s des@@ p@@ on@@ d@@ ent .\n", "\t[TRG] Enẹna Jọn ọ kpako te ikpe udh@@ one gb@@ ọ no , ọ ta nọ ẹsejọ ọ be hai wo el@@ ọ@@ h@@ oma .\n", "2020-01-17 22:07:09,930 First 10 words (src): (0) (1) (2) (3) (4) . (5) , (6) nọ (7) a (8) the (9) na\n", "2020-01-17 22:07:09,931 First 10 words (trg): (0) (1) (2) (3) (4) . (5) , (6) nọ (7) a (8) the (9) na\n", "2020-01-17 22:07:09,931 Number of Src words (types): 4254\n", "2020-01-17 22:07:09,931 Number of Trg words (types): 4254\n", "2020-01-17 22:07:09,931 Model(\n", "\tencoder=TransformerEncoder(num_layers=6, num_heads=4),\n", "\tdecoder=TransformerDecoder(num_layers=6, num_heads=4),\n", "\tsrc_embed=Embeddings(embedding_dim=256, vocab_size=4254),\n", "\ttrg_embed=Embeddings(embedding_dim=256, vocab_size=4254))\n", "2020-01-17 22:07:09,940 EPOCH 1\n", "2020-01-17 22:07:21,758 Epoch 1 Step: 142100 Batch Loss: 0.740027 Tokens per Sec: 18651, Lr: 0.000210\n", "2020-01-17 22:07:33,449 Epoch 1 Step: 142200 Batch Loss: 1.261251 Tokens per Sec: 19317, Lr: 0.000210\n", "2020-01-17 22:07:45,117 Epoch 1 Step: 142300 Batch Loss: 1.285557 Tokens per Sec: 19041, Lr: 0.000210\n", "2020-01-17 22:07:56,724 Epoch 1 Step: 142400 Batch Loss: 1.241936 Tokens per Sec: 19614, Lr: 0.000210\n", "2020-01-17 22:08:08,452 Epoch 1 Step: 142500 Batch Loss: 1.208397 Tokens per Sec: 19052, Lr: 0.000210\n", "2020-01-17 22:08:20,076 Epoch 1 Step: 142600 Batch Loss: 1.229701 Tokens per Sec: 19210, Lr: 0.000210\n", "2020-01-17 22:08:31,774 Epoch 1 Step: 142700 Batch Loss: 1.224977 Tokens per Sec: 19437, Lr: 0.000210\n", "2020-01-17 22:08:43,545 Epoch 1 Step: 142800 Batch Loss: 0.831322 Tokens per Sec: 19106, Lr: 0.000210\n", "2020-01-17 22:08:55,201 Epoch 1 Step: 142900 Batch Loss: 1.163271 Tokens per Sec: 19195, Lr: 0.000210\n", "2020-01-17 22:09:06,845 Epoch 1 Step: 143000 Batch Loss: 1.473182 Tokens per Sec: 19381, Lr: 0.000210\n", "2020-01-17 22:09:36,190 Example #0\n", "2020-01-17 22:09:36,191 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:09:36,191 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:09:36,191 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ udu efuafo . ’\n", "2020-01-17 22:09:36,191 Example #1\n", "2020-01-17 22:09:36,192 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:09:36,192 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:09:36,192 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 22:09:36,192 Example #2\n", "2020-01-17 22:09:36,193 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:09:36,193 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:09:36,193 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ abọ ekuhọ ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho na kpobi i wo oghale ; jọ a jiri ei .\n", "2020-01-17 22:09:36,193 Example #3\n", "2020-01-17 22:09:36,194 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:09:36,194 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:09:36,194 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-17 22:09:36,194 Validation result (greedy) at epoch 1, step 143000: bleu: 31.12, loss: 33366.6094, ppl: 3.6221, duration: 29.3486s\n", "2020-01-17 22:09:47,701 Epoch 1 Step: 143100 Batch Loss: 1.545534 Tokens per Sec: 18988, Lr: 0.000210\n", "2020-01-17 22:09:59,224 Epoch 1 Step: 143200 Batch Loss: 1.318172 Tokens per Sec: 18841, Lr: 0.000210\n", "2020-01-17 22:10:10,849 Epoch 1 Step: 143300 Batch Loss: 1.496062 Tokens per Sec: 19190, Lr: 0.000210\n", "2020-01-17 22:10:22,465 Epoch 1 Step: 143400 Batch Loss: 1.156595 Tokens per Sec: 19038, Lr: 0.000210\n", "2020-01-17 22:10:34,011 Epoch 1 Step: 143500 Batch Loss: 1.254312 Tokens per Sec: 19227, Lr: 0.000210\n", "2020-01-17 22:10:45,718 Epoch 1 Step: 143600 Batch Loss: 1.043414 Tokens per Sec: 19175, Lr: 0.000210\n", "2020-01-17 22:10:57,396 Epoch 1 Step: 143700 Batch Loss: 1.398818 Tokens per Sec: 19155, Lr: 0.000210\n", "2020-01-17 22:11:09,146 Epoch 1 Step: 143800 Batch Loss: 1.293357 Tokens per Sec: 18857, Lr: 0.000210\n", "2020-01-17 22:11:20,777 Epoch 1 Step: 143900 Batch Loss: 1.463160 Tokens per Sec: 19401, Lr: 0.000210\n", "2020-01-17 22:11:32,286 Epoch 1 Step: 144000 Batch Loss: 0.819408 Tokens per Sec: 19124, Lr: 0.000210\n", "2020-01-17 22:12:01,434 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:12:01,434 Saving new checkpoint.\n", "2020-01-17 22:12:02,624 Example #0\n", "2020-01-17 22:12:02,625 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:12:02,625 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:12:02,625 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:12:02,625 Example #1\n", "2020-01-17 22:12:02,626 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:12:02,626 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:12:02,626 \tHypothesis: A te lele ovuẹ Uvie na , Ijo .\n", "2020-01-17 22:12:02,626 Example #2\n", "2020-01-17 22:12:02,627 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:12:02,627 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:12:02,627 \tHypothesis: O wọhọ ẹsenọ mai kpobi ma be rọ evawere fiba ẹjiroro mai na rite abọ urere ọrọ abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo ẹvi , re o wo oghale , re erẹwho na kpobi i wo oghale ; jọ a wo oghale .\n", "2020-01-17 22:12:02,627 Example #3\n", "2020-01-17 22:12:02,628 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:12:02,628 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:12:02,628 \tHypothesis: Ghele na , eme ọ unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:12:02,628 Validation result (greedy) at epoch 1, step 144000: bleu: 30.61, loss: 33235.2031, ppl: 3.6038, duration: 30.3421s\n", "2020-01-17 22:12:14,660 Epoch 1 Step: 144100 Batch Loss: 1.123903 Tokens per Sec: 18581, Lr: 0.000210\n", "2020-01-17 22:12:26,247 Epoch 1 Step: 144200 Batch Loss: 1.263787 Tokens per Sec: 18826, Lr: 0.000210\n", "2020-01-17 22:12:37,922 Epoch 1 Step: 144300 Batch Loss: 1.407585 Tokens per Sec: 19289, Lr: 0.000210\n", "2020-01-17 22:12:49,562 Epoch 1 Step: 144400 Batch Loss: 1.309245 Tokens per Sec: 18992, Lr: 0.000210\n", "2020-01-17 22:13:01,129 Epoch 1 Step: 144500 Batch Loss: 1.313475 Tokens per Sec: 19335, Lr: 0.000210\n", "2020-01-17 22:13:03,604 Epoch 1: total training loss 3163.01\n", "2020-01-17 22:13:03,605 EPOCH 2\n", "2020-01-17 22:13:13,004 Epoch 2 Step: 144600 Batch Loss: 1.203302 Tokens per Sec: 18533, Lr: 0.000210\n", "2020-01-17 22:13:24,627 Epoch 2 Step: 144700 Batch Loss: 1.182161 Tokens per Sec: 18953, Lr: 0.000210\n", "2020-01-17 22:13:36,125 Epoch 2 Step: 144800 Batch Loss: 1.340899 Tokens per Sec: 18953, Lr: 0.000210\n", "2020-01-17 22:13:47,682 Epoch 2 Step: 144900 Batch Loss: 1.135138 Tokens per Sec: 19448, Lr: 0.000210\n", "2020-01-17 22:13:59,275 Epoch 2 Step: 145000 Batch Loss: 1.208559 Tokens per Sec: 19351, Lr: 0.000210\n", "2020-01-17 22:14:28,496 Example #0\n", "2020-01-17 22:14:28,497 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:14:28,497 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:14:28,497 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:14:28,497 Example #1\n", "2020-01-17 22:14:28,498 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:14:28,498 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:14:28,498 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 22:14:28,498 Example #2\n", "2020-01-17 22:14:28,498 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:14:28,499 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:14:28,499 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o jọ o jọ rọ kẹ ae evawere , re erẹwho kpobi i wo oghale .\n", "2020-01-17 22:14:28,499 Example #3\n", "2020-01-17 22:14:28,499 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:14:28,499 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:14:28,499 \tHypothesis: Ghele na , eme ọ eghrorotha yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:14:28,500 Validation result (greedy) at epoch 2, step 145000: bleu: 31.31, loss: 33330.0078, ppl: 3.6170, duration: 29.2246s\n", "2020-01-17 22:14:39,956 Epoch 2 Step: 145100 Batch Loss: 1.171901 Tokens per Sec: 18986, Lr: 0.000210\n", "2020-01-17 22:14:51,528 Epoch 2 Step: 145200 Batch Loss: 1.381863 Tokens per Sec: 19328, Lr: 0.000210\n", "2020-01-17 22:15:03,134 Epoch 2 Step: 145300 Batch Loss: 1.183814 Tokens per Sec: 19241, Lr: 0.000210\n", "2020-01-17 22:15:14,797 Epoch 2 Step: 145400 Batch Loss: 1.219280 Tokens per Sec: 18756, Lr: 0.000210\n", "2020-01-17 22:15:26,416 Epoch 2 Step: 145500 Batch Loss: 1.017421 Tokens per Sec: 19531, Lr: 0.000210\n", "2020-01-17 22:15:37,940 Epoch 2 Step: 145600 Batch Loss: 1.095073 Tokens per Sec: 18745, Lr: 0.000210\n", "2020-01-17 22:15:49,526 Epoch 2 Step: 145700 Batch Loss: 1.256271 Tokens per Sec: 19490, Lr: 0.000210\n", "2020-01-17 22:16:01,098 Epoch 2 Step: 145800 Batch Loss: 1.356281 Tokens per Sec: 19633, Lr: 0.000210\n", "2020-01-17 22:16:12,835 Epoch 2 Step: 145900 Batch Loss: 1.158789 Tokens per Sec: 19236, Lr: 0.000210\n", "2020-01-17 22:16:24,384 Epoch 2 Step: 146000 Batch Loss: 1.405467 Tokens per Sec: 19387, Lr: 0.000210\n", "2020-01-17 22:16:53,591 Example #0\n", "2020-01-17 22:16:53,592 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:16:53,592 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:16:53,592 \tHypothesis: Ma rẹ sae tubẹ lẹ Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:16:53,592 Example #1\n", "2020-01-17 22:16:53,593 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:16:53,593 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:16:53,593 \tHypothesis: A te lele onana , Usi Uvie na O Kiehọ .\n", "2020-01-17 22:16:53,593 Example #2\n", "2020-01-17 22:16:53,594 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:16:53,594 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:16:53,594 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re o jọ eva riẹ , re o jọ erẹwho na kpobi e jiri ei .\n", "2020-01-17 22:16:53,594 Example #3\n", "2020-01-17 22:16:53,594 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:16:53,594 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:16:53,595 \tHypothesis: Ghele na , eme nọ a re wou unu yọ ẹgba nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:16:53,595 Validation result (greedy) at epoch 2, step 146000: bleu: 31.12, loss: 33293.1523, ppl: 3.6118, duration: 29.2099s\n", "2020-01-17 22:17:05,059 Epoch 2 Step: 146100 Batch Loss: 0.973998 Tokens per Sec: 19366, Lr: 0.000210\n", "2020-01-17 22:17:16,723 Epoch 2 Step: 146200 Batch Loss: 1.249921 Tokens per Sec: 18850, Lr: 0.000210\n", "2020-01-17 22:17:28,299 Epoch 2 Step: 146300 Batch Loss: 1.189584 Tokens per Sec: 19458, Lr: 0.000210\n", "2020-01-17 22:17:39,893 Epoch 2 Step: 146400 Batch Loss: 1.337122 Tokens per Sec: 19370, Lr: 0.000210\n", "2020-01-17 22:17:51,508 Epoch 2 Step: 146500 Batch Loss: 1.456862 Tokens per Sec: 19438, Lr: 0.000210\n", "2020-01-17 22:18:02,993 Epoch 2 Step: 146600 Batch Loss: 1.097145 Tokens per Sec: 19304, Lr: 0.000210\n", "2020-01-17 22:18:14,611 Epoch 2 Step: 146700 Batch Loss: 1.299451 Tokens per Sec: 18433, Lr: 0.000210\n", "2020-01-17 22:18:26,369 Epoch 2 Step: 146800 Batch Loss: 1.191642 Tokens per Sec: 19439, Lr: 0.000210\n", "2020-01-17 22:18:37,869 Epoch 2 Step: 146900 Batch Loss: 1.445185 Tokens per Sec: 19160, Lr: 0.000210\n", "2020-01-17 22:18:49,512 Epoch 2 Step: 147000 Batch Loss: 1.175029 Tokens per Sec: 19356, Lr: 0.000210\n", "2020-01-17 22:19:18,684 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:19:18,684 Saving new checkpoint.\n", "2020-01-17 22:19:19,914 Example #0\n", "2020-01-17 22:19:19,917 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:19:19,918 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:19:19,918 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:19:19,918 Example #1\n", "2020-01-17 22:19:19,919 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:19:19,919 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:19:19,919 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-17 22:19:19,920 Example #2\n", "2020-01-17 22:19:19,920 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:19:19,921 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:19:19,921 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba izou mai te abọ urere ọrọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Odẹ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oruaro , re ọ jẹ ghale ae ; jọ o jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-17 22:19:19,921 Example #3\n", "2020-01-17 22:19:19,921 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:19:19,922 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:19:19,922 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 22:19:19,922 Validation result (greedy) at epoch 2, step 147000: bleu: 31.45, loss: 33180.6016, ppl: 3.5962, duration: 30.4088s\n", "2020-01-17 22:19:25,397 Epoch 2: total training loss 3160.05\n", "2020-01-17 22:19:25,397 EPOCH 3\n", "2020-01-17 22:19:31,998 Epoch 3 Step: 147100 Batch Loss: 1.259542 Tokens per Sec: 17844, Lr: 0.000210\n", "2020-01-17 22:19:43,443 Epoch 3 Step: 147200 Batch Loss: 0.749096 Tokens per Sec: 19149, Lr: 0.000210\n", "2020-01-17 22:19:55,036 Epoch 3 Step: 147300 Batch Loss: 1.396860 Tokens per Sec: 19522, Lr: 0.000210\n", "2020-01-17 22:20:06,574 Epoch 3 Step: 147400 Batch Loss: 1.138937 Tokens per Sec: 19393, Lr: 0.000210\n", "2020-01-17 22:20:18,248 Epoch 3 Step: 147500 Batch Loss: 1.192661 Tokens per Sec: 19443, Lr: 0.000210\n", "2020-01-17 22:20:29,863 Epoch 3 Step: 147600 Batch Loss: 0.940397 Tokens per Sec: 18982, Lr: 0.000210\n", "2020-01-17 22:20:41,522 Epoch 3 Step: 147700 Batch Loss: 0.892353 Tokens per Sec: 19512, Lr: 0.000210\n", "2020-01-17 22:20:53,119 Epoch 3 Step: 147800 Batch Loss: 1.388203 Tokens per Sec: 19390, Lr: 0.000210\n", "2020-01-17 22:21:04,609 Epoch 3 Step: 147900 Batch Loss: 1.291800 Tokens per Sec: 19494, Lr: 0.000210\n", "2020-01-17 22:21:16,233 Epoch 3 Step: 148000 Batch Loss: 1.036412 Tokens per Sec: 19342, Lr: 0.000210\n", "2020-01-17 22:21:45,458 Example #0\n", "2020-01-17 22:21:45,459 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:21:45,459 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:21:45,459 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:21:45,459 Example #1\n", "2020-01-17 22:21:45,460 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:21:45,460 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:21:45,460 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 22:21:45,460 Example #2\n", "2020-01-17 22:21:45,461 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:21:45,461 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:21:45,461 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme nọ e rrọ obehru na te ekuhọ ọrọ ekuhọ ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho na kpobi a wo oghale .\n", "2020-01-17 22:21:45,461 Example #3\n", "2020-01-17 22:21:45,462 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:21:45,462 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:21:45,462 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:21:45,462 Validation result (greedy) at epoch 3, step 148000: bleu: 31.32, loss: 33217.9883, ppl: 3.6014, duration: 29.2284s\n", "2020-01-17 22:21:56,937 Epoch 3 Step: 148100 Batch Loss: 1.159129 Tokens per Sec: 19009, Lr: 0.000210\n", "2020-01-17 22:22:08,592 Epoch 3 Step: 148200 Batch Loss: 1.216370 Tokens per Sec: 19179, Lr: 0.000210\n", "2020-01-17 22:22:20,130 Epoch 3 Step: 148300 Batch Loss: 1.428369 Tokens per Sec: 19421, Lr: 0.000210\n", "2020-01-17 22:22:31,699 Epoch 3 Step: 148400 Batch Loss: 1.425403 Tokens per Sec: 19355, Lr: 0.000210\n", "2020-01-17 22:22:43,326 Epoch 3 Step: 148500 Batch Loss: 1.287581 Tokens per Sec: 19105, Lr: 0.000210\n", "2020-01-17 22:22:54,921 Epoch 3 Step: 148600 Batch Loss: 0.997928 Tokens per Sec: 19792, Lr: 0.000210\n", "2020-01-17 22:23:06,425 Epoch 3 Step: 148700 Batch Loss: 1.318381 Tokens per Sec: 19650, Lr: 0.000210\n", "2020-01-17 22:23:17,956 Epoch 3 Step: 148800 Batch Loss: 1.422751 Tokens per Sec: 19140, Lr: 0.000210\n", "2020-01-17 22:23:29,525 Epoch 3 Step: 148900 Batch Loss: 0.951386 Tokens per Sec: 19241, Lr: 0.000210\n", "2020-01-17 22:23:41,276 Epoch 3 Step: 149000 Batch Loss: 1.351686 Tokens per Sec: 18950, Lr: 0.000210\n", "2020-01-17 22:24:10,496 Example #0\n", "2020-01-17 22:24:10,496 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:24:10,496 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:24:10,496 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:24:10,497 Example #1\n", "2020-01-17 22:24:10,497 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:24:10,497 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:24:10,497 \tHypothesis: A te lele onana , Usi Uvie na O Kẹlino .\n", "2020-01-17 22:24:10,497 Example #2\n", "2020-01-17 22:24:10,498 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:24:10,498 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:24:10,498 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹvi mai evaọ abọ urere ọrọ abọ urere ọrọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho na kpobi i wo oghale .\n", "2020-01-17 22:24:10,498 Example #3\n", "2020-01-17 22:24:10,498 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:24:10,499 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:24:10,499 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:24:10,499 Validation result (greedy) at epoch 3, step 149000: bleu: 31.33, loss: 33266.5977, ppl: 3.6081, duration: 29.2226s\n", "2020-01-17 22:24:21,930 Epoch 3 Step: 149100 Batch Loss: 1.188910 Tokens per Sec: 19279, Lr: 0.000210\n", "2020-01-17 22:24:33,502 Epoch 3 Step: 149200 Batch Loss: 1.170844 Tokens per Sec: 19231, Lr: 0.000210\n", "2020-01-17 22:24:45,086 Epoch 3 Step: 149300 Batch Loss: 1.468673 Tokens per Sec: 19235, Lr: 0.000210\n", "2020-01-17 22:24:56,513 Epoch 3 Step: 149400 Batch Loss: 1.445155 Tokens per Sec: 19408, Lr: 0.000210\n", "2020-01-17 22:25:08,058 Epoch 3 Step: 149500 Batch Loss: 1.272056 Tokens per Sec: 18766, Lr: 0.000210\n", "2020-01-17 22:25:15,918 Epoch 3: total training loss 3145.91\n", "2020-01-17 22:25:15,918 EPOCH 4\n", "2020-01-17 22:25:19,683 Epoch 4 Step: 149600 Batch Loss: 1.324152 Tokens per Sec: 18163, Lr: 0.000210\n", "2020-01-17 22:25:31,322 Epoch 4 Step: 149700 Batch Loss: 1.049260 Tokens per Sec: 19513, Lr: 0.000210\n", "2020-01-17 22:25:42,878 Epoch 4 Step: 149800 Batch Loss: 1.117121 Tokens per Sec: 19138, Lr: 0.000210\n", "2020-01-17 22:25:54,255 Epoch 4 Step: 149900 Batch Loss: 1.337767 Tokens per Sec: 19160, Lr: 0.000210\n", "2020-01-17 22:26:05,604 Epoch 4 Step: 150000 Batch Loss: 1.244770 Tokens per Sec: 18817, Lr: 0.000210\n", "2020-01-17 22:26:34,806 Example #0\n", "2020-01-17 22:26:34,807 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:26:34,807 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:26:34,808 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:26:34,808 Example #1\n", "2020-01-17 22:26:34,808 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:26:34,808 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:26:34,808 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 22:26:34,808 Example #2\n", "2020-01-17 22:26:34,809 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:26:34,809 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:26:34,809 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho kpobi i wo oghale .\n", "2020-01-17 22:26:34,809 Example #3\n", "2020-01-17 22:26:34,810 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:26:34,810 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:26:34,810 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:26:34,810 Validation result (greedy) at epoch 4, step 150000: bleu: 30.71, loss: 33267.2617, ppl: 3.6082, duration: 29.2058s\n", "2020-01-17 22:26:46,412 Epoch 4 Step: 150100 Batch Loss: 1.322254 Tokens per Sec: 19476, Lr: 0.000210\n", "2020-01-17 22:26:57,802 Epoch 4 Step: 150200 Batch Loss: 1.358391 Tokens per Sec: 19289, Lr: 0.000210\n", "2020-01-17 22:27:09,429 Epoch 4 Step: 150300 Batch Loss: 1.262227 Tokens per Sec: 19303, Lr: 0.000210\n", "2020-01-17 22:27:20,877 Epoch 4 Step: 150400 Batch Loss: 1.406824 Tokens per Sec: 19203, Lr: 0.000210\n", "2020-01-17 22:27:32,346 Epoch 4 Step: 150500 Batch Loss: 0.975035 Tokens per Sec: 18819, Lr: 0.000210\n", "2020-01-17 22:27:44,084 Epoch 4 Step: 150600 Batch Loss: 1.171049 Tokens per Sec: 19304, Lr: 0.000210\n", "2020-01-17 22:27:55,528 Epoch 4 Step: 150700 Batch Loss: 1.283311 Tokens per Sec: 19333, Lr: 0.000210\n", "2020-01-17 22:28:07,006 Epoch 4 Step: 150800 Batch Loss: 1.265933 Tokens per Sec: 19483, Lr: 0.000210\n", "2020-01-17 22:28:18,524 Epoch 4 Step: 150900 Batch Loss: 1.193900 Tokens per Sec: 19619, Lr: 0.000210\n", "2020-01-17 22:28:29,974 Epoch 4 Step: 151000 Batch Loss: 1.389466 Tokens per Sec: 19324, Lr: 0.000210\n", "2020-01-17 22:28:59,155 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:28:59,155 Saving new checkpoint.\n", "2020-01-17 22:29:00,351 Example #0\n", "2020-01-17 22:29:00,352 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:29:00,352 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:29:00,352 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:29:00,352 Example #1\n", "2020-01-17 22:29:00,353 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:29:00,353 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:29:00,353 \tHypothesis: A te lele Usi Uvie na avọ Ijo .\n", "2020-01-17 22:29:00,353 Example #2\n", "2020-01-17 22:29:00,353 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:29:00,354 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:29:00,354 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba otofa urirẹ mai na rite abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ [ Jesu Kristi Ovie na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo viere , re o wo odẹ riẹ , re erẹwho na kpobi a wo oghale .\n", "2020-01-17 22:29:00,354 Example #3\n", "2020-01-17 22:29:00,355 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:29:00,355 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:29:00,355 \tHypothesis: Ghele na , eme unu - uwou e rẹ kẹ ẹgba gaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 22:29:00,355 Validation result (greedy) at epoch 4, step 151000: bleu: 31.33, loss: 33168.0898, ppl: 3.5944, duration: 30.3807s\n", "2020-01-17 22:29:12,104 Epoch 4 Step: 151100 Batch Loss: 1.339110 Tokens per Sec: 18583, Lr: 0.000210\n", "2020-01-17 22:29:23,643 Epoch 4 Step: 151200 Batch Loss: 1.165817 Tokens per Sec: 19333, Lr: 0.000210\n", "2020-01-17 22:29:35,108 Epoch 4 Step: 151300 Batch Loss: 1.344244 Tokens per Sec: 18975, Lr: 0.000210\n", "2020-01-17 22:29:46,651 Epoch 4 Step: 151400 Batch Loss: 1.349631 Tokens per Sec: 19355, Lr: 0.000210\n", "2020-01-17 22:29:58,165 Epoch 4 Step: 151500 Batch Loss: 1.136694 Tokens per Sec: 19383, Lr: 0.000210\n", "2020-01-17 22:30:09,811 Epoch 4 Step: 151600 Batch Loss: 1.261768 Tokens per Sec: 19563, Lr: 0.000210\n", "2020-01-17 22:30:21,226 Epoch 4 Step: 151700 Batch Loss: 1.260203 Tokens per Sec: 19333, Lr: 0.000210\n", "2020-01-17 22:30:32,626 Epoch 4 Step: 151800 Batch Loss: 1.248260 Tokens per Sec: 19098, Lr: 0.000210\n", "2020-01-17 22:30:44,179 Epoch 4 Step: 151900 Batch Loss: 1.394626 Tokens per Sec: 19528, Lr: 0.000210\n", "2020-01-17 22:30:55,601 Epoch 4 Step: 152000 Batch Loss: 1.379810 Tokens per Sec: 19201, Lr: 0.000210\n", "2020-01-17 22:31:24,709 Example #0\n", "2020-01-17 22:31:24,710 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:31:24,710 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:31:24,710 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:31:24,710 Example #1\n", "2020-01-17 22:31:24,710 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:31:24,711 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:31:24,711 \tHypothesis: Ijo , a te lele iei .\n", "2020-01-17 22:31:24,711 Example #2\n", "2020-01-17 22:31:24,711 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:31:24,711 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:31:24,712 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹthẹ mai te abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo oruaro , re o wo oruaro , re ọ ghale ae ; jọ erẹwho na kpobi i wo oghale .\n", "2020-01-17 22:31:24,712 Example #3\n", "2020-01-17 22:31:24,712 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:31:24,712 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:31:24,712 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:31:24,712 Validation result (greedy) at epoch 4, step 152000: bleu: 30.98, loss: 33182.5664, ppl: 3.5964, duration: 29.1106s\n", "2020-01-17 22:31:36,172 Epoch 4 Step: 152100 Batch Loss: 1.216007 Tokens per Sec: 19303, Lr: 0.000210\n", "2020-01-17 22:31:36,769 Epoch 4: total training loss 3159.77\n", "2020-01-17 22:31:36,769 EPOCH 5\n", "2020-01-17 22:31:47,949 Epoch 5 Step: 152200 Batch Loss: 1.279108 Tokens per Sec: 18929, Lr: 0.000210\n", "2020-01-17 22:31:59,464 Epoch 5 Step: 152300 Batch Loss: 1.103387 Tokens per Sec: 19031, Lr: 0.000210\n", "2020-01-17 22:32:11,047 Epoch 5 Step: 152400 Batch Loss: 1.236609 Tokens per Sec: 19261, Lr: 0.000210\n", "2020-01-17 22:32:22,517 Epoch 5 Step: 152500 Batch Loss: 1.249850 Tokens per Sec: 19806, Lr: 0.000210\n", "2020-01-17 22:32:33,996 Epoch 5 Step: 152600 Batch Loss: 1.041405 Tokens per Sec: 19102, Lr: 0.000210\n", "2020-01-17 22:32:45,592 Epoch 5 Step: 152700 Batch Loss: 1.060859 Tokens per Sec: 19467, Lr: 0.000210\n", "2020-01-17 22:32:57,196 Epoch 5 Step: 152800 Batch Loss: 1.487204 Tokens per Sec: 19332, Lr: 0.000210\n", "2020-01-17 22:33:08,765 Epoch 5 Step: 152900 Batch Loss: 1.383442 Tokens per Sec: 19154, Lr: 0.000210\n", "2020-01-17 22:33:20,212 Epoch 5 Step: 153000 Batch Loss: 1.390426 Tokens per Sec: 19597, Lr: 0.000210\n", "2020-01-17 22:33:49,919 Example #0\n", "2020-01-17 22:33:49,920 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:33:49,920 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:33:49,920 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:33:49,920 Example #1\n", "2020-01-17 22:33:49,920 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:33:49,921 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:33:49,921 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 22:33:49,921 Example #2\n", "2020-01-17 22:33:49,921 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:33:49,921 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:33:49,921 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai kẹ abọ urere ọrọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re o jọ rọ ere kẹ ahwo Egedhọ kpobi ; wha kẹ e ; wha kẹ e udhedhẹ , re o jọ ahwo Egedhọ kpobi ; wha wo oghale .\n", "2020-01-17 22:33:49,922 Example #3\n", "2020-01-17 22:33:49,922 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:33:49,922 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:33:49,922 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:33:49,922 Validation result (greedy) at epoch 5, step 153000: bleu: 31.12, loss: 33219.0586, ppl: 3.6015, duration: 29.7100s\n", "2020-01-17 22:34:01,573 Epoch 5 Step: 153100 Batch Loss: 1.285264 Tokens per Sec: 19608, Lr: 0.000210\n", "2020-01-17 22:34:13,154 Epoch 5 Step: 153200 Batch Loss: 0.902654 Tokens per Sec: 18681, Lr: 0.000210\n", "2020-01-17 22:34:24,675 Epoch 5 Step: 153300 Batch Loss: 1.301311 Tokens per Sec: 19482, Lr: 0.000210\n", "2020-01-17 22:34:36,164 Epoch 5 Step: 153400 Batch Loss: 1.400406 Tokens per Sec: 19509, Lr: 0.000210\n", "2020-01-17 22:34:47,624 Epoch 5 Step: 153500 Batch Loss: 1.321634 Tokens per Sec: 19209, Lr: 0.000210\n", "2020-01-17 22:34:59,023 Epoch 5 Step: 153600 Batch Loss: 1.400404 Tokens per Sec: 19589, Lr: 0.000210\n", "2020-01-17 22:35:10,571 Epoch 5 Step: 153700 Batch Loss: 1.142522 Tokens per Sec: 18926, Lr: 0.000210\n", "2020-01-17 22:35:21,976 Epoch 5 Step: 153800 Batch Loss: 1.389123 Tokens per Sec: 19268, Lr: 0.000210\n", "2020-01-17 22:35:33,576 Epoch 5 Step: 153900 Batch Loss: 1.430856 Tokens per Sec: 19134, Lr: 0.000210\n", "2020-01-17 22:35:45,075 Epoch 5 Step: 154000 Batch Loss: 1.409757 Tokens per Sec: 19233, Lr: 0.000210\n", "2020-01-17 22:36:14,259 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:36:14,259 Saving new checkpoint.\n", "2020-01-17 22:36:15,460 Example #0\n", "2020-01-17 22:36:15,461 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:36:15,461 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:36:15,461 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:36:15,461 Example #1\n", "2020-01-17 22:36:15,462 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:36:15,462 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:36:15,462 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 22:36:15,462 Example #2\n", "2020-01-17 22:36:15,463 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:36:15,463 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:36:15,463 \tHypothesis: Koyehọ , mai kpobi ma te rọ evawere fiba eme nọ e rrọ obehru na evaọ abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re o jẹ ghale ae ; jọ o jọ ahwo Egedhọ kpobi ; wha wo oghale .\n", "2020-01-17 22:36:15,463 Example #3\n", "2020-01-17 22:36:15,463 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:36:15,464 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:36:15,464 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:36:15,464 Validation result (greedy) at epoch 5, step 154000: bleu: 31.05, loss: 33167.3281, ppl: 3.5943, duration: 30.3884s\n", "2020-01-17 22:36:27,280 Epoch 5 Step: 154100 Batch Loss: 1.261879 Tokens per Sec: 18404, Lr: 0.000210\n", "2020-01-17 22:36:38,819 Epoch 5 Step: 154200 Batch Loss: 1.282098 Tokens per Sec: 19576, Lr: 0.000210\n", "2020-01-17 22:36:50,282 Epoch 5 Step: 154300 Batch Loss: 1.256922 Tokens per Sec: 19262, Lr: 0.000210\n", "2020-01-17 22:37:01,860 Epoch 5 Step: 154400 Batch Loss: 1.140816 Tokens per Sec: 19715, Lr: 0.000210\n", "2020-01-17 22:37:13,430 Epoch 5 Step: 154500 Batch Loss: 1.380157 Tokens per Sec: 18719, Lr: 0.000210\n", "2020-01-17 22:37:24,951 Epoch 5 Step: 154600 Batch Loss: 1.394953 Tokens per Sec: 19210, Lr: 0.000210\n", "2020-01-17 22:37:28,876 Epoch 5: total training loss 3142.26\n", "2020-01-17 22:37:28,876 EPOCH 6\n", "2020-01-17 22:37:36,688 Epoch 6 Step: 154700 Batch Loss: 1.176917 Tokens per Sec: 18553, Lr: 0.000210\n", "2020-01-17 22:37:48,114 Epoch 6 Step: 154800 Batch Loss: 1.160845 Tokens per Sec: 19035, Lr: 0.000210\n", "2020-01-17 22:37:59,641 Epoch 6 Step: 154900 Batch Loss: 1.465017 Tokens per Sec: 19601, Lr: 0.000210\n", "2020-01-17 22:38:11,207 Epoch 6 Step: 155000 Batch Loss: 1.310340 Tokens per Sec: 19330, Lr: 0.000210\n", "2020-01-17 22:38:40,274 Example #0\n", "2020-01-17 22:38:40,275 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:38:40,275 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:38:40,275 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:38:40,275 Example #1\n", "2020-01-17 22:38:40,276 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:38:40,276 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:38:40,276 \tHypothesis: A te lele onana , Usi Uvie Na O tẹ te ze .\n", "2020-01-17 22:38:40,276 Example #2\n", "2020-01-17 22:38:40,277 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:38:40,277 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:38:40,277 \tHypothesis: Koyehọ , mai kpobi ma te rọ evawere fiba eme nọ e rrọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho na kpobi a wo oghale .\n", "2020-01-17 22:38:40,277 Example #3\n", "2020-01-17 22:38:40,277 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:38:40,278 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:38:40,278 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 22:38:40,278 Validation result (greedy) at epoch 6, step 155000: bleu: 31.13, loss: 33235.9883, ppl: 3.6039, duration: 29.0703s\n", "2020-01-17 22:38:51,885 Epoch 6 Step: 155100 Batch Loss: 1.396279 Tokens per Sec: 19553, Lr: 0.000210\n", "2020-01-17 22:39:03,287 Epoch 6 Step: 155200 Batch Loss: 1.333278 Tokens per Sec: 19747, Lr: 0.000210\n", "2020-01-17 22:39:15,070 Epoch 6 Step: 155300 Batch Loss: 1.146987 Tokens per Sec: 19066, Lr: 0.000210\n", "2020-01-17 22:39:26,547 Epoch 6 Step: 155400 Batch Loss: 1.277099 Tokens per Sec: 18882, Lr: 0.000210\n", "2020-01-17 22:39:37,942 Epoch 6 Step: 155500 Batch Loss: 1.141752 Tokens per Sec: 19520, Lr: 0.000210\n", "2020-01-17 22:39:49,471 Epoch 6 Step: 155600 Batch Loss: 1.134453 Tokens per Sec: 19667, Lr: 0.000210\n", "2020-01-17 22:40:00,844 Epoch 6 Step: 155700 Batch Loss: 1.095177 Tokens per Sec: 19252, Lr: 0.000210\n", "2020-01-17 22:40:12,327 Epoch 6 Step: 155800 Batch Loss: 1.318939 Tokens per Sec: 19068, Lr: 0.000210\n", "2020-01-17 22:40:23,860 Epoch 6 Step: 155900 Batch Loss: 0.817629 Tokens per Sec: 18897, Lr: 0.000210\n", "2020-01-17 22:40:35,358 Epoch 6 Step: 156000 Batch Loss: 1.122804 Tokens per Sec: 19355, Lr: 0.000210\n", "2020-01-17 22:41:04,417 Example #0\n", "2020-01-17 22:41:04,418 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:41:04,418 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:41:04,418 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:41:04,418 Example #1\n", "2020-01-17 22:41:04,418 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:41:04,418 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:41:04,419 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-17 22:41:04,419 Example #2\n", "2020-01-17 22:41:04,419 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:41:04,419 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:41:04,419 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹvi mai rite abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre na o te wo odẹ riẹ , re o jọ o jọ o jọ rọ ere wha oghale se ahwo Egedhọ .\n", "2020-01-17 22:41:04,419 Example #3\n", "2020-01-17 22:41:04,420 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:41:04,420 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:41:04,420 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:41:04,420 Validation result (greedy) at epoch 6, step 156000: bleu: 31.54, loss: 33227.2930, ppl: 3.6027, duration: 29.0622s\n", "2020-01-17 22:41:15,972 Epoch 6 Step: 156100 Batch Loss: 1.123585 Tokens per Sec: 19029, Lr: 0.000210\n", "2020-01-17 22:41:27,494 Epoch 6 Step: 156200 Batch Loss: 1.331388 Tokens per Sec: 19428, Lr: 0.000210\n", "2020-01-17 22:41:38,938 Epoch 6 Step: 156300 Batch Loss: 1.067647 Tokens per Sec: 19300, Lr: 0.000210\n", "2020-01-17 22:41:50,397 Epoch 6 Step: 156400 Batch Loss: 1.192533 Tokens per Sec: 19319, Lr: 0.000210\n", "2020-01-17 22:42:01,895 Epoch 6 Step: 156500 Batch Loss: 1.479509 Tokens per Sec: 19526, Lr: 0.000210\n", "2020-01-17 22:42:13,444 Epoch 6 Step: 156600 Batch Loss: 1.434275 Tokens per Sec: 19044, Lr: 0.000210\n", "2020-01-17 22:42:24,916 Epoch 6 Step: 156700 Batch Loss: 1.337188 Tokens per Sec: 19142, Lr: 0.000210\n", "2020-01-17 22:42:36,390 Epoch 6 Step: 156800 Batch Loss: 0.996943 Tokens per Sec: 19460, Lr: 0.000210\n", "2020-01-17 22:42:47,855 Epoch 6 Step: 156900 Batch Loss: 1.011930 Tokens per Sec: 19586, Lr: 0.000210\n", "2020-01-17 22:42:59,306 Epoch 6 Step: 157000 Batch Loss: 1.233823 Tokens per Sec: 19316, Lr: 0.000210\n", "2020-01-17 22:43:28,453 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:43:28,453 Saving new checkpoint.\n", "2020-01-17 22:43:29,660 Example #0\n", "2020-01-17 22:43:29,661 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:43:29,661 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:43:29,661 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:43:29,661 Example #1\n", "2020-01-17 22:43:29,662 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:43:29,662 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:43:29,662 \tHypothesis: A te je lele Usi Uvie na Erọ Avọ Ee .\n", "2020-01-17 22:43:29,662 Example #2\n", "2020-01-17 22:43:29,662 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:43:29,663 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:43:29,663 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme nọ e rrọ ekuhọ ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo odẹ riẹ , re ọ ghale ae ; wha jọ erẹwho kpobi a wo oghale .\n", "2020-01-17 22:43:29,663 Example #3\n", "2020-01-17 22:43:29,663 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:43:29,664 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:43:29,664 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:43:29,664 Validation result (greedy) at epoch 6, step 157000: bleu: 31.35, loss: 33150.1367, ppl: 3.5919, duration: 30.3573s\n", "2020-01-17 22:43:41,543 Epoch 6 Step: 157100 Batch Loss: 1.271860 Tokens per Sec: 18895, Lr: 0.000210\n", "2020-01-17 22:43:49,000 Epoch 6: total training loss 3136.83\n", "2020-01-17 22:43:49,000 EPOCH 7\n", "2020-01-17 22:43:53,270 Epoch 7 Step: 157200 Batch Loss: 1.012132 Tokens per Sec: 18204, Lr: 0.000210\n", "2020-01-17 22:44:04,777 Epoch 7 Step: 157300 Batch Loss: 1.286841 Tokens per Sec: 19735, Lr: 0.000210\n", "2020-01-17 22:44:16,308 Epoch 7 Step: 157400 Batch Loss: 1.135420 Tokens per Sec: 19112, Lr: 0.000210\n", "2020-01-17 22:44:27,820 Epoch 7 Step: 157500 Batch Loss: 1.359746 Tokens per Sec: 19097, Lr: 0.000210\n", "2020-01-17 22:44:39,327 Epoch 7 Step: 157600 Batch Loss: 1.414894 Tokens per Sec: 19428, Lr: 0.000210\n", "2020-01-17 22:44:50,865 Epoch 7 Step: 157700 Batch Loss: 1.118271 Tokens per Sec: 19368, Lr: 0.000210\n", "2020-01-17 22:45:02,347 Epoch 7 Step: 157800 Batch Loss: 1.422183 Tokens per Sec: 19638, Lr: 0.000210\n", "2020-01-17 22:45:13,878 Epoch 7 Step: 157900 Batch Loss: 1.411708 Tokens per Sec: 19343, Lr: 0.000210\n", "2020-01-17 22:45:25,220 Epoch 7 Step: 158000 Batch Loss: 1.259148 Tokens per Sec: 19202, Lr: 0.000210\n", "2020-01-17 22:45:54,281 Example #0\n", "2020-01-17 22:45:54,282 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:45:54,282 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:45:54,282 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:45:54,282 Example #1\n", "2020-01-17 22:45:54,283 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:45:54,283 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:45:54,283 \tHypothesis: A te ru onana avọ Usi Uvie na no .\n", "2020-01-17 22:45:54,283 Example #2\n", "2020-01-17 22:45:54,283 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:45:54,284 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:45:54,284 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie Jesu Kristi na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; re erẹwho na kpobi i wo oghale .\n", "2020-01-17 22:45:54,284 Example #3\n", "2020-01-17 22:45:54,284 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:45:54,284 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:45:54,284 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:45:54,285 Validation result (greedy) at epoch 7, step 158000: bleu: 30.74, loss: 33310.5703, ppl: 3.6142, duration: 29.0639s\n", "2020-01-17 22:46:05,766 Epoch 7 Step: 158100 Batch Loss: 1.176388 Tokens per Sec: 19324, Lr: 0.000210\n", "2020-01-17 22:46:17,225 Epoch 7 Step: 158200 Batch Loss: 1.152281 Tokens per Sec: 18990, Lr: 0.000210\n", "2020-01-17 22:46:28,770 Epoch 7 Step: 158300 Batch Loss: 1.110079 Tokens per Sec: 19140, Lr: 0.000210\n", "2020-01-17 22:46:40,355 Epoch 7 Step: 158400 Batch Loss: 1.453068 Tokens per Sec: 19631, Lr: 0.000210\n", "2020-01-17 22:46:51,814 Epoch 7 Step: 158500 Batch Loss: 1.211557 Tokens per Sec: 19090, Lr: 0.000210\n", "2020-01-17 22:47:03,165 Epoch 7 Step: 158600 Batch Loss: 1.214824 Tokens per Sec: 19179, Lr: 0.000210\n", "2020-01-17 22:47:14,712 Epoch 7 Step: 158700 Batch Loss: 1.224254 Tokens per Sec: 19267, Lr: 0.000210\n", "2020-01-17 22:47:26,214 Epoch 7 Step: 158800 Batch Loss: 1.072694 Tokens per Sec: 19275, Lr: 0.000210\n", "2020-01-17 22:47:37,672 Epoch 7 Step: 158900 Batch Loss: 1.147578 Tokens per Sec: 19352, Lr: 0.000210\n", "2020-01-17 22:47:49,112 Epoch 7 Step: 159000 Batch Loss: 1.457334 Tokens per Sec: 19168, Lr: 0.000210\n", "2020-01-17 22:48:18,257 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:48:18,258 Saving new checkpoint.\n", "2020-01-17 22:48:19,484 Example #0\n", "2020-01-17 22:48:19,484 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:48:19,485 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:48:19,485 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ udu efuafo . ’\n", "2020-01-17 22:48:19,485 Example #1\n", "2020-01-17 22:48:19,485 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:48:19,486 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:48:19,486 \tHypothesis: A te ru onana avọ Usi Uvie na , Ijo .\n", "2020-01-17 22:48:19,486 Example #2\n", "2020-01-17 22:48:19,486 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:48:19,487 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:48:19,487 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme mai na te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-17 22:48:19,487 Example #3\n", "2020-01-17 22:48:19,487 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:48:19,487 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:48:19,487 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 22:48:19,488 Validation result (greedy) at epoch 7, step 159000: bleu: 31.32, loss: 33128.3789, ppl: 3.5889, duration: 30.3753s\n", "2020-01-17 22:48:31,252 Epoch 7 Step: 159100 Batch Loss: 1.332062 Tokens per Sec: 19413, Lr: 0.000210\n", "2020-01-17 22:48:42,810 Epoch 7 Step: 159200 Batch Loss: 1.275167 Tokens per Sec: 18652, Lr: 0.000210\n", "2020-01-17 22:48:54,327 Epoch 7 Step: 159300 Batch Loss: 1.304332 Tokens per Sec: 19596, Lr: 0.000210\n", "2020-01-17 22:49:05,792 Epoch 7 Step: 159400 Batch Loss: 1.235653 Tokens per Sec: 19138, Lr: 0.000210\n", "2020-01-17 22:49:17,395 Epoch 7 Step: 159500 Batch Loss: 0.813206 Tokens per Sec: 19197, Lr: 0.000210\n", "2020-01-17 22:49:28,922 Epoch 7 Step: 159600 Batch Loss: 1.285529 Tokens per Sec: 19189, Lr: 0.000210\n", "2020-01-17 22:49:40,247 Epoch 7: total training loss 3135.23\n", "2020-01-17 22:49:40,247 EPOCH 8\n", "2020-01-17 22:49:40,746 Epoch 8 Step: 159700 Batch Loss: 1.237696 Tokens per Sec: 9922, Lr: 0.000210\n", "2020-01-17 22:49:52,224 Epoch 8 Step: 159800 Batch Loss: 1.381256 Tokens per Sec: 19182, Lr: 0.000210\n", "2020-01-17 22:50:03,669 Epoch 8 Step: 159900 Batch Loss: 1.362101 Tokens per Sec: 19413, Lr: 0.000210\n", "2020-01-17 22:50:15,278 Epoch 8 Step: 160000 Batch Loss: 1.364065 Tokens per Sec: 19165, Lr: 0.000210\n", "2020-01-17 22:50:44,484 Example #0\n", "2020-01-17 22:50:44,485 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:50:44,485 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:50:44,485 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:50:44,486 Example #1\n", "2020-01-17 22:50:44,486 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:50:44,486 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:50:44,486 \tHypothesis: A te je ru onana avọ Usi Uvie Na .\n", "2020-01-17 22:50:44,486 Example #2\n", "2020-01-17 22:50:44,487 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:50:44,487 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:50:44,487 \tHypothesis: Koyehọ , mai kpobi ma te rehọ oghọghọ fiba ẹthẹ mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re erẹwho na kpobi a wo oghale ; wha wo oghale .\n", "2020-01-17 22:50:44,487 Example #3\n", "2020-01-17 22:50:44,488 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:50:44,488 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:50:44,488 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:50:44,488 Validation result (greedy) at epoch 8, step 160000: bleu: 31.46, loss: 33135.1875, ppl: 3.5899, duration: 29.2097s\n", "2020-01-17 22:50:55,976 Epoch 8 Step: 160100 Batch Loss: 1.172117 Tokens per Sec: 19534, Lr: 0.000210\n", "2020-01-17 22:51:07,359 Epoch 8 Step: 160200 Batch Loss: 1.353239 Tokens per Sec: 18881, Lr: 0.000210\n", "2020-01-17 22:51:18,767 Epoch 8 Step: 160300 Batch Loss: 0.880828 Tokens per Sec: 19246, Lr: 0.000210\n", "2020-01-17 22:51:30,278 Epoch 8 Step: 160400 Batch Loss: 1.328207 Tokens per Sec: 19165, Lr: 0.000210\n", "2020-01-17 22:51:41,796 Epoch 8 Step: 160500 Batch Loss: 1.249574 Tokens per Sec: 19185, Lr: 0.000210\n", "2020-01-17 22:51:53,324 Epoch 8 Step: 160600 Batch Loss: 1.143306 Tokens per Sec: 19559, Lr: 0.000210\n", "2020-01-17 22:52:04,757 Epoch 8 Step: 160700 Batch Loss: 1.253643 Tokens per Sec: 19749, Lr: 0.000210\n", "2020-01-17 22:52:16,402 Epoch 8 Step: 160800 Batch Loss: 1.169957 Tokens per Sec: 19604, Lr: 0.000210\n", "2020-01-17 22:52:27,904 Epoch 8 Step: 160900 Batch Loss: 1.213345 Tokens per Sec: 19520, Lr: 0.000210\n", "2020-01-17 22:52:39,353 Epoch 8 Step: 161000 Batch Loss: 1.346187 Tokens per Sec: 19653, Lr: 0.000210\n", "2020-01-17 22:53:08,565 Example #0\n", "2020-01-17 22:53:08,566 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:53:08,566 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:53:08,566 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:53:08,566 Example #1\n", "2020-01-17 22:53:08,567 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:53:08,567 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:53:08,567 \tHypothesis: Ijo , a te ru onana .\n", "2020-01-17 22:53:08,567 Example #2\n", "2020-01-17 22:53:08,567 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:53:08,567 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:53:08,568 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo odẹ riẹ , re o jẹ te ghale ae ; wha jọ o wo oghale kẹ ahwo Egedhọ kpobi ; wha wo oghale .\n", "2020-01-17 22:53:08,568 Example #3\n", "2020-01-17 22:53:08,568 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:53:08,568 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:53:08,569 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:53:08,569 Validation result (greedy) at epoch 8, step 161000: bleu: 31.55, loss: 33146.1328, ppl: 3.5914, duration: 29.2154s\n", "2020-01-17 22:53:20,058 Epoch 8 Step: 161100 Batch Loss: 1.375156 Tokens per Sec: 19176, Lr: 0.000210\n", "2020-01-17 22:53:31,545 Epoch 8 Step: 161200 Batch Loss: 1.296042 Tokens per Sec: 19164, Lr: 0.000210\n", "2020-01-17 22:53:43,082 Epoch 8 Step: 161300 Batch Loss: 1.238402 Tokens per Sec: 19839, Lr: 0.000210\n", "2020-01-17 22:53:54,458 Epoch 8 Step: 161400 Batch Loss: 1.200267 Tokens per Sec: 19115, Lr: 0.000210\n", "2020-01-17 22:54:05,890 Epoch 8 Step: 161500 Batch Loss: 1.348035 Tokens per Sec: 19357, Lr: 0.000210\n", "2020-01-17 22:54:17,426 Epoch 8 Step: 161600 Batch Loss: 1.069077 Tokens per Sec: 19072, Lr: 0.000210\n", "2020-01-17 22:54:28,889 Epoch 8 Step: 161700 Batch Loss: 1.377926 Tokens per Sec: 19464, Lr: 0.000210\n", "2020-01-17 22:54:40,482 Epoch 8 Step: 161800 Batch Loss: 0.995355 Tokens per Sec: 19727, Lr: 0.000210\n", "2020-01-17 22:54:52,127 Epoch 8 Step: 161900 Batch Loss: 1.216999 Tokens per Sec: 19373, Lr: 0.000210\n", "2020-01-17 22:55:03,446 Epoch 8 Step: 162000 Batch Loss: 0.852639 Tokens per Sec: 19432, Lr: 0.000210\n", "2020-01-17 22:55:32,674 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:55:32,674 Saving new checkpoint.\n", "2020-01-17 22:55:33,917 Example #0\n", "2020-01-17 22:55:33,921 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:55:33,921 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:55:33,922 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:55:33,922 Example #1\n", "2020-01-17 22:55:33,922 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:55:33,922 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:55:33,922 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 22:55:33,923 Example #2\n", "2020-01-17 22:55:33,923 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:55:33,923 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:55:33,924 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹvi mai rite abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho kpobi i wo oghale .\n", "2020-01-17 22:55:33,924 Example #3\n", "2020-01-17 22:55:33,924 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:55:33,924 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:55:33,924 \tHypothesis: Ghele na , eme unu - uwou e rẹ kẹ ẹgba gaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 22:55:33,925 Validation result (greedy) at epoch 8, step 162000: bleu: 31.55, loss: 33080.0312, ppl: 3.5822, duration: 30.4784s\n", "2020-01-17 22:55:45,776 Epoch 8 Step: 162100 Batch Loss: 0.976185 Tokens per Sec: 18872, Lr: 0.000210\n", "2020-01-17 22:55:57,183 Epoch 8 Step: 162200 Batch Loss: 1.309482 Tokens per Sec: 19500, Lr: 0.000210\n", "2020-01-17 22:55:59,913 Epoch 8: total training loss 3118.34\n", "2020-01-17 22:55:59,913 EPOCH 9\n", "2020-01-17 22:56:09,066 Epoch 9 Step: 162300 Batch Loss: 1.050092 Tokens per Sec: 18565, Lr: 0.000210\n", "2020-01-17 22:56:20,426 Epoch 9 Step: 162400 Batch Loss: 1.260346 Tokens per Sec: 18858, Lr: 0.000210\n", "2020-01-17 22:56:32,032 Epoch 9 Step: 162500 Batch Loss: 1.101206 Tokens per Sec: 19770, Lr: 0.000210\n", "2020-01-17 22:56:43,651 Epoch 9 Step: 162600 Batch Loss: 1.330935 Tokens per Sec: 19694, Lr: 0.000210\n", "2020-01-17 22:56:55,131 Epoch 9 Step: 162700 Batch Loss: 1.119506 Tokens per Sec: 19148, Lr: 0.000210\n", "2020-01-17 22:57:06,554 Epoch 9 Step: 162800 Batch Loss: 1.326720 Tokens per Sec: 19106, Lr: 0.000210\n", "2020-01-17 22:57:18,119 Epoch 9 Step: 162900 Batch Loss: 1.421923 Tokens per Sec: 19360, Lr: 0.000210\n", "2020-01-17 22:57:29,613 Epoch 9 Step: 163000 Batch Loss: 0.791854 Tokens per Sec: 19123, Lr: 0.000210\n", "2020-01-17 22:57:58,728 Hooray! New best validation result [ppl]!\n", "2020-01-17 22:57:58,728 Saving new checkpoint.\n", "2020-01-17 22:57:59,975 Example #0\n", "2020-01-17 22:57:59,976 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 22:57:59,976 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 22:57:59,977 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 22:57:59,977 Example #1\n", "2020-01-17 22:57:59,977 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 22:57:59,978 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 22:57:59,978 \tHypothesis: A te lele onana ẹkwoma Usi Uvie Na , Ijo .\n", "2020-01-17 22:57:59,978 Example #2\n", "2020-01-17 22:57:59,978 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 22:57:59,978 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 22:57:59,979 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba iru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Jesu Kristi Ovie na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re erẹwho kpobi e jọ enọ e be kẹ uyere ; wha wo oghale .\n", "2020-01-17 22:57:59,979 Example #3\n", "2020-01-17 22:57:59,979 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 22:57:59,979 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 22:57:59,979 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 22:57:59,980 Validation result (greedy) at epoch 9, step 163000: bleu: 31.65, loss: 33079.6094, ppl: 3.5822, duration: 30.3661s\n", "2020-01-17 22:58:11,714 Epoch 9 Step: 163100 Batch Loss: 1.317814 Tokens per Sec: 19193, Lr: 0.000210\n", "2020-01-17 22:58:23,557 Epoch 9 Step: 163200 Batch Loss: 0.980308 Tokens per Sec: 19219, Lr: 0.000210\n", "2020-01-17 22:58:35,009 Epoch 9 Step: 163300 Batch Loss: 1.398762 Tokens per Sec: 19160, Lr: 0.000210\n", "2020-01-17 22:58:46,594 Epoch 9 Step: 163400 Batch Loss: 1.199805 Tokens per Sec: 19366, Lr: 0.000210\n", "2020-01-17 22:58:58,005 Epoch 9 Step: 163500 Batch Loss: 1.179914 Tokens per Sec: 18835, Lr: 0.000210\n", "2020-01-17 22:59:09,505 Epoch 9 Step: 163600 Batch Loss: 1.277339 Tokens per Sec: 19413, Lr: 0.000210\n", "2020-01-17 22:59:21,047 Epoch 9 Step: 163700 Batch Loss: 1.196434 Tokens per Sec: 19478, Lr: 0.000210\n", "2020-01-17 22:59:32,590 Epoch 9 Step: 163800 Batch Loss: 1.298343 Tokens per Sec: 19510, Lr: 0.000210\n", "2020-01-17 22:59:44,040 Epoch 9 Step: 163900 Batch Loss: 0.821946 Tokens per Sec: 19003, Lr: 0.000210\n", "2020-01-17 22:59:55,600 Epoch 9 Step: 164000 Batch Loss: 1.285123 Tokens per Sec: 19459, Lr: 0.000210\n", "2020-01-17 23:00:24,738 Example #0\n", "2020-01-17 23:00:24,739 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:00:24,739 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:00:24,739 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:00:24,739 Example #1\n", "2020-01-17 23:00:24,739 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:00:24,740 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:00:24,740 \tHypothesis: A te ru onana ẹkwoma Usi Uvie Na , Ijo .\n", "2020-01-17 23:00:24,740 Example #2\n", "2020-01-17 23:00:24,740 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:00:24,740 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:00:24,740 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba iru mai re ma ku abọ urere ọrọ ole owoma nana họ : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-17 23:00:24,741 Example #3\n", "2020-01-17 23:00:24,741 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:00:24,741 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:00:24,741 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:00:24,741 Validation result (greedy) at epoch 9, step 164000: bleu: 31.53, loss: 33093.6523, ppl: 3.5841, duration: 29.1406s\n", "2020-01-17 23:00:36,222 Epoch 9 Step: 164100 Batch Loss: 1.287186 Tokens per Sec: 19838, Lr: 0.000210\n", "2020-01-17 23:00:47,736 Epoch 9 Step: 164200 Batch Loss: 1.220944 Tokens per Sec: 19342, Lr: 0.000210\n", "2020-01-17 23:00:59,212 Epoch 9 Step: 164300 Batch Loss: 1.117419 Tokens per Sec: 19373, Lr: 0.000210\n", "2020-01-17 23:01:10,671 Epoch 9 Step: 164400 Batch Loss: 1.299977 Tokens per Sec: 19102, Lr: 0.000210\n", "2020-01-17 23:01:22,160 Epoch 9 Step: 164500 Batch Loss: 1.241589 Tokens per Sec: 19483, Lr: 0.000210\n", "2020-01-17 23:01:33,555 Epoch 9 Step: 164600 Batch Loss: 1.176546 Tokens per Sec: 19282, Lr: 0.000210\n", "2020-01-17 23:01:45,050 Epoch 9 Step: 164700 Batch Loss: 1.291648 Tokens per Sec: 19383, Lr: 0.000210\n", "2020-01-17 23:01:50,782 Epoch 9: total training loss 3116.30\n", "2020-01-17 23:01:50,782 EPOCH 10\n", "2020-01-17 23:01:56,756 Epoch 10 Step: 164800 Batch Loss: 1.271975 Tokens per Sec: 18431, Lr: 0.000210\n", "2020-01-17 23:02:08,415 Epoch 10 Step: 164900 Batch Loss: 1.131383 Tokens per Sec: 19455, Lr: 0.000210\n", "2020-01-17 23:02:19,902 Epoch 10 Step: 165000 Batch Loss: 1.345089 Tokens per Sec: 19385, Lr: 0.000210\n", "2020-01-17 23:02:49,084 Example #0\n", "2020-01-17 23:02:49,084 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:02:49,085 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:02:49,085 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:02:49,085 Example #1\n", "2020-01-17 23:02:49,085 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:02:49,085 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:02:49,085 \tHypothesis: A te lele onana Usi Uvie Na no .\n", "2020-01-17 23:02:49,085 Example #2\n", "2020-01-17 23:02:49,086 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:02:49,086 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:02:49,086 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba iru mai rite abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , wha vi odẹ riẹ , re wha wo oghale , re wha wo oghale , wha jiri ai ; wha jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-17 23:02:49,086 Example #3\n", "2020-01-17 23:02:49,087 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:02:49,087 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:02:49,087 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:02:49,087 Validation result (greedy) at epoch 10, step 165000: bleu: 31.68, loss: 33284.8984, ppl: 3.6107, duration: 29.1848s\n", "2020-01-17 23:03:00,526 Epoch 10 Step: 165100 Batch Loss: 1.413257 Tokens per Sec: 19484, Lr: 0.000210\n", "2020-01-17 23:03:12,035 Epoch 10 Step: 165200 Batch Loss: 1.262683 Tokens per Sec: 18917, Lr: 0.000210\n", "2020-01-17 23:03:23,510 Epoch 10 Step: 165300 Batch Loss: 1.105214 Tokens per Sec: 19255, Lr: 0.000210\n", "2020-01-17 23:03:35,071 Epoch 10 Step: 165400 Batch Loss: 1.367981 Tokens per Sec: 19102, Lr: 0.000210\n", "2020-01-17 23:03:46,611 Epoch 10 Step: 165500 Batch Loss: 0.884675 Tokens per Sec: 19189, Lr: 0.000210\n", "2020-01-17 23:03:57,997 Epoch 10 Step: 165600 Batch Loss: 1.193504 Tokens per Sec: 19153, Lr: 0.000210\n", "2020-01-17 23:04:09,594 Epoch 10 Step: 165700 Batch Loss: 1.281458 Tokens per Sec: 19321, Lr: 0.000210\n", "2020-01-17 23:04:21,023 Epoch 10 Step: 165800 Batch Loss: 1.197622 Tokens per Sec: 19585, Lr: 0.000210\n", "2020-01-17 23:04:32,563 Epoch 10 Step: 165900 Batch Loss: 1.475830 Tokens per Sec: 19587, Lr: 0.000210\n", "2020-01-17 23:04:44,133 Epoch 10 Step: 166000 Batch Loss: 1.242070 Tokens per Sec: 19547, Lr: 0.000210\n", "2020-01-17 23:05:13,316 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:05:13,316 Saving new checkpoint.\n", "2020-01-17 23:05:14,521 Example #0\n", "2020-01-17 23:05:14,521 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:05:14,522 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:05:14,522 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:05:14,522 Example #1\n", "2020-01-17 23:05:14,522 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:05:14,523 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:05:14,523 \tHypothesis: A te je ru onana avọ Usi Uvie na Ijo .\n", "2020-01-17 23:05:14,523 Example #2\n", "2020-01-17 23:05:14,523 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:05:14,523 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:05:14,524 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba uvou mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-17 23:05:14,524 Example #3\n", "2020-01-17 23:05:14,524 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:05:14,524 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:05:14,524 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:05:14,525 Validation result (greedy) at epoch 10, step 166000: bleu: 31.05, loss: 33037.5234, ppl: 3.5764, duration: 30.3910s\n", "2020-01-17 23:05:26,373 Epoch 10 Step: 166100 Batch Loss: 1.395832 Tokens per Sec: 19410, Lr: 0.000210\n", "2020-01-17 23:05:37,882 Epoch 10 Step: 166200 Batch Loss: 1.157503 Tokens per Sec: 19202, Lr: 0.000210\n", "2020-01-17 23:05:49,324 Epoch 10 Step: 166300 Batch Loss: 1.072457 Tokens per Sec: 19126, Lr: 0.000210\n", "2020-01-17 23:06:00,706 Epoch 10 Step: 166400 Batch Loss: 1.311912 Tokens per Sec: 19167, Lr: 0.000210\n", "2020-01-17 23:06:12,178 Epoch 10 Step: 166500 Batch Loss: 0.975297 Tokens per Sec: 19009, Lr: 0.000210\n", "2020-01-17 23:06:23,623 Epoch 10 Step: 166600 Batch Loss: 1.288576 Tokens per Sec: 19633, Lr: 0.000210\n", "2020-01-17 23:06:35,188 Epoch 10 Step: 166700 Batch Loss: 1.407210 Tokens per Sec: 19663, Lr: 0.000210\n", "2020-01-17 23:06:46,750 Epoch 10 Step: 166800 Batch Loss: 1.274529 Tokens per Sec: 19330, Lr: 0.000210\n", "2020-01-17 23:06:58,255 Epoch 10 Step: 166900 Batch Loss: 1.457622 Tokens per Sec: 19398, Lr: 0.000210\n", "2020-01-17 23:07:09,753 Epoch 10 Step: 167000 Batch Loss: 1.206718 Tokens per Sec: 19169, Lr: 0.000210\n", "2020-01-17 23:07:38,882 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:07:38,883 Saving new checkpoint.\n", "2020-01-17 23:07:40,111 Example #0\n", "2020-01-17 23:07:40,112 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:07:40,112 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:07:40,112 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:07:40,113 Example #1\n", "2020-01-17 23:07:40,113 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:07:40,113 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:07:40,113 \tHypothesis: A te lele onana ẹkwoma Usi Uvie na Ijo .\n", "2020-01-17 23:07:40,113 Example #2\n", "2020-01-17 23:07:40,114 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:07:40,114 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:07:40,114 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba uvou mai te ekuhọ ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oruaro , re erẹwho kpobi i wo oghale , re o jiri odẹ riẹ ; jọ erẹwho na kpobi i wo oghale .\n", "2020-01-17 23:07:40,115 Example #3\n", "2020-01-17 23:07:40,115 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:07:40,115 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:07:40,115 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:07:40,115 Validation result (greedy) at epoch 10, step 167000: bleu: 31.34, loss: 33014.2852, ppl: 3.5732, duration: 30.3617s\n", "2020-01-17 23:07:51,926 Epoch 10 Step: 167100 Batch Loss: 1.355045 Tokens per Sec: 19202, Lr: 0.000210\n", "2020-01-17 23:08:03,346 Epoch 10 Step: 167200 Batch Loss: 1.336715 Tokens per Sec: 19467, Lr: 0.000210\n", "2020-01-17 23:08:12,199 Epoch 10: total training loss 3106.08\n", "2020-01-17 23:08:12,199 EPOCH 11\n", "2020-01-17 23:08:15,032 Epoch 11 Step: 167300 Batch Loss: 1.313277 Tokens per Sec: 16747, Lr: 0.000210\n", "2020-01-17 23:08:26,591 Epoch 11 Step: 167400 Batch Loss: 1.403317 Tokens per Sec: 19155, Lr: 0.000210\n", "2020-01-17 23:08:37,991 Epoch 11 Step: 167500 Batch Loss: 0.897287 Tokens per Sec: 19387, Lr: 0.000210\n", "2020-01-17 23:08:49,591 Epoch 11 Step: 167600 Batch Loss: 1.317615 Tokens per Sec: 19401, Lr: 0.000210\n", "2020-01-17 23:09:01,110 Epoch 11 Step: 167700 Batch Loss: 1.172890 Tokens per Sec: 19748, Lr: 0.000210\n", "2020-01-17 23:09:12,622 Epoch 11 Step: 167800 Batch Loss: 1.128969 Tokens per Sec: 18971, Lr: 0.000210\n", "2020-01-17 23:09:23,962 Epoch 11 Step: 167900 Batch Loss: 1.155902 Tokens per Sec: 19202, Lr: 0.000210\n", "2020-01-17 23:09:35,357 Epoch 11 Step: 168000 Batch Loss: 1.229939 Tokens per Sec: 19198, Lr: 0.000210\n", "2020-01-17 23:10:04,422 Example #0\n", "2020-01-17 23:10:04,423 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:10:04,423 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:10:04,423 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:10:04,423 Example #1\n", "2020-01-17 23:10:04,423 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:10:04,424 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:10:04,424 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 23:10:04,424 Example #2\n", "2020-01-17 23:10:04,424 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:10:04,424 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:10:04,425 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana , onọ o rẹ kẹ omosasọ : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re erẹwho na kpobi i wo oghale , re o wo oghale .\n", "2020-01-17 23:10:04,425 Example #3\n", "2020-01-17 23:10:04,425 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:10:04,425 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:10:04,425 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:10:04,425 Validation result (greedy) at epoch 11, step 168000: bleu: 31.43, loss: 33147.7930, ppl: 3.5916, duration: 29.0684s\n", "2020-01-17 23:10:16,133 Epoch 11 Step: 168100 Batch Loss: 1.337527 Tokens per Sec: 19087, Lr: 0.000210\n", "2020-01-17 23:10:27,560 Epoch 11 Step: 168200 Batch Loss: 1.002528 Tokens per Sec: 19052, Lr: 0.000210\n", "2020-01-17 23:10:38,926 Epoch 11 Step: 168300 Batch Loss: 1.159041 Tokens per Sec: 19523, Lr: 0.000210\n", "2020-01-17 23:10:50,358 Epoch 11 Step: 168400 Batch Loss: 1.165174 Tokens per Sec: 19430, Lr: 0.000210\n", "2020-01-17 23:11:01,720 Epoch 11 Step: 168500 Batch Loss: 1.180900 Tokens per Sec: 18916, Lr: 0.000210\n", "2020-01-17 23:11:13,173 Epoch 11 Step: 168600 Batch Loss: 1.236332 Tokens per Sec: 19398, Lr: 0.000210\n", "2020-01-17 23:11:24,742 Epoch 11 Step: 168700 Batch Loss: 1.343876 Tokens per Sec: 19546, Lr: 0.000210\n", "2020-01-17 23:11:36,274 Epoch 11 Step: 168800 Batch Loss: 1.294508 Tokens per Sec: 19463, Lr: 0.000210\n", "2020-01-17 23:11:47,811 Epoch 11 Step: 168900 Batch Loss: 1.340992 Tokens per Sec: 19350, Lr: 0.000210\n", "2020-01-17 23:11:59,338 Epoch 11 Step: 169000 Batch Loss: 1.320778 Tokens per Sec: 19328, Lr: 0.000210\n", "2020-01-17 23:12:28,524 Example #0\n", "2020-01-17 23:12:28,525 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:12:28,525 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:12:28,525 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:12:28,525 Example #1\n", "2020-01-17 23:12:28,525 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:12:28,525 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:12:28,526 \tHypothesis: A te lele onana Usi Uvie na No .\n", "2020-01-17 23:12:28,526 Example #2\n", "2020-01-17 23:12:28,526 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:12:28,526 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:12:28,526 \tHypothesis: Koyehọ , mai kpobi ma te rehọ oghọghọ fiba iru mai te abọ urere ọrọ ole owoma nana , onọ o rẹ kẹ omosasọ gbe omosasọ : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo ẹvi , re ọ ghale ae ; wha kẹ ae udhedhẹ , wha jọ erẹwho kpobi jiri ei .\n", "2020-01-17 23:12:28,527 Example #3\n", "2020-01-17 23:12:28,527 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:12:28,527 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:12:28,527 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:12:28,527 Validation result (greedy) at epoch 11, step 169000: bleu: 31.51, loss: 33050.3633, ppl: 3.5781, duration: 29.1887s\n", "2020-01-17 23:12:40,038 Epoch 11 Step: 169100 Batch Loss: 1.093764 Tokens per Sec: 19551, Lr: 0.000210\n", "2020-01-17 23:12:51,596 Epoch 11 Step: 169200 Batch Loss: 1.292966 Tokens per Sec: 19425, Lr: 0.000210\n", "2020-01-17 23:13:03,006 Epoch 11 Step: 169300 Batch Loss: 1.731230 Tokens per Sec: 19158, Lr: 0.000210\n", "2020-01-17 23:13:14,561 Epoch 11 Step: 169400 Batch Loss: 1.214670 Tokens per Sec: 19382, Lr: 0.000210\n", "2020-01-17 23:13:26,022 Epoch 11 Step: 169500 Batch Loss: 1.080474 Tokens per Sec: 19445, Lr: 0.000210\n", "2020-01-17 23:13:37,560 Epoch 11 Step: 169600 Batch Loss: 1.319916 Tokens per Sec: 19736, Lr: 0.000210\n", "2020-01-17 23:13:49,058 Epoch 11 Step: 169700 Batch Loss: 1.329207 Tokens per Sec: 19782, Lr: 0.000210\n", "2020-01-17 23:14:00,460 Epoch 11 Step: 169800 Batch Loss: 1.235063 Tokens per Sec: 19130, Lr: 0.000210\n", "2020-01-17 23:14:01,402 Epoch 11: total training loss 3108.13\n", "2020-01-17 23:14:01,402 EPOCH 12\n", "2020-01-17 23:14:12,200 Epoch 12 Step: 169900 Batch Loss: 1.350130 Tokens per Sec: 18692, Lr: 0.000210\n", "2020-01-17 23:14:23,703 Epoch 12 Step: 170000 Batch Loss: 0.730165 Tokens per Sec: 19527, Lr: 0.000210\n", "2020-01-17 23:14:52,919 Example #0\n", "2020-01-17 23:14:52,920 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:14:52,920 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:14:52,920 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:14:52,920 Example #1\n", "2020-01-17 23:14:52,921 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:14:52,921 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:14:52,921 \tHypothesis: A te lele onana , Ijo .\n", "2020-01-17 23:14:52,921 Example #2\n", "2020-01-17 23:14:52,922 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:14:52,922 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:14:52,922 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba iru mai te abọ urere ọrọ ole owoma nana , onọ o rẹ kẹ omosasọ gbe omosasọ : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo ẹvi , re ọ ghale ae ; wha jọ ahwo kpobi a wo oghale .\n", "2020-01-17 23:14:52,922 Example #3\n", "2020-01-17 23:14:52,922 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:14:52,923 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:14:52,923 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:14:52,923 Validation result (greedy) at epoch 12, step 170000: bleu: 31.29, loss: 33142.0898, ppl: 3.5908, duration: 29.2194s\n", "2020-01-17 23:15:04,439 Epoch 12 Step: 170100 Batch Loss: 1.244488 Tokens per Sec: 19366, Lr: 0.000210\n", "2020-01-17 23:15:16,041 Epoch 12 Step: 170200 Batch Loss: 1.181341 Tokens per Sec: 19676, Lr: 0.000210\n", "2020-01-17 23:15:27,755 Epoch 12 Step: 170300 Batch Loss: 1.186355 Tokens per Sec: 19055, Lr: 0.000210\n", "2020-01-17 23:15:39,296 Epoch 12 Step: 170400 Batch Loss: 1.054004 Tokens per Sec: 19361, Lr: 0.000210\n", "2020-01-17 23:15:50,695 Epoch 12 Step: 170500 Batch Loss: 1.109508 Tokens per Sec: 19001, Lr: 0.000210\n", "2020-01-17 23:16:02,159 Epoch 12 Step: 170600 Batch Loss: 1.368571 Tokens per Sec: 19326, Lr: 0.000210\n", "2020-01-17 23:16:13,772 Epoch 12 Step: 170700 Batch Loss: 1.189864 Tokens per Sec: 19287, Lr: 0.000210\n", "2020-01-17 23:16:25,172 Epoch 12 Step: 170800 Batch Loss: 0.937714 Tokens per Sec: 19148, Lr: 0.000210\n", "2020-01-17 23:16:36,613 Epoch 12 Step: 170900 Batch Loss: 1.218031 Tokens per Sec: 19246, Lr: 0.000210\n", "2020-01-17 23:16:48,049 Epoch 12 Step: 171000 Batch Loss: 1.283996 Tokens per Sec: 19092, Lr: 0.000210\n", "2020-01-17 23:17:17,213 Example #0\n", "2020-01-17 23:17:17,213 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:17:17,214 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:17:17,214 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ udu efuafo . ’\n", "2020-01-17 23:17:17,214 Example #1\n", "2020-01-17 23:17:17,214 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:17:17,214 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:17:17,214 \tHypothesis: A te lele iei , Ijo .\n", "2020-01-17 23:17:17,215 Example #2\n", "2020-01-17 23:17:17,215 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:17:17,215 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:17:17,215 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ukuhọ riẹ kpohọ abọ urere ọrọ ole owoma nana , onọ o rẹ kẹ omosasọ : “ Jọ odẹ riẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo ojọ ole evawere , re erẹwho kpobi i wo oghale .\n", "2020-01-17 23:17:17,215 Example #3\n", "2020-01-17 23:17:17,216 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:17:17,216 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:17:17,216 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:17:17,216 Validation result (greedy) at epoch 12, step 171000: bleu: 31.31, loss: 33071.9375, ppl: 3.5811, duration: 29.1670s\n", "2020-01-17 23:17:28,762 Epoch 12 Step: 171100 Batch Loss: 1.326737 Tokens per Sec: 19586, Lr: 0.000210\n", "2020-01-17 23:17:40,198 Epoch 12 Step: 171200 Batch Loss: 1.316496 Tokens per Sec: 19092, Lr: 0.000210\n", "2020-01-17 23:17:51,651 Epoch 12 Step: 171300 Batch Loss: 1.374431 Tokens per Sec: 18838, Lr: 0.000210\n", "2020-01-17 23:18:03,123 Epoch 12 Step: 171400 Batch Loss: 1.094428 Tokens per Sec: 19485, Lr: 0.000210\n", "2020-01-17 23:18:14,709 Epoch 12 Step: 171500 Batch Loss: 1.450836 Tokens per Sec: 18899, Lr: 0.000210\n", "2020-01-17 23:18:26,202 Epoch 12 Step: 171600 Batch Loss: 1.095320 Tokens per Sec: 19607, Lr: 0.000210\n", "2020-01-17 23:18:37,649 Epoch 12 Step: 171700 Batch Loss: 1.272929 Tokens per Sec: 19223, Lr: 0.000210\n", "2020-01-17 23:18:49,176 Epoch 12 Step: 171800 Batch Loss: 1.233199 Tokens per Sec: 19728, Lr: 0.000210\n", "2020-01-17 23:19:00,641 Epoch 12 Step: 171900 Batch Loss: 1.363588 Tokens per Sec: 19656, Lr: 0.000210\n", "2020-01-17 23:19:12,219 Epoch 12 Step: 172000 Batch Loss: 1.250034 Tokens per Sec: 19319, Lr: 0.000210\n", "2020-01-17 23:19:41,433 Example #0\n", "2020-01-17 23:19:41,434 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:19:41,434 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:19:41,434 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:19:41,434 Example #1\n", "2020-01-17 23:19:41,435 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:19:41,435 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:19:41,435 \tHypothesis: A te lele Usi Uvie na No .\n", "2020-01-17 23:19:41,435 Example #2\n", "2020-01-17 23:19:41,435 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:19:41,436 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:19:41,436 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹvi mai rite abọ urere ọrọ ole owoma nana , onọ o rẹ kẹ omosasọ : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re o jọ kẹ ae evawere .\n", "2020-01-17 23:19:41,436 Example #3\n", "2020-01-17 23:19:41,436 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:19:41,436 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:19:41,437 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:19:41,437 Validation result (greedy) at epoch 12, step 172000: bleu: 31.31, loss: 33148.0430, ppl: 3.5917, duration: 29.2169s\n", "2020-01-17 23:19:53,002 Epoch 12 Step: 172100 Batch Loss: 1.210062 Tokens per Sec: 19388, Lr: 0.000210\n", "2020-01-17 23:20:04,446 Epoch 12 Step: 172200 Batch Loss: 1.173558 Tokens per Sec: 19785, Lr: 0.000210\n", "2020-01-17 23:20:15,949 Epoch 12 Step: 172300 Batch Loss: 1.132403 Tokens per Sec: 19060, Lr: 0.000210\n", "2020-01-17 23:20:20,522 Epoch 12: total training loss 3101.12\n", "2020-01-17 23:20:20,523 EPOCH 13\n", "2020-01-17 23:20:27,668 Epoch 13 Step: 172400 Batch Loss: 1.229459 Tokens per Sec: 18309, Lr: 0.000210\n", "2020-01-17 23:20:39,297 Epoch 13 Step: 172500 Batch Loss: 1.367717 Tokens per Sec: 19012, Lr: 0.000210\n", "2020-01-17 23:20:50,823 Epoch 13 Step: 172600 Batch Loss: 1.187690 Tokens per Sec: 19298, Lr: 0.000210\n", "2020-01-17 23:21:02,304 Epoch 13 Step: 172700 Batch Loss: 1.103716 Tokens per Sec: 19592, Lr: 0.000210\n", "2020-01-17 23:21:13,849 Epoch 13 Step: 172800 Batch Loss: 1.367110 Tokens per Sec: 19066, Lr: 0.000210\n", "2020-01-17 23:21:25,415 Epoch 13 Step: 172900 Batch Loss: 1.279946 Tokens per Sec: 19498, Lr: 0.000210\n", "2020-01-17 23:21:36,852 Epoch 13 Step: 173000 Batch Loss: 1.337855 Tokens per Sec: 19431, Lr: 0.000210\n", "2020-01-17 23:22:05,907 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:22:05,907 Saving new checkpoint.\n", "2020-01-17 23:22:07,249 Example #0\n", "2020-01-17 23:22:07,250 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:22:07,250 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:22:07,250 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:22:07,250 Example #1\n", "2020-01-17 23:22:07,251 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:22:07,251 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:22:07,251 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:22:07,252 Example #2\n", "2020-01-17 23:22:07,252 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:22:07,253 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:22:07,253 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na , inọ : “ Jọ odẹ [ Jesu Kristi Ovie na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re erẹwho na kpobi e jọ e wo oghale , re o jiri odẹ riẹ ; wha wo oghale , wha wo oghale .\n", "2020-01-17 23:22:07,253 Example #3\n", "2020-01-17 23:22:07,253 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:22:07,253 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:22:07,254 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:22:07,254 Validation result (greedy) at epoch 13, step 173000: bleu: 31.67, loss: 32963.5195, ppl: 3.5662, duration: 30.4013s\n", "2020-01-17 23:22:18,806 Epoch 13 Step: 173100 Batch Loss: 0.836257 Tokens per Sec: 18976, Lr: 0.000210\n", "2020-01-17 23:22:30,434 Epoch 13 Step: 173200 Batch Loss: 1.087598 Tokens per Sec: 18885, Lr: 0.000210\n", "2020-01-17 23:22:42,025 Epoch 13 Step: 173300 Batch Loss: 1.273221 Tokens per Sec: 19622, Lr: 0.000210\n", "2020-01-17 23:22:53,482 Epoch 13 Step: 173400 Batch Loss: 1.184062 Tokens per Sec: 19619, Lr: 0.000210\n", "2020-01-17 23:23:04,938 Epoch 13 Step: 173500 Batch Loss: 1.292021 Tokens per Sec: 19364, Lr: 0.000210\n", "2020-01-17 23:23:16,502 Epoch 13 Step: 173600 Batch Loss: 1.322716 Tokens per Sec: 19239, Lr: 0.000210\n", "2020-01-17 23:23:28,101 Epoch 13 Step: 173700 Batch Loss: 1.320813 Tokens per Sec: 19408, Lr: 0.000210\n", "2020-01-17 23:23:39,560 Epoch 13 Step: 173800 Batch Loss: 1.431084 Tokens per Sec: 19292, Lr: 0.000210\n", "2020-01-17 23:23:51,106 Epoch 13 Step: 173900 Batch Loss: 1.264096 Tokens per Sec: 19515, Lr: 0.000210\n", "2020-01-17 23:24:02,498 Epoch 13 Step: 174000 Batch Loss: 0.848138 Tokens per Sec: 19183, Lr: 0.000210\n", "2020-01-17 23:24:31,672 Example #0\n", "2020-01-17 23:24:31,672 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:24:31,672 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:24:31,673 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:24:31,673 Example #1\n", "2020-01-17 23:24:31,673 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:24:31,673 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:24:31,673 \tHypothesis: Ijo , Usi Uvie na o tẹ te jọ Ijo .\n", "2020-01-17 23:24:31,673 Example #2\n", "2020-01-17 23:24:31,674 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:24:31,674 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:24:31,674 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹvi mai te abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ orọ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo oruaro , re ọ ghale ae ; re o jiri ai kpobi ; wha jọ ahwo Egedhọ a wo oghale .\n", "2020-01-17 23:24:31,674 Example #3\n", "2020-01-17 23:24:31,675 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:24:31,675 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:24:31,675 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:24:31,675 Validation result (greedy) at epoch 13, step 174000: bleu: 31.16, loss: 33111.2578, ppl: 3.5866, duration: 29.1764s\n", "2020-01-17 23:24:43,192 Epoch 13 Step: 174100 Batch Loss: 1.469330 Tokens per Sec: 19469, Lr: 0.000210\n", "2020-01-17 23:24:54,672 Epoch 13 Step: 174200 Batch Loss: 1.270743 Tokens per Sec: 19612, Lr: 0.000210\n", "2020-01-17 23:25:06,270 Epoch 13 Step: 174300 Batch Loss: 1.100757 Tokens per Sec: 19295, Lr: 0.000210\n", "2020-01-17 23:25:17,753 Epoch 13 Step: 174400 Batch Loss: 1.234659 Tokens per Sec: 18857, Lr: 0.000210\n", "2020-01-17 23:25:29,127 Epoch 13 Step: 174500 Batch Loss: 1.411634 Tokens per Sec: 19013, Lr: 0.000210\n", "2020-01-17 23:25:40,707 Epoch 13 Step: 174600 Batch Loss: 1.333072 Tokens per Sec: 19593, Lr: 0.000210\n", "2020-01-17 23:25:52,381 Epoch 13 Step: 174700 Batch Loss: 1.199309 Tokens per Sec: 19335, Lr: 0.000210\n", "2020-01-17 23:26:03,877 Epoch 13 Step: 174800 Batch Loss: 1.367553 Tokens per Sec: 19560, Lr: 0.000210\n", "2020-01-17 23:26:11,454 Epoch 13: total training loss 3087.02\n", "2020-01-17 23:26:11,455 EPOCH 14\n", "2020-01-17 23:26:15,697 Epoch 14 Step: 174900 Batch Loss: 1.354827 Tokens per Sec: 19025, Lr: 0.000210\n", "2020-01-17 23:26:27,182 Epoch 14 Step: 175000 Batch Loss: 1.245415 Tokens per Sec: 19162, Lr: 0.000210\n", "2020-01-17 23:26:56,417 Example #0\n", "2020-01-17 23:26:56,418 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:26:56,418 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:26:56,418 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:26:56,418 Example #1\n", "2020-01-17 23:26:56,419 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:26:56,419 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:26:56,419 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-17 23:26:56,419 Example #2\n", "2020-01-17 23:26:56,420 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:26:56,420 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:26:56,420 \tHypothesis: Koyehọ , mai kpobi ma te rehọ oghọghọ fiba eme urere na evaọ abọ ekuhọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho kpobi e ghale ae ; wha jo wo oghale , re erẹwho na kpobi i wo oghale .\n", "2020-01-17 23:26:56,420 Example #3\n", "2020-01-17 23:26:56,420 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:26:56,421 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:26:56,421 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:26:56,421 Validation result (greedy) at epoch 14, step 175000: bleu: 31.42, loss: 33074.2695, ppl: 3.5814, duration: 29.2386s\n", "2020-01-17 23:27:07,905 Epoch 14 Step: 175100 Batch Loss: 1.289449 Tokens per Sec: 18723, Lr: 0.000210\n", "2020-01-17 23:27:19,394 Epoch 14 Step: 175200 Batch Loss: 1.048058 Tokens per Sec: 19749, Lr: 0.000210\n", "2020-01-17 23:27:30,887 Epoch 14 Step: 175300 Batch Loss: 1.230202 Tokens per Sec: 19436, Lr: 0.000210\n", "2020-01-17 23:27:42,461 Epoch 14 Step: 175400 Batch Loss: 1.269010 Tokens per Sec: 19266, Lr: 0.000210\n", "2020-01-17 23:27:53,794 Epoch 14 Step: 175500 Batch Loss: 1.246533 Tokens per Sec: 19273, Lr: 0.000210\n", "2020-01-17 23:28:05,169 Epoch 14 Step: 175600 Batch Loss: 1.153708 Tokens per Sec: 19190, Lr: 0.000210\n", "2020-01-17 23:28:16,812 Epoch 14 Step: 175700 Batch Loss: 0.867013 Tokens per Sec: 19455, Lr: 0.000210\n", "2020-01-17 23:28:28,241 Epoch 14 Step: 175800 Batch Loss: 1.133702 Tokens per Sec: 19545, Lr: 0.000210\n", "2020-01-17 23:28:39,803 Epoch 14 Step: 175900 Batch Loss: 1.153745 Tokens per Sec: 19648, Lr: 0.000210\n", "2020-01-17 23:28:51,265 Epoch 14 Step: 176000 Batch Loss: 1.266856 Tokens per Sec: 19307, Lr: 0.000210\n", "2020-01-17 23:29:20,366 Example #0\n", "2020-01-17 23:29:20,367 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:29:20,368 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:29:20,368 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:29:20,368 Example #1\n", "2020-01-17 23:29:20,368 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:29:20,368 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:29:20,369 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-17 23:29:20,369 Example #2\n", "2020-01-17 23:29:20,369 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:29:20,369 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:29:20,369 \tHypothesis: Koyehọ , mai kpobi ma te rehọ oghọghọ fiba iru mai te abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo odẹ riẹ , re o wo oghale , re erẹwho kpobi e jọ ahwo nọ a wo oghale ; wha jọ ahwo Egedhọ a wo oghale .\n", "2020-01-17 23:29:20,369 Example #3\n", "2020-01-17 23:29:20,370 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:29:20,370 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:29:20,370 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:29:20,370 Validation result (greedy) at epoch 14, step 176000: bleu: 31.51, loss: 33076.1719, ppl: 3.5817, duration: 29.1053s\n", "2020-01-17 23:29:31,832 Epoch 14 Step: 176100 Batch Loss: 1.211954 Tokens per Sec: 19247, Lr: 0.000210\n", "2020-01-17 23:29:43,324 Epoch 14 Step: 176200 Batch Loss: 1.362894 Tokens per Sec: 18887, Lr: 0.000210\n", "2020-01-17 23:29:54,824 Epoch 14 Step: 176300 Batch Loss: 0.848465 Tokens per Sec: 19605, Lr: 0.000210\n", "2020-01-17 23:30:06,275 Epoch 14 Step: 176400 Batch Loss: 1.289703 Tokens per Sec: 19589, Lr: 0.000210\n", "2020-01-17 23:30:17,839 Epoch 14 Step: 176500 Batch Loss: 1.145749 Tokens per Sec: 19639, Lr: 0.000210\n", "2020-01-17 23:30:29,411 Epoch 14 Step: 176600 Batch Loss: 1.523393 Tokens per Sec: 19398, Lr: 0.000210\n", "2020-01-17 23:30:40,754 Epoch 14 Step: 176700 Batch Loss: 1.166180 Tokens per Sec: 19333, Lr: 0.000210\n", "2020-01-17 23:30:52,195 Epoch 14 Step: 176800 Batch Loss: 1.342311 Tokens per Sec: 19197, Lr: 0.000210\n", "2020-01-17 23:31:03,635 Epoch 14 Step: 176900 Batch Loss: 1.205019 Tokens per Sec: 19148, Lr: 0.000210\n", "2020-01-17 23:31:15,113 Epoch 14 Step: 177000 Batch Loss: 1.373970 Tokens per Sec: 19134, Lr: 0.000210\n", "2020-01-17 23:31:44,281 Example #0\n", "2020-01-17 23:31:44,282 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:31:44,283 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:31:44,283 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:31:44,283 Example #1\n", "2020-01-17 23:31:44,284 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:31:44,284 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:31:44,284 \tHypothesis: Ijo , Usi Uvie na o tẹ wariẹ kuhọ .\n", "2020-01-17 23:31:44,284 Example #2\n", "2020-01-17 23:31:44,285 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:31:44,285 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:31:44,285 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re o wo oghale , re erẹwho kpobi i wo oghale , re o wo oghale .\n", "2020-01-17 23:31:44,285 Example #3\n", "2020-01-17 23:31:44,286 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:31:44,286 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:31:44,286 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:31:44,286 Validation result (greedy) at epoch 14, step 177000: bleu: 31.14, loss: 33007.6055, ppl: 3.5723, duration: 29.1725s\n", "2020-01-17 23:31:55,740 Epoch 14 Step: 177100 Batch Loss: 1.197686 Tokens per Sec: 19544, Lr: 0.000210\n", "2020-01-17 23:32:07,238 Epoch 14 Step: 177200 Batch Loss: 1.381161 Tokens per Sec: 19203, Lr: 0.000210\n", "2020-01-17 23:32:18,716 Epoch 14 Step: 177300 Batch Loss: 1.367085 Tokens per Sec: 19446, Lr: 0.000210\n", "2020-01-17 23:32:29,768 Epoch 14: total training loss 3091.69\n", "2020-01-17 23:32:29,769 EPOCH 15\n", "2020-01-17 23:32:30,363 Epoch 15 Step: 177400 Batch Loss: 1.258168 Tokens per Sec: 10690, Lr: 0.000210\n", "2020-01-17 23:32:41,735 Epoch 15 Step: 177500 Batch Loss: 1.383172 Tokens per Sec: 19394, Lr: 0.000210\n", "2020-01-17 23:32:53,122 Epoch 15 Step: 177600 Batch Loss: 1.100789 Tokens per Sec: 19234, Lr: 0.000210\n", "2020-01-17 23:33:04,622 Epoch 15 Step: 177700 Batch Loss: 1.144062 Tokens per Sec: 19397, Lr: 0.000210\n", "2020-01-17 23:33:16,075 Epoch 15 Step: 177800 Batch Loss: 1.198874 Tokens per Sec: 19186, Lr: 0.000210\n", "2020-01-17 23:33:27,553 Epoch 15 Step: 177900 Batch Loss: 1.242762 Tokens per Sec: 19608, Lr: 0.000210\n", "2020-01-17 23:33:39,095 Epoch 15 Step: 178000 Batch Loss: 1.144183 Tokens per Sec: 19717, Lr: 0.000210\n", "2020-01-17 23:34:08,315 Example #0\n", "2020-01-17 23:34:08,317 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:34:08,319 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:34:08,319 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:34:08,319 Example #1\n", "2020-01-17 23:34:08,320 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:34:08,320 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:34:08,320 \tHypothesis: A te lele onana ẹkwoma Usi Uvie na Ijo .\n", "2020-01-17 23:34:08,320 Example #2\n", "2020-01-17 23:34:08,320 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:34:08,321 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:34:08,321 \tHypothesis: Koyehọ , mai kpobi ma te rehọ oghọghọ fiba eme urere na kpohọ abọ ekuhọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re erẹwho kpobi i wo oghale ; wha jọ erẹwho akpọ na kpobi i wo oghale .\n", "2020-01-17 23:34:08,321 Example #3\n", "2020-01-17 23:34:08,321 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:34:08,321 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:34:08,322 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:34:08,322 Validation result (greedy) at epoch 15, step 178000: bleu: 31.27, loss: 33101.4336, ppl: 3.5852, duration: 29.2267s\n", "2020-01-17 23:34:19,787 Epoch 15 Step: 178100 Batch Loss: 0.982650 Tokens per Sec: 19421, Lr: 0.000210\n", "2020-01-17 23:34:31,368 Epoch 15 Step: 178200 Batch Loss: 1.253147 Tokens per Sec: 19500, Lr: 0.000210\n", "2020-01-17 23:34:42,815 Epoch 15 Step: 178300 Batch Loss: 1.239084 Tokens per Sec: 18996, Lr: 0.000210\n", "2020-01-17 23:34:54,201 Epoch 15 Step: 178400 Batch Loss: 1.028141 Tokens per Sec: 19362, Lr: 0.000210\n", "2020-01-17 23:35:05,598 Epoch 15 Step: 178500 Batch Loss: 1.371735 Tokens per Sec: 19081, Lr: 0.000210\n", "2020-01-17 23:35:17,116 Epoch 15 Step: 178600 Batch Loss: 1.246498 Tokens per Sec: 19351, Lr: 0.000210\n", "2020-01-17 23:35:28,622 Epoch 15 Step: 178700 Batch Loss: 1.373153 Tokens per Sec: 19438, Lr: 0.000210\n", "2020-01-17 23:35:40,137 Epoch 15 Step: 178800 Batch Loss: 1.264148 Tokens per Sec: 19431, Lr: 0.000210\n", "2020-01-17 23:35:51,671 Epoch 15 Step: 178900 Batch Loss: 1.139075 Tokens per Sec: 19572, Lr: 0.000210\n", "2020-01-17 23:36:03,003 Epoch 15 Step: 179000 Batch Loss: 1.305347 Tokens per Sec: 19220, Lr: 0.000210\n", "2020-01-17 23:36:32,216 Example #0\n", "2020-01-17 23:36:32,217 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:36:32,217 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:36:32,217 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:36:32,217 Example #1\n", "2020-01-17 23:36:32,218 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:36:32,218 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:36:32,218 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:36:32,218 Example #2\n", "2020-01-17 23:36:32,219 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:36:32,219 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:36:32,219 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba otofa urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te vihọ , re o wo oghale , re erẹwho kpobi i wo oghale .\n", "2020-01-17 23:36:32,219 Example #3\n", "2020-01-17 23:36:32,219 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:36:32,220 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:36:32,220 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 23:36:32,220 Validation result (greedy) at epoch 15, step 179000: bleu: 31.59, loss: 32988.2188, ppl: 3.5696, duration: 29.2167s\n", "2020-01-17 23:36:43,744 Epoch 15 Step: 179100 Batch Loss: 1.262599 Tokens per Sec: 19411, Lr: 0.000147\n", "2020-01-17 23:36:55,116 Epoch 15 Step: 179200 Batch Loss: 1.037930 Tokens per Sec: 18963, Lr: 0.000147\n", "2020-01-17 23:37:06,757 Epoch 15 Step: 179300 Batch Loss: 1.275629 Tokens per Sec: 19559, Lr: 0.000147\n", "2020-01-17 23:37:18,249 Epoch 15 Step: 179400 Batch Loss: 1.271178 Tokens per Sec: 19485, Lr: 0.000147\n", "2020-01-17 23:37:29,718 Epoch 15 Step: 179500 Batch Loss: 1.127659 Tokens per Sec: 19374, Lr: 0.000147\n", "2020-01-17 23:37:41,186 Epoch 15 Step: 179600 Batch Loss: 1.257067 Tokens per Sec: 19608, Lr: 0.000147\n", "2020-01-17 23:37:52,593 Epoch 15 Step: 179700 Batch Loss: 1.342498 Tokens per Sec: 19264, Lr: 0.000147\n", "2020-01-17 23:38:03,999 Epoch 15 Step: 179800 Batch Loss: 1.389653 Tokens per Sec: 19332, Lr: 0.000147\n", "2020-01-17 23:38:15,633 Epoch 15 Step: 179900 Batch Loss: 1.236038 Tokens per Sec: 19693, Lr: 0.000147\n", "2020-01-17 23:38:18,592 Epoch 15: total training loss 3073.71\n", "2020-01-17 23:38:18,592 EPOCH 16\n", "2020-01-17 23:38:27,282 Epoch 16 Step: 180000 Batch Loss: 1.201871 Tokens per Sec: 18343, Lr: 0.000147\n", "2020-01-17 23:38:56,465 Example #0\n", "2020-01-17 23:38:56,466 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:38:56,466 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:38:56,467 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:38:56,467 Example #1\n", "2020-01-17 23:38:56,467 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:38:56,467 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:38:56,467 \tHypothesis: Ijo , a te lele iei .\n", "2020-01-17 23:38:56,468 Example #2\n", "2020-01-17 23:38:56,468 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:38:56,468 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:38:56,468 \tHypothesis: Koyehọ , mai kpobi ma te rehọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo odẹ riẹ , re o jọ o jọ o jọ o jọ rọ ere , re erẹwho kpobi a wo oghale , re o wo oghale .\n", "2020-01-17 23:38:56,469 Example #3\n", "2020-01-17 23:38:56,469 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:38:56,469 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:38:56,470 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 23:38:56,470 Validation result (greedy) at epoch 16, step 180000: bleu: 31.01, loss: 33004.1875, ppl: 3.5718, duration: 29.1875s\n", "2020-01-17 23:39:07,976 Epoch 16 Step: 180100 Batch Loss: 1.332811 Tokens per Sec: 19081, Lr: 0.000147\n", "2020-01-17 23:39:19,352 Epoch 16 Step: 180200 Batch Loss: 1.106748 Tokens per Sec: 19602, Lr: 0.000147\n", "2020-01-17 23:39:30,832 Epoch 16 Step: 180300 Batch Loss: 0.948929 Tokens per Sec: 19223, Lr: 0.000147\n", "2020-01-17 23:39:42,372 Epoch 16 Step: 180400 Batch Loss: 1.187718 Tokens per Sec: 19598, Lr: 0.000147\n", "2020-01-17 23:39:53,746 Epoch 16 Step: 180500 Batch Loss: 1.287346 Tokens per Sec: 18926, Lr: 0.000147\n", "2020-01-17 23:40:05,118 Epoch 16 Step: 180600 Batch Loss: 1.364193 Tokens per Sec: 19319, Lr: 0.000147\n", "2020-01-17 23:40:16,709 Epoch 16 Step: 180700 Batch Loss: 1.208142 Tokens per Sec: 19617, Lr: 0.000147\n", "2020-01-17 23:40:28,089 Epoch 16 Step: 180800 Batch Loss: 1.084882 Tokens per Sec: 18829, Lr: 0.000147\n", "2020-01-17 23:40:39,463 Epoch 16 Step: 180900 Batch Loss: 1.093226 Tokens per Sec: 19660, Lr: 0.000147\n", "2020-01-17 23:40:50,990 Epoch 16 Step: 181000 Batch Loss: 1.051159 Tokens per Sec: 19387, Lr: 0.000147\n", "2020-01-17 23:41:20,207 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:41:20,207 Saving new checkpoint.\n", "2020-01-17 23:41:21,432 Example #0\n", "2020-01-17 23:41:21,433 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:41:21,433 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:41:21,433 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:41:21,433 Example #1\n", "2020-01-17 23:41:21,434 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:41:21,434 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:41:21,434 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-17 23:41:21,434 Example #2\n", "2020-01-17 23:41:21,435 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:41:21,435 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:41:21,435 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re o wo oghale , re erẹwho kpobi i wo oghale , re o wo oghale .\n", "2020-01-17 23:41:21,435 Example #3\n", "2020-01-17 23:41:21,436 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:41:21,436 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:41:21,436 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:41:21,436 Validation result (greedy) at epoch 16, step 181000: bleu: 31.42, loss: 32881.0625, ppl: 3.5549, duration: 30.4457s\n", "2020-01-17 23:41:33,335 Epoch 16 Step: 181100 Batch Loss: 1.028068 Tokens per Sec: 19335, Lr: 0.000147\n", "2020-01-17 23:41:44,744 Epoch 16 Step: 181200 Batch Loss: 1.409139 Tokens per Sec: 18977, Lr: 0.000147\n", "2020-01-17 23:41:56,128 Epoch 16 Step: 181300 Batch Loss: 1.095228 Tokens per Sec: 19362, Lr: 0.000147\n", "2020-01-17 23:42:07,651 Epoch 16 Step: 181400 Batch Loss: 1.377388 Tokens per Sec: 19480, Lr: 0.000147\n", "2020-01-17 23:42:19,067 Epoch 16 Step: 181500 Batch Loss: 1.169482 Tokens per Sec: 19262, Lr: 0.000147\n", "2020-01-17 23:42:30,590 Epoch 16 Step: 181600 Batch Loss: 1.299935 Tokens per Sec: 19391, Lr: 0.000147\n", "2020-01-17 23:42:42,068 Epoch 16 Step: 181700 Batch Loss: 1.196810 Tokens per Sec: 19544, Lr: 0.000147\n", "2020-01-17 23:42:53,459 Epoch 16 Step: 181800 Batch Loss: 0.966294 Tokens per Sec: 19284, Lr: 0.000147\n", "2020-01-17 23:43:04,990 Epoch 16 Step: 181900 Batch Loss: 1.033572 Tokens per Sec: 20135, Lr: 0.000147\n", "2020-01-17 23:43:16,474 Epoch 16 Step: 182000 Batch Loss: 1.327736 Tokens per Sec: 19161, Lr: 0.000147\n", "2020-01-17 23:43:45,662 Example #0\n", "2020-01-17 23:43:45,663 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:43:45,663 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:43:45,664 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:43:45,664 Example #1\n", "2020-01-17 23:43:45,664 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:43:45,664 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:43:45,664 \tHypothesis: A te lele Usi Uvie na No .\n", "2020-01-17 23:43:45,665 Example #2\n", "2020-01-17 23:43:45,665 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:43:45,665 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:43:45,665 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere na rite abọ ekuhọ ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re ọ ghale ae ; jọ o wo oghale , re erẹwho kpobi i wo oghale .\n", "2020-01-17 23:43:45,665 Example #3\n", "2020-01-17 23:43:45,666 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:43:45,666 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:43:45,666 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:43:45,666 Validation result (greedy) at epoch 16, step 182000: bleu: 31.24, loss: 32908.0156, ppl: 3.5586, duration: 29.1917s\n", "2020-01-17 23:43:57,245 Epoch 16 Step: 182100 Batch Loss: 1.084496 Tokens per Sec: 19549, Lr: 0.000147\n", "2020-01-17 23:44:08,861 Epoch 16 Step: 182200 Batch Loss: 1.126654 Tokens per Sec: 19135, Lr: 0.000147\n", "2020-01-17 23:44:20,298 Epoch 16 Step: 182300 Batch Loss: 1.485939 Tokens per Sec: 19778, Lr: 0.000147\n", "2020-01-17 23:44:31,743 Epoch 16 Step: 182400 Batch Loss: 1.401012 Tokens per Sec: 18995, Lr: 0.000147\n", "2020-01-17 23:44:38,297 Epoch 16: total training loss 3044.06\n", "2020-01-17 23:44:38,298 EPOCH 17\n", "2020-01-17 23:44:43,306 Epoch 17 Step: 182500 Batch Loss: 1.395236 Tokens per Sec: 17524, Lr: 0.000147\n", "2020-01-17 23:44:54,797 Epoch 17 Step: 182600 Batch Loss: 1.407193 Tokens per Sec: 19265, Lr: 0.000147\n", "2020-01-17 23:45:06,440 Epoch 17 Step: 182700 Batch Loss: 1.069616 Tokens per Sec: 19649, Lr: 0.000147\n", "2020-01-17 23:45:17,940 Epoch 17 Step: 182800 Batch Loss: 1.149178 Tokens per Sec: 19359, Lr: 0.000147\n", "2020-01-17 23:45:29,330 Epoch 17 Step: 182900 Batch Loss: 1.239136 Tokens per Sec: 19327, Lr: 0.000147\n", "2020-01-17 23:45:40,779 Epoch 17 Step: 183000 Batch Loss: 1.144313 Tokens per Sec: 19532, Lr: 0.000147\n", "2020-01-17 23:46:09,993 Example #0\n", "2020-01-17 23:46:09,993 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:46:09,994 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:46:09,994 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:46:09,994 Example #1\n", "2020-01-17 23:46:09,994 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:46:09,994 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:46:09,995 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:46:09,995 Example #2\n", "2020-01-17 23:46:09,995 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:46:09,995 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:46:09,995 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere na kpohọ abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo odẹ riẹ , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale .\n", "2020-01-17 23:46:09,996 Example #3\n", "2020-01-17 23:46:09,996 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:46:09,996 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:46:09,996 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:46:09,996 Validation result (greedy) at epoch 17, step 183000: bleu: 31.49, loss: 32924.6875, ppl: 3.5608, duration: 29.2166s\n", "2020-01-17 23:46:21,491 Epoch 17 Step: 183100 Batch Loss: 1.108262 Tokens per Sec: 19569, Lr: 0.000147\n", "2020-01-17 23:46:33,085 Epoch 17 Step: 183200 Batch Loss: 1.298536 Tokens per Sec: 19254, Lr: 0.000147\n", "2020-01-17 23:46:44,564 Epoch 17 Step: 183300 Batch Loss: 1.164132 Tokens per Sec: 19314, Lr: 0.000147\n", "2020-01-17 23:46:55,886 Epoch 17 Step: 183400 Batch Loss: 0.961109 Tokens per Sec: 19058, Lr: 0.000147\n", "2020-01-17 23:47:07,349 Epoch 17 Step: 183500 Batch Loss: 1.244375 Tokens per Sec: 19394, Lr: 0.000147\n", "2020-01-17 23:47:18,886 Epoch 17 Step: 183600 Batch Loss: 1.194736 Tokens per Sec: 20004, Lr: 0.000147\n", "2020-01-17 23:47:30,342 Epoch 17 Step: 183700 Batch Loss: 0.862820 Tokens per Sec: 19440, Lr: 0.000147\n", "2020-01-17 23:47:41,746 Epoch 17 Step: 183800 Batch Loss: 1.205523 Tokens per Sec: 18882, Lr: 0.000147\n", "2020-01-17 23:47:53,180 Epoch 17 Step: 183900 Batch Loss: 1.224223 Tokens per Sec: 19356, Lr: 0.000147\n", "2020-01-17 23:48:04,621 Epoch 17 Step: 184000 Batch Loss: 1.469756 Tokens per Sec: 19444, Lr: 0.000147\n", "2020-01-17 23:48:33,682 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:48:33,682 Saving new checkpoint.\n", "2020-01-17 23:48:34,889 Example #0\n", "2020-01-17 23:48:34,890 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:48:34,890 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:48:34,890 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:48:34,891 Example #1\n", "2020-01-17 23:48:34,891 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:48:34,891 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:48:34,891 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:48:34,891 Example #2\n", "2020-01-17 23:48:34,892 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:48:34,892 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:48:34,892 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo odẹ riẹ , re ọ ghale ae ; jọ o jiri ai kpobi ; wha wo oghale .\n", "2020-01-17 23:48:34,892 Example #3\n", "2020-01-17 23:48:34,893 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:48:34,893 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:48:34,893 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-17 23:48:34,893 Validation result (greedy) at epoch 17, step 184000: bleu: 31.21, loss: 32812.7852, ppl: 3.5455, duration: 30.2716s\n", "2020-01-17 23:48:46,635 Epoch 17 Step: 184100 Batch Loss: 1.011200 Tokens per Sec: 19009, Lr: 0.000147\n", "2020-01-17 23:48:58,238 Epoch 17 Step: 184200 Batch Loss: 1.277999 Tokens per Sec: 19631, Lr: 0.000147\n", "2020-01-17 23:49:09,838 Epoch 17 Step: 184300 Batch Loss: 1.290747 Tokens per Sec: 19018, Lr: 0.000147\n", "2020-01-17 23:49:21,265 Epoch 17 Step: 184400 Batch Loss: 1.252911 Tokens per Sec: 19576, Lr: 0.000147\n", "2020-01-17 23:49:32,677 Epoch 17 Step: 184500 Batch Loss: 1.317471 Tokens per Sec: 19277, Lr: 0.000147\n", "2020-01-17 23:49:44,172 Epoch 17 Step: 184600 Batch Loss: 1.375715 Tokens per Sec: 19038, Lr: 0.000147\n", "2020-01-17 23:49:55,658 Epoch 17 Step: 184700 Batch Loss: 1.308747 Tokens per Sec: 20085, Lr: 0.000147\n", "2020-01-17 23:50:07,162 Epoch 17 Step: 184800 Batch Loss: 1.326169 Tokens per Sec: 19306, Lr: 0.000147\n", "2020-01-17 23:50:18,598 Epoch 17 Step: 184900 Batch Loss: 1.312071 Tokens per Sec: 19509, Lr: 0.000147\n", "2020-01-17 23:50:28,352 Epoch 17: total training loss 3027.05\n", "2020-01-17 23:50:28,353 EPOCH 18\n", "2020-01-17 23:50:30,405 Epoch 18 Step: 185000 Batch Loss: 1.057760 Tokens per Sec: 16325, Lr: 0.000147\n", "2020-01-17 23:50:59,499 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:50:59,500 Saving new checkpoint.\n", "2020-01-17 23:51:00,676 Example #0\n", "2020-01-17 23:51:00,677 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:51:00,677 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:51:00,677 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:51:00,678 Example #1\n", "2020-01-17 23:51:00,678 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:51:00,678 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:51:00,678 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:51:00,678 Example #2\n", "2020-01-17 23:51:00,680 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:51:00,680 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:51:00,680 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Odẹ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te wo odẹ riẹ , re ọ ghale ae ; re erẹwho kpobi i wo oghale .\n", "2020-01-17 23:51:00,680 Example #3\n", "2020-01-17 23:51:00,681 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:51:00,681 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:51:00,681 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 23:51:00,681 Validation result (greedy) at epoch 18, step 185000: bleu: 31.33, loss: 32739.6797, ppl: 3.5355, duration: 30.2760s\n", "2020-01-17 23:51:12,553 Epoch 18 Step: 185100 Batch Loss: 1.309870 Tokens per Sec: 19116, Lr: 0.000147\n", "2020-01-17 23:51:23,997 Epoch 18 Step: 185200 Batch Loss: 0.770134 Tokens per Sec: 19558, Lr: 0.000147\n", "2020-01-17 23:51:35,547 Epoch 18 Step: 185300 Batch Loss: 1.102031 Tokens per Sec: 19348, Lr: 0.000147\n", "2020-01-17 23:51:47,218 Epoch 18 Step: 185400 Batch Loss: 0.993282 Tokens per Sec: 19452, Lr: 0.000147\n", "2020-01-17 23:51:58,560 Epoch 18 Step: 185500 Batch Loss: 1.330352 Tokens per Sec: 19384, Lr: 0.000147\n", "2020-01-17 23:52:10,021 Epoch 18 Step: 185600 Batch Loss: 1.067499 Tokens per Sec: 19354, Lr: 0.000147\n", "2020-01-17 23:52:21,353 Epoch 18 Step: 185700 Batch Loss: 1.245664 Tokens per Sec: 19061, Lr: 0.000147\n", "2020-01-17 23:52:32,753 Epoch 18 Step: 185800 Batch Loss: 0.966925 Tokens per Sec: 19349, Lr: 0.000147\n", "2020-01-17 23:52:44,288 Epoch 18 Step: 185900 Batch Loss: 1.223407 Tokens per Sec: 19593, Lr: 0.000147\n", "2020-01-17 23:52:55,822 Epoch 18 Step: 186000 Batch Loss: 1.285505 Tokens per Sec: 19799, Lr: 0.000147\n", "2020-01-17 23:53:24,847 Example #0\n", "2020-01-17 23:53:24,848 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:53:24,848 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:53:24,849 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:53:24,849 Example #1\n", "2020-01-17 23:53:24,849 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:53:24,849 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:53:24,849 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:53:24,849 Example #2\n", "2020-01-17 23:53:24,850 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:53:24,850 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:53:24,850 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te wo odẹ riẹ , re ọ ghale ae ; jọ o jọ ere kẹ erẹwho kpobi ; wha wo oghale .\n", "2020-01-17 23:53:24,850 Example #3\n", "2020-01-17 23:53:24,851 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:53:24,851 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:53:24,851 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 23:53:24,851 Validation result (greedy) at epoch 18, step 186000: bleu: 31.49, loss: 32805.1719, ppl: 3.5445, duration: 29.0289s\n", "2020-01-17 23:53:36,169 Epoch 18 Step: 186100 Batch Loss: 1.168050 Tokens per Sec: 19079, Lr: 0.000147\n", "2020-01-17 23:53:47,579 Epoch 18 Step: 186200 Batch Loss: 0.950215 Tokens per Sec: 19263, Lr: 0.000147\n", "2020-01-17 23:53:59,098 Epoch 18 Step: 186300 Batch Loss: 1.324398 Tokens per Sec: 19789, Lr: 0.000147\n", "2020-01-17 23:54:10,593 Epoch 18 Step: 186400 Batch Loss: 1.364364 Tokens per Sec: 19223, Lr: 0.000147\n", "2020-01-17 23:54:22,031 Epoch 18 Step: 186500 Batch Loss: 1.240694 Tokens per Sec: 19661, Lr: 0.000147\n", "2020-01-17 23:54:33,442 Epoch 18 Step: 186600 Batch Loss: 1.047631 Tokens per Sec: 19258, Lr: 0.000147\n", "2020-01-17 23:54:44,819 Epoch 18 Step: 186700 Batch Loss: 1.105826 Tokens per Sec: 19410, Lr: 0.000147\n", "2020-01-17 23:54:56,139 Epoch 18 Step: 186800 Batch Loss: 1.188432 Tokens per Sec: 19551, Lr: 0.000147\n", "2020-01-17 23:55:07,675 Epoch 18 Step: 186900 Batch Loss: 1.308543 Tokens per Sec: 19420, Lr: 0.000147\n", "2020-01-17 23:55:19,136 Epoch 18 Step: 187000 Batch Loss: 1.306616 Tokens per Sec: 19840, Lr: 0.000147\n", "2020-01-17 23:55:48,231 Hooray! New best validation result [ppl]!\n", "2020-01-17 23:55:48,231 Saving new checkpoint.\n", "2020-01-17 23:55:49,821 Example #0\n", "2020-01-17 23:55:49,822 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:55:49,822 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:55:49,822 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:55:49,822 Example #1\n", "2020-01-17 23:55:49,823 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:55:49,823 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:55:49,823 \tHypothesis: A te lele Usi Uvie na No .\n", "2020-01-17 23:55:49,823 Example #2\n", "2020-01-17 23:55:49,824 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:55:49,824 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:55:49,824 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te wo oruaro , re evaọ aro riẹ , re ọ ghale ae ; re erẹwho na kpobi i wo oghale .\n", "2020-01-17 23:55:49,824 Example #3\n", "2020-01-17 23:55:49,825 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:55:49,825 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:55:49,825 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ ma re ro ru udhedhẹ .\n", "2020-01-17 23:55:49,825 Validation result (greedy) at epoch 18, step 187000: bleu: 31.45, loss: 32724.6309, ppl: 3.5335, duration: 30.6887s\n", "2020-01-17 23:56:01,386 Epoch 18 Step: 187100 Batch Loss: 1.159979 Tokens per Sec: 18682, Lr: 0.000147\n", "2020-01-17 23:56:13,114 Epoch 18 Step: 187200 Batch Loss: 1.134557 Tokens per Sec: 18994, Lr: 0.000147\n", "2020-01-17 23:56:24,624 Epoch 18 Step: 187300 Batch Loss: 1.293951 Tokens per Sec: 19712, Lr: 0.000147\n", "2020-01-17 23:56:35,899 Epoch 18 Step: 187400 Batch Loss: 1.211888 Tokens per Sec: 18872, Lr: 0.000147\n", "2020-01-17 23:56:47,421 Epoch 18 Step: 187500 Batch Loss: 1.090710 Tokens per Sec: 19365, Lr: 0.000147\n", "2020-01-17 23:56:48,999 Epoch 18: total training loss 3023.32\n", "2020-01-17 23:56:49,000 EPOCH 19\n", "2020-01-17 23:56:59,143 Epoch 19 Step: 187600 Batch Loss: 1.257462 Tokens per Sec: 18942, Lr: 0.000147\n", "2020-01-17 23:57:10,634 Epoch 19 Step: 187700 Batch Loss: 1.045886 Tokens per Sec: 19108, Lr: 0.000147\n", "2020-01-17 23:57:22,112 Epoch 19 Step: 187800 Batch Loss: 1.268106 Tokens per Sec: 19595, Lr: 0.000147\n", "2020-01-17 23:57:33,556 Epoch 19 Step: 187900 Batch Loss: 1.356215 Tokens per Sec: 18926, Lr: 0.000147\n", "2020-01-17 23:57:45,105 Epoch 19 Step: 188000 Batch Loss: 1.178874 Tokens per Sec: 19761, Lr: 0.000147\n", "2020-01-17 23:58:14,245 Example #0\n", "2020-01-17 23:58:14,246 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-17 23:58:14,246 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-17 23:58:14,246 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-17 23:58:14,246 Example #1\n", "2020-01-17 23:58:14,247 \tSource: This was followed by Kingdom News No .\n", "2020-01-17 23:58:14,247 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-17 23:58:14,247 \tHypothesis: A te lele Usi Uvie na , Ijo .\n", "2020-01-17 23:58:14,247 Example #2\n", "2020-01-17 23:58:14,247 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-17 23:58:14,248 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-17 23:58:14,248 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹjiroro urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te wo oruaro , re ọ ghale ae , re erẹwho na kpobi i wo oghale ; jọ o se ai kpobi .\n", "2020-01-17 23:58:14,248 Example #3\n", "2020-01-17 23:58:14,248 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-17 23:58:14,248 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-17 23:58:14,249 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-17 23:58:14,249 Validation result (greedy) at epoch 19, step 188000: bleu: 31.76, loss: 32821.3477, ppl: 3.5467, duration: 29.1429s\n", "2020-01-17 23:58:25,747 Epoch 19 Step: 188100 Batch Loss: 1.364902 Tokens per Sec: 19560, Lr: 0.000147\n", "2020-01-17 23:58:37,253 Epoch 19 Step: 188200 Batch Loss: 1.129950 Tokens per Sec: 19759, Lr: 0.000147\n", "2020-01-17 23:58:48,673 Epoch 19 Step: 188300 Batch Loss: 1.314671 Tokens per Sec: 19105, Lr: 0.000147\n", "2020-01-17 23:59:00,107 Epoch 19 Step: 188400 Batch Loss: 1.393379 Tokens per Sec: 19300, Lr: 0.000147\n", "2020-01-17 23:59:11,662 Epoch 19 Step: 188500 Batch Loss: 1.144819 Tokens per Sec: 19055, Lr: 0.000147\n", "2020-01-17 23:59:23,007 Epoch 19 Step: 188600 Batch Loss: 1.146836 Tokens per Sec: 19466, Lr: 0.000147\n", "2020-01-17 23:59:34,513 Epoch 19 Step: 188700 Batch Loss: 0.832289 Tokens per Sec: 19209, Lr: 0.000147\n", "2020-01-17 23:59:45,982 Epoch 19 Step: 188800 Batch Loss: 1.244966 Tokens per Sec: 19551, Lr: 0.000147\n", "2020-01-17 23:59:57,450 Epoch 19 Step: 188900 Batch Loss: 1.248454 Tokens per Sec: 19289, Lr: 0.000147\n", "2020-01-18 00:00:09,149 Epoch 19 Step: 189000 Batch Loss: 1.353044 Tokens per Sec: 19612, Lr: 0.000147\n", "2020-01-18 00:00:38,272 Example #0\n", "2020-01-18 00:00:38,273 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:00:38,273 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:00:38,274 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:00:38,274 Example #1\n", "2020-01-18 00:00:38,274 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:00:38,274 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:00:38,274 \tHypothesis: A te lele Usi Uvie Na , Ọrọ Avọ Ọrọ Avọ 4\n", "2020-01-18 00:00:38,275 Example #2\n", "2020-01-18 00:00:38,275 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:00:38,275 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:00:38,275 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo via eva riẹ , re o je jiri odẹ riẹ , re erẹwho na kpobi i wo oghale .\n", "2020-01-18 00:00:38,275 Example #3\n", "2020-01-18 00:00:38,276 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:00:38,276 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:00:38,276 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:00:38,276 Validation result (greedy) at epoch 19, step 189000: bleu: 31.41, loss: 32760.1855, ppl: 3.5383, duration: 29.1268s\n", "2020-01-18 00:00:49,646 Epoch 19 Step: 189100 Batch Loss: 1.231762 Tokens per Sec: 19360, Lr: 0.000147\n", "2020-01-18 00:01:01,096 Epoch 19 Step: 189200 Batch Loss: 1.239738 Tokens per Sec: 19763, Lr: 0.000147\n", "2020-01-18 00:01:12,762 Epoch 19 Step: 189300 Batch Loss: 1.395421 Tokens per Sec: 19401, Lr: 0.000147\n", "2020-01-18 00:01:24,232 Epoch 19 Step: 189400 Batch Loss: 1.149854 Tokens per Sec: 19433, Lr: 0.000147\n", "2020-01-18 00:01:35,616 Epoch 19 Step: 189500 Batch Loss: 1.256537 Tokens per Sec: 19062, Lr: 0.000147\n", "2020-01-18 00:01:47,075 Epoch 19 Step: 189600 Batch Loss: 1.226351 Tokens per Sec: 19248, Lr: 0.000147\n", "2020-01-18 00:01:58,633 Epoch 19 Step: 189700 Batch Loss: 1.223882 Tokens per Sec: 19452, Lr: 0.000147\n", "2020-01-18 00:02:10,149 Epoch 19 Step: 189800 Batch Loss: 1.138501 Tokens per Sec: 19378, Lr: 0.000147\n", "2020-01-18 00:02:21,578 Epoch 19 Step: 189900 Batch Loss: 1.196659 Tokens per Sec: 19208, Lr: 0.000147\n", "2020-01-18 00:02:33,216 Epoch 19 Step: 190000 Batch Loss: 1.120179 Tokens per Sec: 19783, Lr: 0.000147\n", "2020-01-18 00:03:02,334 Example #0\n", "2020-01-18 00:03:02,335 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:03:02,335 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:03:02,335 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:03:02,335 Example #1\n", "2020-01-18 00:03:02,336 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:03:02,336 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:03:02,336 \tHypothesis: A te je ru onana avọ Usi Uvie na , Ọrọ Avọ 4 .\n", "2020-01-18 00:03:02,336 Example #2\n", "2020-01-18 00:03:02,337 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:03:02,337 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:03:02,338 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ekuhọ ọrọ abọ ekuhọ ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 00:03:02,338 Example #3\n", "2020-01-18 00:03:02,338 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:03:02,339 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:03:02,339 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:03:02,339 Validation result (greedy) at epoch 19, step 190000: bleu: 31.82, loss: 32747.7246, ppl: 3.5366, duration: 29.1225s\n", "2020-01-18 00:03:06,656 Epoch 19: total training loss 3010.53\n", "2020-01-18 00:03:06,657 EPOCH 20\n", "2020-01-18 00:03:14,145 Epoch 20 Step: 190100 Batch Loss: 1.244693 Tokens per Sec: 18346, Lr: 0.000147\n", "2020-01-18 00:03:25,721 Epoch 20 Step: 190200 Batch Loss: 1.044246 Tokens per Sec: 19642, Lr: 0.000147\n", "2020-01-18 00:03:37,127 Epoch 20 Step: 190300 Batch Loss: 1.062461 Tokens per Sec: 19388, Lr: 0.000147\n", "2020-01-18 00:03:48,651 Epoch 20 Step: 190400 Batch Loss: 1.357498 Tokens per Sec: 19632, Lr: 0.000147\n", "2020-01-18 00:04:00,085 Epoch 20 Step: 190500 Batch Loss: 1.077049 Tokens per Sec: 19313, Lr: 0.000147\n", "2020-01-18 00:04:11,618 Epoch 20 Step: 190600 Batch Loss: 1.137801 Tokens per Sec: 19591, Lr: 0.000147\n", "2020-01-18 00:04:23,101 Epoch 20 Step: 190700 Batch Loss: 1.282429 Tokens per Sec: 19449, Lr: 0.000147\n", "2020-01-18 00:04:34,554 Epoch 20 Step: 190800 Batch Loss: 1.028485 Tokens per Sec: 19412, Lr: 0.000147\n", "2020-01-18 00:04:46,031 Epoch 20 Step: 190900 Batch Loss: 1.272597 Tokens per Sec: 19204, Lr: 0.000147\n", "2020-01-18 00:04:57,397 Epoch 20 Step: 191000 Batch Loss: 1.113578 Tokens per Sec: 19211, Lr: 0.000147\n", "2020-01-18 00:05:26,747 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:05:26,747 Saving new checkpoint.\n", "2020-01-18 00:05:27,958 Example #0\n", "2020-01-18 00:05:27,958 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:05:27,959 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:05:27,959 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:05:27,959 Example #1\n", "2020-01-18 00:05:27,959 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:05:27,960 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:05:27,960 \tHypothesis: Ijo , Usi Uvie na o tẹ wariẹ muhọ .\n", "2020-01-18 00:05:27,960 Example #2\n", "2020-01-18 00:05:27,960 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:05:27,961 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:05:27,961 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ekuhọ ọrọ abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre na o te lo , re ọre o te lo via , re o te ti vihọ , re ahwo kpobi a jiri ei ; wha jọ ahwo Egedhọ a wo oghale .\n", "2020-01-18 00:05:27,961 Example #3\n", "2020-01-18 00:05:27,961 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:05:27,962 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:05:27,962 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:05:27,962 Validation result (greedy) at epoch 20, step 191000: bleu: 31.89, loss: 32657.4336, ppl: 3.5243, duration: 30.5649s\n", "2020-01-18 00:05:39,633 Epoch 20 Step: 191100 Batch Loss: 1.286265 Tokens per Sec: 19002, Lr: 0.000147\n", "2020-01-18 00:05:51,030 Epoch 20 Step: 191200 Batch Loss: 1.150373 Tokens per Sec: 19608, Lr: 0.000147\n", "2020-01-18 00:06:02,507 Epoch 20 Step: 191300 Batch Loss: 1.181873 Tokens per Sec: 19478, Lr: 0.000147\n", "2020-01-18 00:06:14,063 Epoch 20 Step: 191400 Batch Loss: 1.455721 Tokens per Sec: 19307, Lr: 0.000147\n", "2020-01-18 00:06:25,563 Epoch 20 Step: 191500 Batch Loss: 1.264992 Tokens per Sec: 19133, Lr: 0.000147\n", "2020-01-18 00:06:36,981 Epoch 20 Step: 191600 Batch Loss: 1.040658 Tokens per Sec: 19276, Lr: 0.000147\n", "2020-01-18 00:06:48,430 Epoch 20 Step: 191700 Batch Loss: 1.379192 Tokens per Sec: 19111, Lr: 0.000147\n", "2020-01-18 00:06:59,843 Epoch 20 Step: 191800 Batch Loss: 1.170561 Tokens per Sec: 19382, Lr: 0.000147\n", "2020-01-18 00:07:11,554 Epoch 20 Step: 191900 Batch Loss: 1.230094 Tokens per Sec: 19150, Lr: 0.000147\n", "2020-01-18 00:07:23,027 Epoch 20 Step: 192000 Batch Loss: 1.337745 Tokens per Sec: 19751, Lr: 0.000147\n", "2020-01-18 00:07:52,145 Example #0\n", "2020-01-18 00:07:52,146 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:07:52,146 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:07:52,146 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:07:52,146 Example #1\n", "2020-01-18 00:07:52,147 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:07:52,147 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:07:52,147 \tHypothesis: A te lele onana Usi Uvie na , Ijo .\n", "2020-01-18 00:07:52,147 Example #2\n", "2020-01-18 00:07:52,148 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:07:52,148 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:07:52,148 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oghale , re erẹwho kpobi i wo oghale .\n", "2020-01-18 00:07:52,148 Example #3\n", "2020-01-18 00:07:52,149 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:07:52,149 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:07:52,149 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:07:52,149 Validation result (greedy) at epoch 20, step 192000: bleu: 31.20, loss: 32732.0781, ppl: 3.5345, duration: 29.1222s\n", "2020-01-18 00:08:03,597 Epoch 20 Step: 192100 Batch Loss: 1.254797 Tokens per Sec: 19543, Lr: 0.000147\n", "2020-01-18 00:08:15,139 Epoch 20 Step: 192200 Batch Loss: 1.154180 Tokens per Sec: 19166, Lr: 0.000147\n", "2020-01-18 00:08:26,614 Epoch 20 Step: 192300 Batch Loss: 0.989972 Tokens per Sec: 19387, Lr: 0.000147\n", "2020-01-18 00:08:38,034 Epoch 20 Step: 192400 Batch Loss: 1.346615 Tokens per Sec: 19792, Lr: 0.000147\n", "2020-01-18 00:08:49,510 Epoch 20 Step: 192500 Batch Loss: 1.394684 Tokens per Sec: 18995, Lr: 0.000147\n", "2020-01-18 00:08:56,975 Epoch 20: total training loss 3010.24\n", "2020-01-18 00:08:56,975 EPOCH 21\n", "2020-01-18 00:09:01,248 Epoch 21 Step: 192600 Batch Loss: 1.386908 Tokens per Sec: 18861, Lr: 0.000147\n", "2020-01-18 00:09:12,765 Epoch 21 Step: 192700 Batch Loss: 1.245965 Tokens per Sec: 18900, Lr: 0.000147\n", "2020-01-18 00:09:24,240 Epoch 21 Step: 192800 Batch Loss: 1.108674 Tokens per Sec: 19431, Lr: 0.000147\n", "2020-01-18 00:09:35,825 Epoch 21 Step: 192900 Batch Loss: 1.053933 Tokens per Sec: 19607, Lr: 0.000147\n", "2020-01-18 00:09:47,331 Epoch 21 Step: 193000 Batch Loss: 1.263215 Tokens per Sec: 19323, Lr: 0.000147\n", "2020-01-18 00:10:16,449 Example #0\n", "2020-01-18 00:10:16,450 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:10:16,450 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:10:16,450 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:10:16,450 Example #1\n", "2020-01-18 00:10:16,451 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:10:16,451 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:10:16,451 \tHypothesis: Ijo , a te lele iei , Usi Uvie na o tẹ wariẹ muhọ .\n", "2020-01-18 00:10:16,451 Example #2\n", "2020-01-18 00:10:16,452 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:10:16,452 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:10:16,452 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere na kpohọ abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oghale eva rai , re erẹwho kpobi i wo oghale .\n", "2020-01-18 00:10:16,452 Example #3\n", "2020-01-18 00:10:16,452 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:10:16,453 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:10:16,453 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:10:16,453 Validation result (greedy) at epoch 21, step 193000: bleu: 31.56, loss: 32709.1387, ppl: 3.5314, duration: 29.1212s\n", "2020-01-18 00:10:27,974 Epoch 21 Step: 193100 Batch Loss: 1.265232 Tokens per Sec: 19194, Lr: 0.000147\n", "2020-01-18 00:10:39,422 Epoch 21 Step: 193200 Batch Loss: 0.925502 Tokens per Sec: 19005, Lr: 0.000147\n", "2020-01-18 00:10:50,906 Epoch 21 Step: 193300 Batch Loss: 1.109345 Tokens per Sec: 19302, Lr: 0.000147\n", "2020-01-18 00:11:02,332 Epoch 21 Step: 193400 Batch Loss: 1.011814 Tokens per Sec: 19699, Lr: 0.000147\n", "2020-01-18 00:11:13,893 Epoch 21 Step: 193500 Batch Loss: 1.259112 Tokens per Sec: 19062, Lr: 0.000147\n", "2020-01-18 00:11:25,390 Epoch 21 Step: 193600 Batch Loss: 1.294667 Tokens per Sec: 19497, Lr: 0.000147\n", "2020-01-18 00:11:36,968 Epoch 21 Step: 193700 Batch Loss: 1.073016 Tokens per Sec: 19870, Lr: 0.000147\n", "2020-01-18 00:11:48,589 Epoch 21 Step: 193800 Batch Loss: 1.137770 Tokens per Sec: 19643, Lr: 0.000147\n", "2020-01-18 00:12:00,076 Epoch 21 Step: 193900 Batch Loss: 1.277497 Tokens per Sec: 19464, Lr: 0.000147\n", "2020-01-18 00:12:11,536 Epoch 21 Step: 194000 Batch Loss: 1.271874 Tokens per Sec: 19324, Lr: 0.000147\n", "2020-01-18 00:12:40,722 Example #0\n", "2020-01-18 00:12:40,723 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:12:40,723 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:12:40,724 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:12:40,724 Example #1\n", "2020-01-18 00:12:40,724 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:12:40,724 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:12:40,725 \tHypothesis: Ijo , a te lele iei , Ijo .\n", "2020-01-18 00:12:40,725 Example #2\n", "2020-01-18 00:12:40,725 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:12:40,726 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:12:40,726 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jiri ahwo Egedhọ kpobi ; wha wo oghale .\n", "2020-01-18 00:12:40,726 Example #3\n", "2020-01-18 00:12:40,727 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:12:40,727 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:12:40,728 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:12:40,728 Validation result (greedy) at epoch 21, step 194000: bleu: 31.60, loss: 32802.2617, ppl: 3.5441, duration: 29.1916s\n", "2020-01-18 00:12:52,269 Epoch 21 Step: 194100 Batch Loss: 1.106887 Tokens per Sec: 19475, Lr: 0.000147\n", "2020-01-18 00:13:03,719 Epoch 21 Step: 194200 Batch Loss: 0.991459 Tokens per Sec: 19504, Lr: 0.000147\n", "2020-01-18 00:13:15,288 Epoch 21 Step: 194300 Batch Loss: 1.099398 Tokens per Sec: 19073, Lr: 0.000147\n", "2020-01-18 00:13:26,752 Epoch 21 Step: 194400 Batch Loss: 1.187212 Tokens per Sec: 19094, Lr: 0.000147\n", "2020-01-18 00:13:38,102 Epoch 21 Step: 194500 Batch Loss: 1.081608 Tokens per Sec: 19372, Lr: 0.000147\n", "2020-01-18 00:13:49,653 Epoch 21 Step: 194600 Batch Loss: 1.359140 Tokens per Sec: 19537, Lr: 0.000147\n", "2020-01-18 00:14:01,074 Epoch 21 Step: 194700 Batch Loss: 1.354969 Tokens per Sec: 19658, Lr: 0.000147\n", "2020-01-18 00:14:12,620 Epoch 21 Step: 194800 Batch Loss: 1.064215 Tokens per Sec: 19195, Lr: 0.000147\n", "2020-01-18 00:14:24,106 Epoch 21 Step: 194900 Batch Loss: 1.091578 Tokens per Sec: 19541, Lr: 0.000147\n", "2020-01-18 00:14:35,576 Epoch 21 Step: 195000 Batch Loss: 1.148316 Tokens per Sec: 19191, Lr: 0.000147\n", "2020-01-18 00:15:04,628 Example #0\n", "2020-01-18 00:15:04,629 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:15:04,629 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:15:04,629 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:15:04,629 Example #1\n", "2020-01-18 00:15:04,630 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:15:04,630 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:15:04,630 \tHypothesis: Ijo , a te lele iei , Ijo .\n", "2020-01-18 00:15:04,630 Example #2\n", "2020-01-18 00:15:04,631 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:15:04,631 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:15:04,631 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jiri ahwo Egedhọ kpobi ; wha kẹ ae uyere .\n", "2020-01-18 00:15:04,631 Example #3\n", "2020-01-18 00:15:04,631 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:15:04,632 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:15:04,632 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:15:04,632 Validation result (greedy) at epoch 21, step 195000: bleu: 31.61, loss: 32665.6680, ppl: 3.5254, duration: 29.0553s\n", "2020-01-18 00:15:14,887 Epoch 21: total training loss 3000.15\n", "2020-01-18 00:15:14,888 EPOCH 22\n", "2020-01-18 00:15:16,389 Epoch 22 Step: 195100 Batch Loss: 1.249880 Tokens per Sec: 16553, Lr: 0.000147\n", "2020-01-18 00:15:27,925 Epoch 22 Step: 195200 Batch Loss: 1.272563 Tokens per Sec: 19962, Lr: 0.000147\n", "2020-01-18 00:15:39,388 Epoch 22 Step: 195300 Batch Loss: 1.372898 Tokens per Sec: 19694, Lr: 0.000147\n", "2020-01-18 00:15:50,789 Epoch 22 Step: 195400 Batch Loss: 1.041767 Tokens per Sec: 19359, Lr: 0.000147\n", "2020-01-18 00:16:02,202 Epoch 22 Step: 195500 Batch Loss: 0.974716 Tokens per Sec: 18990, Lr: 0.000147\n", "2020-01-18 00:16:13,686 Epoch 22 Step: 195600 Batch Loss: 0.707493 Tokens per Sec: 19433, Lr: 0.000147\n", "2020-01-18 00:16:25,170 Epoch 22 Step: 195700 Batch Loss: 1.145055 Tokens per Sec: 19420, Lr: 0.000147\n", "2020-01-18 00:16:36,533 Epoch 22 Step: 195800 Batch Loss: 1.295316 Tokens per Sec: 19372, Lr: 0.000147\n", "2020-01-18 00:16:47,860 Epoch 22 Step: 195900 Batch Loss: 1.215176 Tokens per Sec: 19292, Lr: 0.000147\n", "2020-01-18 00:16:59,293 Epoch 22 Step: 196000 Batch Loss: 1.299451 Tokens per Sec: 19052, Lr: 0.000147\n", "2020-01-18 00:17:28,553 Example #0\n", "2020-01-18 00:17:28,554 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:17:28,554 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:17:28,554 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:17:28,554 Example #1\n", "2020-01-18 00:17:28,555 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:17:28,555 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:17:28,555 \tHypothesis: Ijo , a te lele iei , Ijo .\n", "2020-01-18 00:17:28,555 Example #2\n", "2020-01-18 00:17:28,556 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:17:28,556 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:17:28,556 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba uwoho mai te abọ urere ọrọ ole owoma nana : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te vihọ , re o te lo via eva riẹ , re ọ ghale ae ; jọ ahwo kpobi a wo oghale .\n", "2020-01-18 00:17:28,556 Example #3\n", "2020-01-18 00:17:28,556 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:17:28,557 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:17:28,557 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:17:28,557 Validation result (greedy) at epoch 22, step 196000: bleu: 31.68, loss: 32666.7754, ppl: 3.5256, duration: 29.2637s\n", "2020-01-18 00:17:40,077 Epoch 22 Step: 196100 Batch Loss: 1.136639 Tokens per Sec: 19643, Lr: 0.000147\n", "2020-01-18 00:17:51,489 Epoch 22 Step: 196200 Batch Loss: 1.271243 Tokens per Sec: 19329, Lr: 0.000147\n", "2020-01-18 00:18:02,931 Epoch 22 Step: 196300 Batch Loss: 1.244118 Tokens per Sec: 19441, Lr: 0.000147\n", "2020-01-18 00:18:14,427 Epoch 22 Step: 196400 Batch Loss: 1.208056 Tokens per Sec: 19609, Lr: 0.000147\n", "2020-01-18 00:18:25,906 Epoch 22 Step: 196500 Batch Loss: 1.161454 Tokens per Sec: 19300, Lr: 0.000147\n", "2020-01-18 00:18:37,272 Epoch 22 Step: 196600 Batch Loss: 1.100202 Tokens per Sec: 19382, Lr: 0.000147\n", "2020-01-18 00:18:48,854 Epoch 22 Step: 196700 Batch Loss: 1.157027 Tokens per Sec: 19967, Lr: 0.000147\n", "2020-01-18 00:19:00,257 Epoch 22 Step: 196800 Batch Loss: 1.238233 Tokens per Sec: 19695, Lr: 0.000147\n", "2020-01-18 00:19:11,786 Epoch 22 Step: 196900 Batch Loss: 1.290434 Tokens per Sec: 19083, Lr: 0.000147\n", "2020-01-18 00:19:23,165 Epoch 22 Step: 197000 Batch Loss: 0.977742 Tokens per Sec: 19155, Lr: 0.000147\n", "2020-01-18 00:19:52,291 Example #0\n", "2020-01-18 00:19:52,292 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:19:52,292 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:19:52,292 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:19:52,293 Example #1\n", "2020-01-18 00:19:52,293 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:19:52,293 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:19:52,293 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 00:19:52,293 Example #2\n", "2020-01-18 00:19:52,294 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:19:52,294 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:19:52,294 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re ọ jẹ ghale ae ; jọ o te ru re ahwo kpobi a wo oghale .\n", "2020-01-18 00:19:52,294 Example #3\n", "2020-01-18 00:19:52,294 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:19:52,295 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:19:52,295 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:19:52,295 Validation result (greedy) at epoch 22, step 197000: bleu: 32.01, loss: 32716.4023, ppl: 3.5324, duration: 29.1296s\n", "2020-01-18 00:20:03,801 Epoch 22 Step: 197100 Batch Loss: 1.334640 Tokens per Sec: 19476, Lr: 0.000103\n", "2020-01-18 00:20:15,396 Epoch 22 Step: 197200 Batch Loss: 1.114425 Tokens per Sec: 18998, Lr: 0.000103\n", "2020-01-18 00:20:26,877 Epoch 22 Step: 197300 Batch Loss: 0.929367 Tokens per Sec: 19474, Lr: 0.000103\n", "2020-01-18 00:20:38,224 Epoch 22 Step: 197400 Batch Loss: 1.271189 Tokens per Sec: 19408, Lr: 0.000103\n", "2020-01-18 00:20:49,624 Epoch 22 Step: 197500 Batch Loss: 1.037708 Tokens per Sec: 19205, Lr: 0.000103\n", "2020-01-18 00:21:01,033 Epoch 22 Step: 197600 Batch Loss: 0.873171 Tokens per Sec: 19446, Lr: 0.000103\n", "2020-01-18 00:21:03,288 Epoch 22: total training loss 2999.80\n", "2020-01-18 00:21:03,288 EPOCH 23\n", "2020-01-18 00:21:12,826 Epoch 23 Step: 197700 Batch Loss: 1.000977 Tokens per Sec: 18332, Lr: 0.000103\n", "2020-01-18 00:21:24,332 Epoch 23 Step: 197800 Batch Loss: 1.099030 Tokens per Sec: 19413, Lr: 0.000103\n", "2020-01-18 00:21:35,825 Epoch 23 Step: 197900 Batch Loss: 1.340010 Tokens per Sec: 19491, Lr: 0.000103\n", "2020-01-18 00:21:47,373 Epoch 23 Step: 198000 Batch Loss: 1.150984 Tokens per Sec: 19730, Lr: 0.000103\n", "2020-01-18 00:22:16,492 Example #0\n", "2020-01-18 00:22:16,493 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:22:16,493 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:22:16,493 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:22:16,493 Example #1\n", "2020-01-18 00:22:16,494 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:22:16,494 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:22:16,494 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-18 00:22:16,494 Example #2\n", "2020-01-18 00:22:16,495 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:22:16,495 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:22:16,495 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ , re ọre o te lo , re o te lo , re o je wo oghale , re erẹwho kpobi i wo oghale ; wha jo wo oghale .\n", "2020-01-18 00:22:16,495 Example #3\n", "2020-01-18 00:22:16,495 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:22:16,495 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:22:16,496 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:22:16,496 Validation result (greedy) at epoch 23, step 198000: bleu: 31.86, loss: 32703.6191, ppl: 3.5306, duration: 29.1219s\n", "2020-01-18 00:22:28,048 Epoch 23 Step: 198100 Batch Loss: 1.246097 Tokens per Sec: 19814, Lr: 0.000103\n", "2020-01-18 00:22:39,465 Epoch 23 Step: 198200 Batch Loss: 1.211610 Tokens per Sec: 19098, Lr: 0.000103\n", "2020-01-18 00:22:51,036 Epoch 23 Step: 198300 Batch Loss: 1.259890 Tokens per Sec: 19146, Lr: 0.000103\n", "2020-01-18 00:23:02,401 Epoch 23 Step: 198400 Batch Loss: 1.300787 Tokens per Sec: 19072, Lr: 0.000103\n", "2020-01-18 00:23:13,980 Epoch 23 Step: 198500 Batch Loss: 1.185351 Tokens per Sec: 19537, Lr: 0.000103\n", "2020-01-18 00:23:25,535 Epoch 23 Step: 198600 Batch Loss: 1.411885 Tokens per Sec: 19381, Lr: 0.000103\n", "2020-01-18 00:23:36,896 Epoch 23 Step: 198700 Batch Loss: 1.504266 Tokens per Sec: 19307, Lr: 0.000103\n", "2020-01-18 00:23:48,345 Epoch 23 Step: 198800 Batch Loss: 1.139144 Tokens per Sec: 19391, Lr: 0.000103\n", "2020-01-18 00:23:59,903 Epoch 23 Step: 198900 Batch Loss: 1.219274 Tokens per Sec: 19457, Lr: 0.000103\n", "2020-01-18 00:24:11,439 Epoch 23 Step: 199000 Batch Loss: 1.362953 Tokens per Sec: 19159, Lr: 0.000103\n", "2020-01-18 00:24:40,535 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:24:40,535 Saving new checkpoint.\n", "2020-01-18 00:24:41,923 Example #0\n", "2020-01-18 00:24:41,924 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:24:41,924 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:24:41,925 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:24:41,925 Example #1\n", "2020-01-18 00:24:41,925 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:24:41,925 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:24:41,926 \tHypothesis: Ijo , a te lele onana .\n", "2020-01-18 00:24:41,926 Example #2\n", "2020-01-18 00:24:41,926 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:24:41,926 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:24:41,927 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ ; re ọre o te lo via , re o te lo , re o je wo oghale rọ kẹ ae ; jọ o jọ kẹ ae kpobi evawere .\n", "2020-01-18 00:24:41,927 Example #3\n", "2020-01-18 00:24:41,927 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:24:41,927 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:24:41,928 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:24:41,928 Validation result (greedy) at epoch 23, step 199000: bleu: 31.54, loss: 32634.6875, ppl: 3.5212, duration: 30.4884s\n", "2020-01-18 00:24:53,555 Epoch 23 Step: 199100 Batch Loss: 1.145780 Tokens per Sec: 19306, Lr: 0.000103\n", "2020-01-18 00:25:05,077 Epoch 23 Step: 199200 Batch Loss: 1.146134 Tokens per Sec: 19118, Lr: 0.000103\n", "2020-01-18 00:25:16,597 Epoch 23 Step: 199300 Batch Loss: 1.104913 Tokens per Sec: 19462, Lr: 0.000103\n", "2020-01-18 00:25:28,120 Epoch 23 Step: 199400 Batch Loss: 1.221113 Tokens per Sec: 19127, Lr: 0.000103\n", "2020-01-18 00:25:39,529 Epoch 23 Step: 199500 Batch Loss: 1.002820 Tokens per Sec: 19748, Lr: 0.000103\n", "2020-01-18 00:25:50,990 Epoch 23 Step: 199600 Batch Loss: 1.435136 Tokens per Sec: 19513, Lr: 0.000103\n", "2020-01-18 00:26:02,327 Epoch 23 Step: 199700 Batch Loss: 1.246626 Tokens per Sec: 19370, Lr: 0.000103\n", "2020-01-18 00:26:13,824 Epoch 23 Step: 199800 Batch Loss: 0.620877 Tokens per Sec: 19320, Lr: 0.000103\n", "2020-01-18 00:26:25,258 Epoch 23 Step: 199900 Batch Loss: 1.195468 Tokens per Sec: 19184, Lr: 0.000103\n", "2020-01-18 00:26:36,746 Epoch 23 Step: 200000 Batch Loss: 1.256501 Tokens per Sec: 19674, Lr: 0.000103\n", "2020-01-18 00:27:05,757 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:27:05,757 Saving new checkpoint.\n", "2020-01-18 00:27:07,025 Example #0\n", "2020-01-18 00:27:07,026 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:27:07,026 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:27:07,026 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:27:07,026 Example #1\n", "2020-01-18 00:27:07,027 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:27:07,027 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:27:07,027 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 00:27:07,027 Example #2\n", "2020-01-18 00:27:07,028 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:27:07,028 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:27:07,029 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re o je jiri odẹ riẹ , re ọ ghale ae ; jọ o jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 00:27:07,029 Example #3\n", "2020-01-18 00:27:07,029 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:27:07,029 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:27:07,029 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:27:07,030 Validation result (greedy) at epoch 23, step 200000: bleu: 32.01, loss: 32606.6504, ppl: 3.5174, duration: 30.2826s\n", "2020-01-18 00:27:18,802 Epoch 23 Step: 200100 Batch Loss: 1.248008 Tokens per Sec: 19231, Lr: 0.000103\n", "2020-01-18 00:27:24,054 Epoch 23: total training loss 2968.13\n", "2020-01-18 00:27:24,055 EPOCH 24\n", "2020-01-18 00:27:30,590 Epoch 24 Step: 200200 Batch Loss: 1.368369 Tokens per Sec: 19048, Lr: 0.000103\n", "2020-01-18 00:27:41,975 Epoch 24 Step: 200300 Batch Loss: 1.251953 Tokens per Sec: 18880, Lr: 0.000103\n", "2020-01-18 00:27:53,494 Epoch 24 Step: 200400 Batch Loss: 1.275050 Tokens per Sec: 19152, Lr: 0.000103\n", "2020-01-18 00:28:04,959 Epoch 24 Step: 200500 Batch Loss: 0.968806 Tokens per Sec: 19473, Lr: 0.000103\n", "2020-01-18 00:28:16,486 Epoch 24 Step: 200600 Batch Loss: 1.199147 Tokens per Sec: 19122, Lr: 0.000103\n", "2020-01-18 00:28:28,010 Epoch 24 Step: 200700 Batch Loss: 1.162505 Tokens per Sec: 19574, Lr: 0.000103\n", "2020-01-18 00:28:39,419 Epoch 24 Step: 200800 Batch Loss: 1.142846 Tokens per Sec: 19201, Lr: 0.000103\n", "2020-01-18 00:28:50,910 Epoch 24 Step: 200900 Batch Loss: 1.375316 Tokens per Sec: 19876, Lr: 0.000103\n", "2020-01-18 00:29:02,363 Epoch 24 Step: 201000 Batch Loss: 1.273799 Tokens per Sec: 19676, Lr: 0.000103\n", "2020-01-18 00:29:31,499 Example #0\n", "2020-01-18 00:29:31,500 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:29:31,500 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:29:31,500 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:29:31,500 Example #1\n", "2020-01-18 00:29:31,501 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:29:31,501 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:29:31,501 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 00:29:31,501 Example #2\n", "2020-01-18 00:29:31,501 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:29:31,502 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:29:31,502 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re o wo oghale , re ọ ghale ae ; jọ o jiri ahwo Egedhọ kpobi ; wha kẹ odẹ riẹ .\n", "2020-01-18 00:29:31,502 Example #3\n", "2020-01-18 00:29:31,502 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:29:31,502 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:29:31,503 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:29:31,503 Validation result (greedy) at epoch 24, step 201000: bleu: 31.92, loss: 32624.0996, ppl: 3.5198, duration: 29.1397s\n", "2020-01-18 00:29:43,045 Epoch 24 Step: 201100 Batch Loss: 0.986847 Tokens per Sec: 19261, Lr: 0.000103\n", "2020-01-18 00:29:54,421 Epoch 24 Step: 201200 Batch Loss: 1.138899 Tokens per Sec: 19232, Lr: 0.000103\n", "2020-01-18 00:30:05,838 Epoch 24 Step: 201300 Batch Loss: 1.374568 Tokens per Sec: 19375, Lr: 0.000103\n", "2020-01-18 00:30:17,423 Epoch 24 Step: 201400 Batch Loss: 1.353243 Tokens per Sec: 19196, Lr: 0.000103\n", "2020-01-18 00:30:28,913 Epoch 24 Step: 201500 Batch Loss: 1.249016 Tokens per Sec: 19120, Lr: 0.000103\n", "2020-01-18 00:30:40,340 Epoch 24 Step: 201600 Batch Loss: 1.265815 Tokens per Sec: 19556, Lr: 0.000103\n", "2020-01-18 00:30:51,914 Epoch 24 Step: 201700 Batch Loss: 1.125395 Tokens per Sec: 19690, Lr: 0.000103\n", "2020-01-18 00:31:03,196 Epoch 24 Step: 201800 Batch Loss: 1.297719 Tokens per Sec: 19082, Lr: 0.000103\n", "2020-01-18 00:31:14,817 Epoch 24 Step: 201900 Batch Loss: 1.223060 Tokens per Sec: 19624, Lr: 0.000103\n", "2020-01-18 00:31:26,386 Epoch 24 Step: 202000 Batch Loss: 1.173726 Tokens per Sec: 19755, Lr: 0.000103\n", "2020-01-18 00:31:55,496 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:31:55,496 Saving new checkpoint.\n", "2020-01-18 00:31:56,690 Example #0\n", "2020-01-18 00:31:56,691 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:31:56,691 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:31:56,691 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:31:56,691 Example #1\n", "2020-01-18 00:31:56,692 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:31:56,692 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:31:56,692 \tHypothesis: A te lele iei , Usi Uvie na O gbẹ jariẹ hẹ .\n", "2020-01-18 00:31:56,692 Example #2\n", "2020-01-18 00:31:56,693 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:31:56,693 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:31:56,693 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere na kpohọ abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re ọ ghale ae ; jọ o jọ ọnọ ọ rẹ ghale ae ; wha kẹ erẹwho kpobi oghale .\n", "2020-01-18 00:31:56,693 Example #3\n", "2020-01-18 00:31:56,694 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:31:56,694 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:31:56,694 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:31:56,694 Validation result (greedy) at epoch 24, step 202000: bleu: 32.11, loss: 32595.1855, ppl: 3.5159, duration: 30.3073s\n", "2020-01-18 00:32:08,394 Epoch 24 Step: 202100 Batch Loss: 0.958822 Tokens per Sec: 18709, Lr: 0.000103\n", "2020-01-18 00:32:19,881 Epoch 24 Step: 202200 Batch Loss: 1.151894 Tokens per Sec: 18934, Lr: 0.000103\n", "2020-01-18 00:32:31,369 Epoch 24 Step: 202300 Batch Loss: 1.094054 Tokens per Sec: 19574, Lr: 0.000103\n", "2020-01-18 00:32:42,844 Epoch 24 Step: 202400 Batch Loss: 1.300417 Tokens per Sec: 19277, Lr: 0.000103\n", "2020-01-18 00:32:54,289 Epoch 24 Step: 202500 Batch Loss: 1.248383 Tokens per Sec: 19564, Lr: 0.000103\n", "2020-01-18 00:33:05,807 Epoch 24 Step: 202600 Batch Loss: 1.335575 Tokens per Sec: 19193, Lr: 0.000103\n", "2020-01-18 00:33:14,297 Epoch 24: total training loss 2963.72\n", "2020-01-18 00:33:14,297 EPOCH 25\n", "2020-01-18 00:33:17,654 Epoch 25 Step: 202700 Batch Loss: 0.792018 Tokens per Sec: 18247, Lr: 0.000103\n", "2020-01-18 00:33:29,049 Epoch 25 Step: 202800 Batch Loss: 1.152484 Tokens per Sec: 19239, Lr: 0.000103\n", "2020-01-18 00:33:40,507 Epoch 25 Step: 202900 Batch Loss: 1.098915 Tokens per Sec: 19637, Lr: 0.000103\n", "2020-01-18 00:33:52,029 Epoch 25 Step: 203000 Batch Loss: 1.235822 Tokens per Sec: 19613, Lr: 0.000103\n", "2020-01-18 00:34:21,150 Example #0\n", "2020-01-18 00:34:21,150 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:34:21,151 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:34:21,151 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:34:21,151 Example #1\n", "2020-01-18 00:34:21,151 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:34:21,151 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:34:21,151 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 00:34:21,152 Example #2\n", "2020-01-18 00:34:21,152 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:34:21,152 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:34:21,152 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba otofa urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ ; re ọre o te lo via , re o te lo , re o je jiri odẹ riẹ ; jọ o jọ rọ kẹ ae evawere .\n", "2020-01-18 00:34:21,153 Example #3\n", "2020-01-18 00:34:21,153 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:34:21,153 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:34:21,153 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:34:21,153 Validation result (greedy) at epoch 25, step 203000: bleu: 31.76, loss: 32705.6641, ppl: 3.5309, duration: 29.1239s\n", "2020-01-18 00:34:32,545 Epoch 25 Step: 203100 Batch Loss: 1.109021 Tokens per Sec: 19005, Lr: 0.000103\n", "2020-01-18 00:34:44,217 Epoch 25 Step: 203200 Batch Loss: 1.111510 Tokens per Sec: 19707, Lr: 0.000103\n", "2020-01-18 00:34:55,707 Epoch 25 Step: 203300 Batch Loss: 0.955422 Tokens per Sec: 19353, Lr: 0.000103\n", "2020-01-18 00:35:07,138 Epoch 25 Step: 203400 Batch Loss: 1.118241 Tokens per Sec: 19053, Lr: 0.000103\n", "2020-01-18 00:35:18,633 Epoch 25 Step: 203500 Batch Loss: 1.388727 Tokens per Sec: 19560, Lr: 0.000103\n", "2020-01-18 00:35:30,128 Epoch 25 Step: 203600 Batch Loss: 1.206552 Tokens per Sec: 19154, Lr: 0.000103\n", "2020-01-18 00:35:41,673 Epoch 25 Step: 203700 Batch Loss: 1.406430 Tokens per Sec: 19385, Lr: 0.000103\n", "2020-01-18 00:35:53,119 Epoch 25 Step: 203800 Batch Loss: 0.961438 Tokens per Sec: 19337, Lr: 0.000103\n", "2020-01-18 00:36:04,636 Epoch 25 Step: 203900 Batch Loss: 1.145739 Tokens per Sec: 19372, Lr: 0.000103\n", "2020-01-18 00:36:16,121 Epoch 25 Step: 204000 Batch Loss: 1.064088 Tokens per Sec: 19037, Lr: 0.000103\n", "2020-01-18 00:36:45,235 Example #0\n", "2020-01-18 00:36:45,236 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:36:45,236 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:36:45,236 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:36:45,237 Example #1\n", "2020-01-18 00:36:45,237 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:36:45,237 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:36:45,237 \tHypothesis: A te lele onana Usi Uvie na Ijo .\n", "2020-01-18 00:36:45,237 Example #2\n", "2020-01-18 00:36:45,238 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:36:45,238 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:36:45,238 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oghale , re erẹwho kpobi i wo oghale .\n", "2020-01-18 00:36:45,238 Example #3\n", "2020-01-18 00:36:45,238 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:36:45,239 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:36:45,239 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:36:45,239 Validation result (greedy) at epoch 25, step 204000: bleu: 31.77, loss: 32633.0449, ppl: 3.5210, duration: 29.1177s\n", "2020-01-18 00:36:56,794 Epoch 25 Step: 204100 Batch Loss: 1.208786 Tokens per Sec: 19681, Lr: 0.000103\n", "2020-01-18 00:37:08,339 Epoch 25 Step: 204200 Batch Loss: 1.181620 Tokens per Sec: 19327, Lr: 0.000103\n", "2020-01-18 00:37:19,745 Epoch 25 Step: 204300 Batch Loss: 1.335954 Tokens per Sec: 19272, Lr: 0.000103\n", "2020-01-18 00:37:31,243 Epoch 25 Step: 204400 Batch Loss: 0.946929 Tokens per Sec: 19336, Lr: 0.000103\n", "2020-01-18 00:37:42,684 Epoch 25 Step: 204500 Batch Loss: 1.104514 Tokens per Sec: 19263, Lr: 0.000103\n", "2020-01-18 00:37:54,159 Epoch 25 Step: 204600 Batch Loss: 1.298589 Tokens per Sec: 19214, Lr: 0.000103\n", "2020-01-18 00:38:05,542 Epoch 25 Step: 204700 Batch Loss: 1.295501 Tokens per Sec: 19109, Lr: 0.000103\n", "2020-01-18 00:38:17,274 Epoch 25 Step: 204800 Batch Loss: 0.941868 Tokens per Sec: 19476, Lr: 0.000103\n", "2020-01-18 00:38:28,706 Epoch 25 Step: 204900 Batch Loss: 1.128126 Tokens per Sec: 19545, Lr: 0.000103\n", "2020-01-18 00:38:40,196 Epoch 25 Step: 205000 Batch Loss: 1.119698 Tokens per Sec: 19493, Lr: 0.000103\n", "2020-01-18 00:39:09,401 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:39:09,401 Saving new checkpoint.\n", "2020-01-18 00:39:11,007 Example #0\n", "2020-01-18 00:39:11,007 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:39:11,007 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:39:11,008 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:39:11,008 Example #1\n", "2020-01-18 00:39:11,008 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:39:11,009 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:39:11,009 \tHypothesis: A te lele Usi Uvie Na , Ijo .\n", "2020-01-18 00:39:11,009 Example #2\n", "2020-01-18 00:39:11,010 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:39:11,010 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:39:11,010 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹvi mai rite abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oghale , re erẹwho na kpobi i wo oghale ; jọ o jọ odẹ riẹ o jọ o jọ odẹ riẹ .\n", "2020-01-18 00:39:11,010 Example #3\n", "2020-01-18 00:39:11,011 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:39:11,011 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:39:11,011 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:39:11,011 Validation result (greedy) at epoch 25, step 205000: bleu: 31.64, loss: 32577.7227, ppl: 3.5135, duration: 30.8146s\n", "2020-01-18 00:39:22,674 Epoch 25 Step: 205100 Batch Loss: 1.335173 Tokens per Sec: 19007, Lr: 0.000103\n", "2020-01-18 00:39:34,096 Epoch 25 Step: 205200 Batch Loss: 1.245010 Tokens per Sec: 19146, Lr: 0.000103\n", "2020-01-18 00:39:34,396 Epoch 25: total training loss 2957.09\n", "2020-01-18 00:39:34,397 EPOCH 26\n", "2020-01-18 00:39:45,767 Epoch 26 Step: 205300 Batch Loss: 1.149063 Tokens per Sec: 19157, Lr: 0.000103\n", "2020-01-18 00:39:57,247 Epoch 26 Step: 205400 Batch Loss: 1.209381 Tokens per Sec: 19331, Lr: 0.000103\n", "2020-01-18 00:40:08,841 Epoch 26 Step: 205500 Batch Loss: 0.882918 Tokens per Sec: 19112, Lr: 0.000103\n", "2020-01-18 00:40:20,192 Epoch 26 Step: 205600 Batch Loss: 1.157488 Tokens per Sec: 19240, Lr: 0.000103\n", "2020-01-18 00:40:31,555 Epoch 26 Step: 205700 Batch Loss: 1.179061 Tokens per Sec: 19248, Lr: 0.000103\n", "2020-01-18 00:40:43,056 Epoch 26 Step: 205800 Batch Loss: 1.143656 Tokens per Sec: 19599, Lr: 0.000103\n", "2020-01-18 00:40:54,476 Epoch 26 Step: 205900 Batch Loss: 1.280299 Tokens per Sec: 19272, Lr: 0.000103\n", "2020-01-18 00:41:05,917 Epoch 26 Step: 206000 Batch Loss: 1.195301 Tokens per Sec: 19255, Lr: 0.000103\n", "2020-01-18 00:41:35,115 Example #0\n", "2020-01-18 00:41:35,116 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:41:35,116 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:41:35,116 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:41:35,116 Example #1\n", "2020-01-18 00:41:35,117 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:41:35,117 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:41:35,117 \tHypothesis: A te lele onana Usi Uvie Na .\n", "2020-01-18 00:41:35,117 Example #2\n", "2020-01-18 00:41:35,117 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:41:35,118 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:41:35,118 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹvi mai rite abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re erẹwho kpobi i wo oghale ; wha jọ ahwo nọ a wo oghale .\n", "2020-01-18 00:41:35,118 Example #3\n", "2020-01-18 00:41:35,118 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:41:35,118 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:41:35,118 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:41:35,119 Validation result (greedy) at epoch 26, step 206000: bleu: 31.78, loss: 32588.1660, ppl: 3.5149, duration: 29.2008s\n", "2020-01-18 00:41:46,548 Epoch 26 Step: 206100 Batch Loss: 0.995686 Tokens per Sec: 19429, Lr: 0.000103\n", "2020-01-18 00:41:57,834 Epoch 26 Step: 206200 Batch Loss: 1.028672 Tokens per Sec: 18872, Lr: 0.000103\n", "2020-01-18 00:42:09,373 Epoch 26 Step: 206300 Batch Loss: 1.057320 Tokens per Sec: 19181, Lr: 0.000103\n", "2020-01-18 00:42:20,871 Epoch 26 Step: 206400 Batch Loss: 1.309634 Tokens per Sec: 19664, Lr: 0.000103\n", "2020-01-18 00:42:32,419 Epoch 26 Step: 206500 Batch Loss: 1.314451 Tokens per Sec: 19441, Lr: 0.000103\n", "2020-01-18 00:42:44,034 Epoch 26 Step: 206600 Batch Loss: 0.888785 Tokens per Sec: 19754, Lr: 0.000103\n", "2020-01-18 00:42:55,421 Epoch 26 Step: 206700 Batch Loss: 1.301665 Tokens per Sec: 19547, Lr: 0.000103\n", "2020-01-18 00:43:07,006 Epoch 26 Step: 206800 Batch Loss: 0.966350 Tokens per Sec: 19706, Lr: 0.000103\n", "2020-01-18 00:43:18,567 Epoch 26 Step: 206900 Batch Loss: 1.168084 Tokens per Sec: 19518, Lr: 0.000103\n", "2020-01-18 00:43:30,178 Epoch 26 Step: 207000 Batch Loss: 1.011042 Tokens per Sec: 19276, Lr: 0.000103\n", "2020-01-18 00:43:59,227 Example #0\n", "2020-01-18 00:43:59,228 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:43:59,228 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:43:59,228 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:43:59,228 Example #1\n", "2020-01-18 00:43:59,229 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:43:59,229 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:43:59,229 \tHypothesis: A te lele Usi Uvie Na , Ijo .\n", "2020-01-18 00:43:59,229 Example #2\n", "2020-01-18 00:43:59,230 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:43:59,230 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:43:59,230 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re o je wo oghale eva aro riẹ ; jọ o jọ kẹ erẹwho kpobi ; re wha wo oghale .\n", "2020-01-18 00:43:59,230 Example #3\n", "2020-01-18 00:43:59,231 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:43:59,231 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:43:59,231 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 00:43:59,231 Validation result (greedy) at epoch 26, step 207000: bleu: 31.81, loss: 32693.0117, ppl: 3.5292, duration: 29.0524s\n", "2020-01-18 00:44:10,779 Epoch 26 Step: 207100 Batch Loss: 1.195977 Tokens per Sec: 19045, Lr: 0.000103\n", "2020-01-18 00:44:22,266 Epoch 26 Step: 207200 Batch Loss: 1.145106 Tokens per Sec: 19670, Lr: 0.000103\n", "2020-01-18 00:44:33,689 Epoch 26 Step: 207300 Batch Loss: 1.210863 Tokens per Sec: 19220, Lr: 0.000103\n", "2020-01-18 00:44:45,146 Epoch 26 Step: 207400 Batch Loss: 1.181246 Tokens per Sec: 19409, Lr: 0.000103\n", "2020-01-18 00:44:56,543 Epoch 26 Step: 207500 Batch Loss: 1.120276 Tokens per Sec: 19469, Lr: 0.000103\n", "2020-01-18 00:45:08,112 Epoch 26 Step: 207600 Batch Loss: 1.115861 Tokens per Sec: 19139, Lr: 0.000103\n", "2020-01-18 00:45:19,534 Epoch 26 Step: 207700 Batch Loss: 1.191836 Tokens per Sec: 19269, Lr: 0.000103\n", "2020-01-18 00:45:23,037 Epoch 26: total training loss 2952.32\n", "2020-01-18 00:45:23,038 EPOCH 27\n", "2020-01-18 00:45:31,329 Epoch 27 Step: 207800 Batch Loss: 1.217276 Tokens per Sec: 18536, Lr: 0.000103\n", "2020-01-18 00:45:42,786 Epoch 27 Step: 207900 Batch Loss: 1.401794 Tokens per Sec: 19040, Lr: 0.000103\n", "2020-01-18 00:45:54,122 Epoch 27 Step: 208000 Batch Loss: 1.192732 Tokens per Sec: 19360, Lr: 0.000103\n", "2020-01-18 00:46:23,182 Example #0\n", "2020-01-18 00:46:23,183 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:46:23,183 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:46:23,183 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:46:23,183 Example #1\n", "2020-01-18 00:46:23,184 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:46:23,184 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:46:23,184 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 00:46:23,184 Example #2\n", "2020-01-18 00:46:23,184 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:46:23,185 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:46:23,185 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re o je jiri odẹ riẹ , re erẹwho na kpobi e jọ e wo oghale ; wha jọ ahwo nọ a wo oghale .\n", "2020-01-18 00:46:23,185 Example #3\n", "2020-01-18 00:46:23,185 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:46:23,185 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:46:23,186 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:46:23,186 Validation result (greedy) at epoch 27, step 208000: bleu: 32.10, loss: 32622.9473, ppl: 3.5196, duration: 29.0637s\n", "2020-01-18 00:46:34,695 Epoch 27 Step: 208100 Batch Loss: 1.241701 Tokens per Sec: 19275, Lr: 0.000103\n", "2020-01-18 00:46:46,112 Epoch 27 Step: 208200 Batch Loss: 1.275929 Tokens per Sec: 19296, Lr: 0.000103\n", "2020-01-18 00:46:57,439 Epoch 27 Step: 208300 Batch Loss: 1.149933 Tokens per Sec: 19148, Lr: 0.000103\n", "2020-01-18 00:47:09,028 Epoch 27 Step: 208400 Batch Loss: 0.949798 Tokens per Sec: 19485, Lr: 0.000103\n", "2020-01-18 00:47:20,437 Epoch 27 Step: 208500 Batch Loss: 1.149296 Tokens per Sec: 19229, Lr: 0.000103\n", "2020-01-18 00:47:31,940 Epoch 27 Step: 208600 Batch Loss: 1.210865 Tokens per Sec: 19703, Lr: 0.000103\n", "2020-01-18 00:47:43,437 Epoch 27 Step: 208700 Batch Loss: 0.945381 Tokens per Sec: 19533, Lr: 0.000103\n", "2020-01-18 00:47:54,821 Epoch 27 Step: 208800 Batch Loss: 1.187539 Tokens per Sec: 19571, Lr: 0.000103\n", "2020-01-18 00:48:06,277 Epoch 27 Step: 208900 Batch Loss: 1.299103 Tokens per Sec: 19456, Lr: 0.000103\n", "2020-01-18 00:48:17,744 Epoch 27 Step: 209000 Batch Loss: 0.994488 Tokens per Sec: 19379, Lr: 0.000103\n", "2020-01-18 00:48:46,901 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:48:46,902 Saving new checkpoint.\n", "2020-01-18 00:48:48,110 Example #0\n", "2020-01-18 00:48:48,110 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:48:48,110 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:48:48,110 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:48:48,111 Example #1\n", "2020-01-18 00:48:48,111 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:48:48,111 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:48:48,111 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 00:48:48,112 Example #2\n", "2020-01-18 00:48:48,112 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:48:48,112 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:48:48,112 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oghale , re erẹwho na kpobi i wo oghale ; jọ o jọ odẹ riẹ ; re o jọ o jọ kẹ ahwo Egedhọ evawere .\n", "2020-01-18 00:48:48,113 Example #3\n", "2020-01-18 00:48:48,113 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:48:48,113 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:48:48,113 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:48:48,113 Validation result (greedy) at epoch 27, step 209000: bleu: 32.01, loss: 32566.5195, ppl: 3.5120, duration: 30.3694s\n", "2020-01-18 00:48:59,798 Epoch 27 Step: 209100 Batch Loss: 1.344033 Tokens per Sec: 19507, Lr: 0.000103\n", "2020-01-18 00:49:11,379 Epoch 27 Step: 209200 Batch Loss: 1.324685 Tokens per Sec: 18971, Lr: 0.000103\n", "2020-01-18 00:49:22,851 Epoch 27 Step: 209300 Batch Loss: 1.372454 Tokens per Sec: 19544, Lr: 0.000103\n", "2020-01-18 00:49:34,321 Epoch 27 Step: 209400 Batch Loss: 1.133602 Tokens per Sec: 19313, Lr: 0.000103\n", "2020-01-18 00:49:45,854 Epoch 27 Step: 209500 Batch Loss: 0.947770 Tokens per Sec: 19524, Lr: 0.000103\n", "2020-01-18 00:49:57,259 Epoch 27 Step: 209600 Batch Loss: 1.076588 Tokens per Sec: 19620, Lr: 0.000103\n", "2020-01-18 00:50:08,903 Epoch 27 Step: 209700 Batch Loss: 1.338597 Tokens per Sec: 19321, Lr: 0.000103\n", "2020-01-18 00:50:20,296 Epoch 27 Step: 209800 Batch Loss: 1.027487 Tokens per Sec: 19482, Lr: 0.000103\n", "2020-01-18 00:50:31,862 Epoch 27 Step: 209900 Batch Loss: 1.179063 Tokens per Sec: 19689, Lr: 0.000103\n", "2020-01-18 00:50:43,457 Epoch 27 Step: 210000 Batch Loss: 1.024960 Tokens per Sec: 19296, Lr: 0.000103\n", "2020-01-18 00:51:12,586 Hooray! New best validation result [ppl]!\n", "2020-01-18 00:51:12,586 Saving new checkpoint.\n", "2020-01-18 00:51:14,571 Example #0\n", "2020-01-18 00:51:14,572 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:51:14,572 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:51:14,572 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:51:14,573 Example #1\n", "2020-01-18 00:51:14,573 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:51:14,573 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:51:14,573 \tHypothesis: A te lele onana Usi Uvie na Ijo .\n", "2020-01-18 00:51:14,573 Example #2\n", "2020-01-18 00:51:14,574 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:51:14,574 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:51:14,574 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jọ evaọ erẹwho kpobi nọ i wo oghale .\n", "2020-01-18 00:51:14,574 Example #3\n", "2020-01-18 00:51:14,575 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:51:14,575 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:51:14,575 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:51:14,575 Validation result (greedy) at epoch 27, step 210000: bleu: 31.81, loss: 32484.9258, ppl: 3.5010, duration: 31.1175s\n", "2020-01-18 00:51:26,305 Epoch 27 Step: 210100 Batch Loss: 1.353611 Tokens per Sec: 18687, Lr: 0.000103\n", "2020-01-18 00:51:37,804 Epoch 27 Step: 210200 Batch Loss: 1.246027 Tokens per Sec: 19618, Lr: 0.000103\n", "2020-01-18 00:51:44,116 Epoch 27: total training loss 2965.62\n", "2020-01-18 00:51:44,116 EPOCH 28\n", "2020-01-18 00:51:49,520 Epoch 28 Step: 210300 Batch Loss: 1.004676 Tokens per Sec: 18956, Lr: 0.000103\n", "2020-01-18 00:52:01,035 Epoch 28 Step: 210400 Batch Loss: 1.295610 Tokens per Sec: 19386, Lr: 0.000103\n", "2020-01-18 00:52:12,579 Epoch 28 Step: 210500 Batch Loss: 1.255818 Tokens per Sec: 19186, Lr: 0.000103\n", "2020-01-18 00:52:24,063 Epoch 28 Step: 210600 Batch Loss: 1.231524 Tokens per Sec: 19618, Lr: 0.000103\n", "2020-01-18 00:52:35,508 Epoch 28 Step: 210700 Batch Loss: 1.281718 Tokens per Sec: 19383, Lr: 0.000103\n", "2020-01-18 00:52:47,030 Epoch 28 Step: 210800 Batch Loss: 1.306345 Tokens per Sec: 19109, Lr: 0.000103\n", "2020-01-18 00:52:58,385 Epoch 28 Step: 210900 Batch Loss: 1.339565 Tokens per Sec: 19281, Lr: 0.000103\n", "2020-01-18 00:53:09,867 Epoch 28 Step: 211000 Batch Loss: 1.168923 Tokens per Sec: 19235, Lr: 0.000103\n", "2020-01-18 00:53:38,994 Example #0\n", "2020-01-18 00:53:38,995 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:53:38,995 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:53:38,995 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:53:38,995 Example #1\n", "2020-01-18 00:53:38,996 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:53:38,996 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:53:38,996 \tHypothesis: A te lele onana , Usi Uvie na O tẹ wariẹ muhọ .\n", "2020-01-18 00:53:38,996 Example #2\n", "2020-01-18 00:53:38,997 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:53:38,997 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:53:38,997 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re o je jiri ai , re erẹwho kpobi i wo oghale .\n", "2020-01-18 00:53:38,997 Example #3\n", "2020-01-18 00:53:38,998 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:53:38,998 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:53:38,998 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:53:38,998 Validation result (greedy) at epoch 28, step 211000: bleu: 31.98, loss: 32594.2383, ppl: 3.5157, duration: 29.1302s\n", "2020-01-18 00:53:50,778 Epoch 28 Step: 211100 Batch Loss: 1.255435 Tokens per Sec: 19505, Lr: 0.000103\n", "2020-01-18 00:54:02,247 Epoch 28 Step: 211200 Batch Loss: 1.311082 Tokens per Sec: 19300, Lr: 0.000103\n", "2020-01-18 00:54:13,761 Epoch 28 Step: 211300 Batch Loss: 1.266061 Tokens per Sec: 19113, Lr: 0.000103\n", "2020-01-18 00:54:25,229 Epoch 28 Step: 211400 Batch Loss: 1.130575 Tokens per Sec: 19044, Lr: 0.000103\n", "2020-01-18 00:54:36,694 Epoch 28 Step: 211500 Batch Loss: 1.362310 Tokens per Sec: 19142, Lr: 0.000103\n", "2020-01-18 00:54:48,253 Epoch 28 Step: 211600 Batch Loss: 1.297597 Tokens per Sec: 19193, Lr: 0.000103\n", "2020-01-18 00:54:59,732 Epoch 28 Step: 211700 Batch Loss: 1.106270 Tokens per Sec: 19728, Lr: 0.000103\n", "2020-01-18 00:55:11,427 Epoch 28 Step: 211800 Batch Loss: 1.313098 Tokens per Sec: 19488, Lr: 0.000103\n", "2020-01-18 00:55:22,731 Epoch 28 Step: 211900 Batch Loss: 1.098230 Tokens per Sec: 19018, Lr: 0.000103\n", "2020-01-18 00:55:34,240 Epoch 28 Step: 212000 Batch Loss: 1.291996 Tokens per Sec: 19359, Lr: 0.000103\n", "2020-01-18 00:56:03,282 Example #0\n", "2020-01-18 00:56:03,282 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:56:03,283 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:56:03,283 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:56:03,283 Example #1\n", "2020-01-18 00:56:03,283 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:56:03,283 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:56:03,284 \tHypothesis: A te lele onana Usi Uvie Na No .\n", "2020-01-18 00:56:03,284 Example #2\n", "2020-01-18 00:56:03,284 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:56:03,284 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:56:03,284 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi na ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; wha jọ erẹwho na kpobi i wo oghale .\n", "2020-01-18 00:56:03,285 Example #3\n", "2020-01-18 00:56:03,285 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:56:03,285 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:56:03,285 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 00:56:03,285 Validation result (greedy) at epoch 28, step 212000: bleu: 31.63, loss: 32526.5801, ppl: 3.5066, duration: 29.0453s\n", "2020-01-18 00:56:14,879 Epoch 28 Step: 212100 Batch Loss: 1.266643 Tokens per Sec: 19835, Lr: 0.000103\n", "2020-01-18 00:56:26,379 Epoch 28 Step: 212200 Batch Loss: 0.999875 Tokens per Sec: 19672, Lr: 0.000103\n", "2020-01-18 00:56:37,859 Epoch 28 Step: 212300 Batch Loss: 0.928518 Tokens per Sec: 19415, Lr: 0.000103\n", "2020-01-18 00:56:49,435 Epoch 28 Step: 212400 Batch Loss: 1.193413 Tokens per Sec: 19525, Lr: 0.000103\n", "2020-01-18 00:57:00,908 Epoch 28 Step: 212500 Batch Loss: 1.151461 Tokens per Sec: 19427, Lr: 0.000103\n", "2020-01-18 00:57:12,410 Epoch 28 Step: 212600 Batch Loss: 1.257875 Tokens per Sec: 19090, Lr: 0.000103\n", "2020-01-18 00:57:23,850 Epoch 28 Step: 212700 Batch Loss: 1.240545 Tokens per Sec: 19700, Lr: 0.000103\n", "2020-01-18 00:57:32,662 Epoch 28: total training loss 2961.07\n", "2020-01-18 00:57:32,662 EPOCH 29\n", "2020-01-18 00:57:35,610 Epoch 29 Step: 212800 Batch Loss: 1.150780 Tokens per Sec: 17683, Lr: 0.000103\n", "2020-01-18 00:57:47,064 Epoch 29 Step: 212900 Batch Loss: 0.794869 Tokens per Sec: 19129, Lr: 0.000103\n", "2020-01-18 00:57:58,355 Epoch 29 Step: 213000 Batch Loss: 1.370737 Tokens per Sec: 19111, Lr: 0.000103\n", "2020-01-18 00:58:27,508 Example #0\n", "2020-01-18 00:58:27,509 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 00:58:27,509 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 00:58:27,509 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 00:58:27,509 Example #1\n", "2020-01-18 00:58:27,509 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 00:58:27,510 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 00:58:27,510 \tHypothesis: A te lele iei Usi Uvie Na , Ijo .\n", "2020-01-18 00:58:27,510 Example #2\n", "2020-01-18 00:58:27,510 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 00:58:27,510 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 00:58:27,511 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo viere , re o je vihọ , re erẹwho kpobi i wo oghale , re erẹwho na kpobi i wo oghale .\n", "2020-01-18 00:58:27,511 Example #3\n", "2020-01-18 00:58:27,511 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 00:58:27,511 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 00:58:27,511 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 00:58:27,512 Validation result (greedy) at epoch 29, step 213000: bleu: 31.88, loss: 32552.9121, ppl: 3.5101, duration: 29.1563s\n", "2020-01-18 00:58:38,966 Epoch 29 Step: 213100 Batch Loss: 1.306688 Tokens per Sec: 19238, Lr: 0.000103\n", "2020-01-18 00:58:50,420 Epoch 29 Step: 213200 Batch Loss: 0.958002 Tokens per Sec: 19159, Lr: 0.000103\n", "2020-01-18 00:59:01,993 Epoch 29 Step: 213300 Batch Loss: 0.853123 Tokens per Sec: 19358, Lr: 0.000103\n", "2020-01-18 00:59:13,567 Epoch 29 Step: 213400 Batch Loss: 1.193323 Tokens per Sec: 19259, Lr: 0.000103\n", "2020-01-18 00:59:25,029 Epoch 29 Step: 213500 Batch Loss: 1.297809 Tokens per Sec: 19429, Lr: 0.000103\n", "2020-01-18 00:59:36,484 Epoch 29 Step: 213600 Batch Loss: 1.106004 Tokens per Sec: 19398, Lr: 0.000103\n", "2020-01-18 00:59:47,929 Epoch 29 Step: 213700 Batch Loss: 1.251305 Tokens per Sec: 19353, Lr: 0.000103\n", "2020-01-18 00:59:59,450 Epoch 29 Step: 213800 Batch Loss: 1.110938 Tokens per Sec: 19567, Lr: 0.000103\n", "2020-01-18 01:00:11,057 Epoch 29 Step: 213900 Batch Loss: 1.198494 Tokens per Sec: 19211, Lr: 0.000103\n", "2020-01-18 01:00:22,398 Epoch 29 Step: 214000 Batch Loss: 1.342595 Tokens per Sec: 18974, Lr: 0.000103\n", "2020-01-18 01:00:51,463 Example #0\n", "2020-01-18 01:00:51,463 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:00:51,463 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:00:51,464 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:00:51,464 Example #1\n", "2020-01-18 01:00:51,464 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:00:51,464 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:00:51,464 \tHypothesis: A te je ru onana ẹkwoma Usi Uvie Na .\n", "2020-01-18 01:00:51,464 Example #2\n", "2020-01-18 01:00:51,465 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:00:51,465 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:00:51,465 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ abọ urere ọrọ ole nana nọ ọ rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo , re ọ ghale ae ; wha jo o jiri ai kpobi ; wha kẹ ahwo Egedhọ .\n", "2020-01-18 01:00:51,465 Example #3\n", "2020-01-18 01:00:51,466 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:00:51,466 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:00:51,466 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:00:51,466 Validation result (greedy) at epoch 29, step 214000: bleu: 32.05, loss: 32582.3691, ppl: 3.5141, duration: 29.0672s\n", "2020-01-18 01:01:02,971 Epoch 29 Step: 214100 Batch Loss: 1.189102 Tokens per Sec: 19746, Lr: 0.000103\n", "2020-01-18 01:01:14,651 Epoch 29 Step: 214200 Batch Loss: 1.305243 Tokens per Sec: 19085, Lr: 0.000103\n", "2020-01-18 01:01:26,062 Epoch 29 Step: 214300 Batch Loss: 1.175599 Tokens per Sec: 19114, Lr: 0.000103\n", "2020-01-18 01:01:37,574 Epoch 29 Step: 214400 Batch Loss: 0.934635 Tokens per Sec: 19871, Lr: 0.000103\n", "2020-01-18 01:01:48,982 Epoch 29 Step: 214500 Batch Loss: 1.338872 Tokens per Sec: 19568, Lr: 0.000103\n", "2020-01-18 01:02:00,386 Epoch 29 Step: 214600 Batch Loss: 0.905600 Tokens per Sec: 19245, Lr: 0.000103\n", "2020-01-18 01:02:11,934 Epoch 29 Step: 214700 Batch Loss: 1.340073 Tokens per Sec: 19566, Lr: 0.000103\n", "2020-01-18 01:02:23,260 Epoch 29 Step: 214800 Batch Loss: 1.326074 Tokens per Sec: 19209, Lr: 0.000103\n", "2020-01-18 01:02:34,797 Epoch 29 Step: 214900 Batch Loss: 0.977334 Tokens per Sec: 19685, Lr: 0.000103\n", "2020-01-18 01:02:46,415 Epoch 29 Step: 215000 Batch Loss: 1.263762 Tokens per Sec: 19425, Lr: 0.000103\n", "2020-01-18 01:03:15,604 Example #0\n", "2020-01-18 01:03:15,604 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:03:15,604 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:03:15,605 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:03:15,605 Example #1\n", "2020-01-18 01:03:15,605 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:03:15,605 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:03:15,605 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:03:15,605 Example #2\n", "2020-01-18 01:03:15,606 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:03:15,606 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:03:15,606 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te vihọ , re o wo oghale , re erẹwho kpobi i wo oghale ; wha jọ erẹwho akpọ na i wo oghale .\n", "2020-01-18 01:03:15,606 Example #3\n", "2020-01-18 01:03:15,607 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:03:15,607 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:03:15,607 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 01:03:15,607 Validation result (greedy) at epoch 29, step 215000: bleu: 32.10, loss: 32548.7305, ppl: 3.5096, duration: 29.1913s\n", "2020-01-18 01:03:27,102 Epoch 29 Step: 215100 Batch Loss: 1.315503 Tokens per Sec: 19324, Lr: 0.000103\n", "2020-01-18 01:03:38,550 Epoch 29 Step: 215200 Batch Loss: 1.164742 Tokens per Sec: 19214, Lr: 0.000103\n", "2020-01-18 01:03:50,091 Epoch 29 Step: 215300 Batch Loss: 1.250112 Tokens per Sec: 19723, Lr: 0.000103\n", "2020-01-18 01:03:50,786 Epoch 29: total training loss 2965.48\n", "2020-01-18 01:03:50,786 EPOCH 30\n", "2020-01-18 01:04:01,835 Epoch 30 Step: 215400 Batch Loss: 1.245371 Tokens per Sec: 18843, Lr: 0.000103\n", "2020-01-18 01:04:13,517 Epoch 30 Step: 215500 Batch Loss: 0.985246 Tokens per Sec: 19234, Lr: 0.000103\n", "2020-01-18 01:04:25,027 Epoch 30 Step: 215600 Batch Loss: 1.194273 Tokens per Sec: 19347, Lr: 0.000103\n", "2020-01-18 01:04:36,496 Epoch 30 Step: 215700 Batch Loss: 1.329800 Tokens per Sec: 19555, Lr: 0.000103\n", "2020-01-18 01:04:47,954 Epoch 30 Step: 215800 Batch Loss: 1.337397 Tokens per Sec: 19251, Lr: 0.000103\n", "2020-01-18 01:04:59,382 Epoch 30 Step: 215900 Batch Loss: 1.443240 Tokens per Sec: 19141, Lr: 0.000103\n", "2020-01-18 01:05:10,902 Epoch 30 Step: 216000 Batch Loss: 1.383061 Tokens per Sec: 19065, Lr: 0.000103\n", "2020-01-18 01:05:40,125 Example #0\n", "2020-01-18 01:05:40,125 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:05:40,125 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:05:40,126 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:05:40,126 Example #1\n", "2020-01-18 01:05:40,126 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:05:40,126 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:05:40,126 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:05:40,127 Example #2\n", "2020-01-18 01:05:40,127 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:05:40,127 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:05:40,127 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana , onọ o rẹ kẹ omosasọ gbe omosasọ : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo odẹ riẹ , re ọ ghale ae ; wha jọ erẹwho na kpobi i wo oghale .\n", "2020-01-18 01:05:40,127 Example #3\n", "2020-01-18 01:05:40,128 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:05:40,128 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:05:40,128 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 01:05:40,128 Validation result (greedy) at epoch 30, step 216000: bleu: 31.68, loss: 32515.7441, ppl: 3.5051, duration: 29.2255s\n", "2020-01-18 01:05:51,651 Epoch 30 Step: 216100 Batch Loss: 1.132023 Tokens per Sec: 19354, Lr: 0.000072\n", "2020-01-18 01:06:02,954 Epoch 30 Step: 216200 Batch Loss: 1.083845 Tokens per Sec: 19584, Lr: 0.000072\n", "2020-01-18 01:06:14,526 Epoch 30 Step: 216300 Batch Loss: 0.955738 Tokens per Sec: 19448, Lr: 0.000072\n", "2020-01-18 01:06:26,136 Epoch 30 Step: 216400 Batch Loss: 1.078840 Tokens per Sec: 19635, Lr: 0.000072\n", "2020-01-18 01:06:37,575 Epoch 30 Step: 216500 Batch Loss: 1.119457 Tokens per Sec: 19458, Lr: 0.000072\n", "2020-01-18 01:06:49,126 Epoch 30 Step: 216600 Batch Loss: 0.787524 Tokens per Sec: 19711, Lr: 0.000072\n", "2020-01-18 01:07:00,505 Epoch 30 Step: 216700 Batch Loss: 1.314929 Tokens per Sec: 19457, Lr: 0.000072\n", "2020-01-18 01:07:12,145 Epoch 30 Step: 216800 Batch Loss: 1.243498 Tokens per Sec: 19529, Lr: 0.000072\n", "2020-01-18 01:07:23,574 Epoch 30 Step: 216900 Batch Loss: 1.078862 Tokens per Sec: 19237, Lr: 0.000072\n", "2020-01-18 01:07:35,043 Epoch 30 Step: 217000 Batch Loss: 0.985709 Tokens per Sec: 19137, Lr: 0.000072\n", "2020-01-18 01:08:04,178 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:08:04,179 Saving new checkpoint.\n", "2020-01-18 01:08:05,786 Example #0\n", "2020-01-18 01:08:05,786 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:08:05,786 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:08:05,787 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:08:05,787 Example #1\n", "2020-01-18 01:08:05,787 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:08:05,787 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:08:05,787 \tHypothesis: A te lele Usi Uvie na No .\n", "2020-01-18 01:08:05,788 Example #2\n", "2020-01-18 01:08:05,788 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:08:05,788 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:08:05,789 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 01:08:05,789 Example #3\n", "2020-01-18 01:08:05,789 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:08:05,789 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:08:05,790 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:08:05,790 Validation result (greedy) at epoch 30, step 217000: bleu: 31.80, loss: 32480.0020, ppl: 3.5003, duration: 30.7464s\n", "2020-01-18 01:08:17,536 Epoch 30 Step: 217100 Batch Loss: 1.169954 Tokens per Sec: 18349, Lr: 0.000072\n", "2020-01-18 01:08:29,079 Epoch 30 Step: 217200 Batch Loss: 1.362291 Tokens per Sec: 19285, Lr: 0.000072\n", "2020-01-18 01:08:40,473 Epoch 30 Step: 217300 Batch Loss: 1.244033 Tokens per Sec: 19081, Lr: 0.000072\n", "2020-01-18 01:08:52,004 Epoch 30 Step: 217400 Batch Loss: 1.341730 Tokens per Sec: 19374, Lr: 0.000072\n", "2020-01-18 01:09:03,411 Epoch 30 Step: 217500 Batch Loss: 1.033991 Tokens per Sec: 19247, Lr: 0.000072\n", "2020-01-18 01:09:15,194 Epoch 30 Step: 217600 Batch Loss: 1.038673 Tokens per Sec: 19402, Lr: 0.000072\n", "2020-01-18 01:09:26,770 Epoch 30 Step: 217700 Batch Loss: 1.069124 Tokens per Sec: 19456, Lr: 0.000072\n", "2020-01-18 01:09:38,255 Epoch 30 Step: 217800 Batch Loss: 1.197029 Tokens per Sec: 19240, Lr: 0.000072\n", "2020-01-18 01:09:42,176 Epoch 30: total training loss 2971.95\n", "2020-01-18 01:09:42,177 EPOCH 31\n", "2020-01-18 01:09:49,972 Epoch 31 Step: 217900 Batch Loss: 1.412171 Tokens per Sec: 19264, Lr: 0.000072\n", "2020-01-18 01:10:01,338 Epoch 31 Step: 218000 Batch Loss: 1.059543 Tokens per Sec: 18994, Lr: 0.000072\n", "2020-01-18 01:10:30,468 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:10:30,468 Saving new checkpoint.\n", "2020-01-18 01:10:31,775 Example #0\n", "2020-01-18 01:10:31,776 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:10:31,776 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:10:31,776 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:10:31,776 Example #1\n", "2020-01-18 01:10:31,777 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:10:31,777 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:10:31,777 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:10:31,777 Example #2\n", "2020-01-18 01:10:31,778 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:10:31,778 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:10:31,778 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te vihọ , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 01:10:31,779 Example #3\n", "2020-01-18 01:10:31,779 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:10:31,779 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:10:31,779 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 01:10:31,780 Validation result (greedy) at epoch 31, step 218000: bleu: 31.84, loss: 32434.7285, ppl: 3.4942, duration: 30.4409s\n", "2020-01-18 01:10:43,703 Epoch 31 Step: 218100 Batch Loss: 1.357469 Tokens per Sec: 18952, Lr: 0.000072\n", "2020-01-18 01:10:55,053 Epoch 31 Step: 218200 Batch Loss: 1.173456 Tokens per Sec: 19175, Lr: 0.000072\n", "2020-01-18 01:11:06,562 Epoch 31 Step: 218300 Batch Loss: 0.984103 Tokens per Sec: 19581, Lr: 0.000072\n", "2020-01-18 01:11:18,094 Epoch 31 Step: 218400 Batch Loss: 1.269217 Tokens per Sec: 19524, Lr: 0.000072\n", "2020-01-18 01:11:29,486 Epoch 31 Step: 218500 Batch Loss: 1.409162 Tokens per Sec: 19048, Lr: 0.000072\n", "2020-01-18 01:11:41,007 Epoch 31 Step: 218600 Batch Loss: 1.067921 Tokens per Sec: 19229, Lr: 0.000072\n", "2020-01-18 01:11:52,483 Epoch 31 Step: 218700 Batch Loss: 1.253791 Tokens per Sec: 19753, Lr: 0.000072\n", "2020-01-18 01:12:03,930 Epoch 31 Step: 218800 Batch Loss: 1.215443 Tokens per Sec: 19247, Lr: 0.000072\n", "2020-01-18 01:12:15,640 Epoch 31 Step: 218900 Batch Loss: 1.159128 Tokens per Sec: 19761, Lr: 0.000072\n", "2020-01-18 01:12:27,095 Epoch 31 Step: 219000 Batch Loss: 1.237336 Tokens per Sec: 19322, Lr: 0.000072\n", "2020-01-18 01:12:56,290 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:12:56,290 Saving new checkpoint.\n", "2020-01-18 01:12:57,502 Example #0\n", "2020-01-18 01:12:57,503 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:12:57,503 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:12:57,504 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:12:57,504 Example #1\n", "2020-01-18 01:12:57,504 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:12:57,504 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:12:57,504 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:12:57,505 Example #2\n", "2020-01-18 01:12:57,505 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:12:57,505 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:12:57,506 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 01:12:57,506 Example #3\n", "2020-01-18 01:12:57,506 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:12:57,506 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:12:57,507 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 01:12:57,507 Validation result (greedy) at epoch 31, step 219000: bleu: 32.30, loss: 32366.5977, ppl: 3.4850, duration: 30.4109s\n", "2020-01-18 01:13:09,385 Epoch 31 Step: 219100 Batch Loss: 1.239947 Tokens per Sec: 18863, Lr: 0.000072\n", "2020-01-18 01:13:20,780 Epoch 31 Step: 219200 Batch Loss: 1.227579 Tokens per Sec: 19439, Lr: 0.000072\n", "2020-01-18 01:13:32,255 Epoch 31 Step: 219300 Batch Loss: 1.021120 Tokens per Sec: 19010, Lr: 0.000072\n", "2020-01-18 01:13:43,826 Epoch 31 Step: 219400 Batch Loss: 1.297503 Tokens per Sec: 19415, Lr: 0.000072\n", "2020-01-18 01:13:55,236 Epoch 31 Step: 219500 Batch Loss: 0.948979 Tokens per Sec: 19526, Lr: 0.000072\n", "2020-01-18 01:14:06,683 Epoch 31 Step: 219600 Batch Loss: 1.204172 Tokens per Sec: 19376, Lr: 0.000072\n", "2020-01-18 01:14:18,167 Epoch 31 Step: 219700 Batch Loss: 1.291466 Tokens per Sec: 19505, Lr: 0.000072\n", "2020-01-18 01:14:29,752 Epoch 31 Step: 219800 Batch Loss: 1.112710 Tokens per Sec: 19197, Lr: 0.000072\n", "2020-01-18 01:14:41,160 Epoch 31 Step: 219900 Batch Loss: 0.821829 Tokens per Sec: 19083, Lr: 0.000072\n", "2020-01-18 01:14:52,757 Epoch 31 Step: 220000 Batch Loss: 1.270613 Tokens per Sec: 19682, Lr: 0.000072\n", "2020-01-18 01:15:21,980 Example #0\n", "2020-01-18 01:15:21,980 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:15:21,981 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:15:21,981 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:15:21,981 Example #1\n", "2020-01-18 01:15:21,981 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:15:21,982 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:15:21,982 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:15:21,982 Example #2\n", "2020-01-18 01:15:21,982 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:15:21,982 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:15:21,983 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva aro riẹ , re ọ ghale ae ; jọ ọ ghale ae ; wha kẹ ahwo Egedhọ kpobi .\n", "2020-01-18 01:15:21,983 Example #3\n", "2020-01-18 01:15:21,983 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:15:21,983 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:15:21,983 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:15:21,984 Validation result (greedy) at epoch 31, step 220000: bleu: 32.05, loss: 32395.3945, ppl: 3.4889, duration: 29.2257s\n", "2020-01-18 01:15:33,520 Epoch 31 Step: 220100 Batch Loss: 1.039361 Tokens per Sec: 18904, Lr: 0.000072\n", "2020-01-18 01:15:45,016 Epoch 31 Step: 220200 Batch Loss: 1.021366 Tokens per Sec: 19467, Lr: 0.000072\n", "2020-01-18 01:15:56,470 Epoch 31 Step: 220300 Batch Loss: 1.063344 Tokens per Sec: 19467, Lr: 0.000072\n", "2020-01-18 01:16:03,344 Epoch 31: total training loss 2988.38\n", "2020-01-18 01:16:03,344 EPOCH 32\n", "2020-01-18 01:16:08,235 Epoch 32 Step: 220400 Batch Loss: 1.290668 Tokens per Sec: 17994, Lr: 0.000072\n", "2020-01-18 01:16:19,666 Epoch 32 Step: 220500 Batch Loss: 1.262649 Tokens per Sec: 19348, Lr: 0.000072\n", "2020-01-18 01:16:31,214 Epoch 32 Step: 220600 Batch Loss: 1.258557 Tokens per Sec: 19403, Lr: 0.000072\n", "2020-01-18 01:16:42,619 Epoch 32 Step: 220700 Batch Loss: 1.400679 Tokens per Sec: 18983, Lr: 0.000072\n", "2020-01-18 01:16:54,033 Epoch 32 Step: 220800 Batch Loss: 1.065515 Tokens per Sec: 19152, Lr: 0.000072\n", "2020-01-18 01:17:05,427 Epoch 32 Step: 220900 Batch Loss: 0.981061 Tokens per Sec: 19506, Lr: 0.000072\n", "2020-01-18 01:17:17,018 Epoch 32 Step: 221000 Batch Loss: 1.295386 Tokens per Sec: 19156, Lr: 0.000072\n", "2020-01-18 01:17:46,115 Example #0\n", "2020-01-18 01:17:46,116 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:17:46,116 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:17:46,117 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:17:46,117 Example #1\n", "2020-01-18 01:17:46,117 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:17:46,117 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:17:46,118 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:17:46,118 Example #2\n", "2020-01-18 01:17:46,118 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:17:46,118 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:17:46,118 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; wha jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 01:17:46,118 Example #3\n", "2020-01-18 01:17:46,119 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:17:46,119 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:17:46,119 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:17:46,119 Validation result (greedy) at epoch 32, step 221000: bleu: 31.81, loss: 32392.7031, ppl: 3.4885, duration: 29.1011s\n", "2020-01-18 01:17:57,644 Epoch 32 Step: 221100 Batch Loss: 1.167038 Tokens per Sec: 19259, Lr: 0.000072\n", "2020-01-18 01:18:09,302 Epoch 32 Step: 221200 Batch Loss: 1.496487 Tokens per Sec: 19375, Lr: 0.000072\n", "2020-01-18 01:18:20,792 Epoch 32 Step: 221300 Batch Loss: 1.313065 Tokens per Sec: 19671, Lr: 0.000072\n", "2020-01-18 01:18:32,268 Epoch 32 Step: 221400 Batch Loss: 1.170732 Tokens per Sec: 19514, Lr: 0.000072\n", "2020-01-18 01:18:43,814 Epoch 32 Step: 221500 Batch Loss: 1.223384 Tokens per Sec: 19569, Lr: 0.000072\n", "2020-01-18 01:18:55,247 Epoch 32 Step: 221600 Batch Loss: 1.097842 Tokens per Sec: 19366, Lr: 0.000072\n", "2020-01-18 01:19:06,684 Epoch 32 Step: 221700 Batch Loss: 1.292664 Tokens per Sec: 19134, Lr: 0.000072\n", "2020-01-18 01:19:18,158 Epoch 32 Step: 221800 Batch Loss: 1.210566 Tokens per Sec: 19312, Lr: 0.000072\n", "2020-01-18 01:19:29,610 Epoch 32 Step: 221900 Batch Loss: 1.276534 Tokens per Sec: 19443, Lr: 0.000072\n", "2020-01-18 01:19:41,347 Epoch 32 Step: 222000 Batch Loss: 0.801273 Tokens per Sec: 19423, Lr: 0.000072\n", "2020-01-18 01:20:10,498 Example #0\n", "2020-01-18 01:20:10,499 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:20:10,500 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:20:10,500 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:20:10,500 Example #1\n", "2020-01-18 01:20:10,500 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:20:10,500 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:20:10,500 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:20:10,501 Example #2\n", "2020-01-18 01:20:10,501 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:20:10,501 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:20:10,501 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via , re o je vihọ , re ọ ghale ae ; wha jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 01:20:10,501 Example #3\n", "2020-01-18 01:20:10,502 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:20:10,502 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:20:10,502 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:20:10,502 Validation result (greedy) at epoch 32, step 222000: bleu: 32.10, loss: 32367.4336, ppl: 3.4851, duration: 29.1550s\n", "2020-01-18 01:20:22,014 Epoch 32 Step: 222100 Batch Loss: 1.168340 Tokens per Sec: 19534, Lr: 0.000072\n", "2020-01-18 01:20:33,408 Epoch 32 Step: 222200 Batch Loss: 1.194444 Tokens per Sec: 19047, Lr: 0.000072\n", "2020-01-18 01:20:45,023 Epoch 32 Step: 222300 Batch Loss: 1.159809 Tokens per Sec: 19291, Lr: 0.000072\n", "2020-01-18 01:20:56,455 Epoch 32 Step: 222400 Batch Loss: 1.061316 Tokens per Sec: 19704, Lr: 0.000072\n", "2020-01-18 01:21:07,958 Epoch 32 Step: 222500 Batch Loss: 1.102813 Tokens per Sec: 18888, Lr: 0.000072\n", "2020-01-18 01:21:19,440 Epoch 32 Step: 222600 Batch Loss: 1.285910 Tokens per Sec: 19857, Lr: 0.000072\n", "2020-01-18 01:21:30,944 Epoch 32 Step: 222700 Batch Loss: 1.278294 Tokens per Sec: 19332, Lr: 0.000072\n", "2020-01-18 01:21:42,459 Epoch 32 Step: 222800 Batch Loss: 1.298539 Tokens per Sec: 19188, Lr: 0.000072\n", "2020-01-18 01:21:52,342 Epoch 32: total training loss 2982.63\n", "2020-01-18 01:21:52,343 EPOCH 33\n", "2020-01-18 01:21:54,181 Epoch 33 Step: 222900 Batch Loss: 1.118222 Tokens per Sec: 16993, Lr: 0.000072\n", "2020-01-18 01:22:05,662 Epoch 33 Step: 223000 Batch Loss: 0.884293 Tokens per Sec: 19563, Lr: 0.000072\n", "2020-01-18 01:22:34,784 Example #0\n", "2020-01-18 01:22:34,785 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:22:34,785 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:22:34,785 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:22:34,785 Example #1\n", "2020-01-18 01:22:34,786 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:22:34,786 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:22:34,786 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:22:34,786 Example #2\n", "2020-01-18 01:22:34,787 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:22:34,787 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:22:34,787 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; wha jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 01:22:34,787 Example #3\n", "2020-01-18 01:22:34,788 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:22:34,788 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:22:34,788 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:22:34,788 Validation result (greedy) at epoch 33, step 223000: bleu: 31.98, loss: 32454.5957, ppl: 3.4969, duration: 29.1258s\n", "2020-01-18 01:22:46,322 Epoch 33 Step: 223100 Batch Loss: 1.235407 Tokens per Sec: 19024, Lr: 0.000072\n", "2020-01-18 01:22:57,805 Epoch 33 Step: 223200 Batch Loss: 1.116643 Tokens per Sec: 19604, Lr: 0.000072\n", "2020-01-18 01:23:09,327 Epoch 33 Step: 223300 Batch Loss: 1.152722 Tokens per Sec: 19260, Lr: 0.000072\n", "2020-01-18 01:23:20,830 Epoch 33 Step: 223400 Batch Loss: 1.127280 Tokens per Sec: 19772, Lr: 0.000072\n", "2020-01-18 01:23:32,299 Epoch 33 Step: 223500 Batch Loss: 1.214467 Tokens per Sec: 19576, Lr: 0.000072\n", "2020-01-18 01:23:43,818 Epoch 33 Step: 223600 Batch Loss: 0.957459 Tokens per Sec: 19210, Lr: 0.000072\n", "2020-01-18 01:23:55,206 Epoch 33 Step: 223700 Batch Loss: 1.310325 Tokens per Sec: 19375, Lr: 0.000072\n", "2020-01-18 01:24:06,749 Epoch 33 Step: 223800 Batch Loss: 0.854199 Tokens per Sec: 19476, Lr: 0.000072\n", "2020-01-18 01:24:18,380 Epoch 33 Step: 223900 Batch Loss: 1.327492 Tokens per Sec: 19648, Lr: 0.000072\n", "2020-01-18 01:24:29,762 Epoch 33 Step: 224000 Batch Loss: 1.184726 Tokens per Sec: 19305, Lr: 0.000072\n", "2020-01-18 01:24:58,858 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:24:58,858 Saving new checkpoint.\n", "2020-01-18 01:25:00,051 Example #0\n", "2020-01-18 01:25:00,052 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:25:00,052 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:25:00,052 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:25:00,052 Example #1\n", "2020-01-18 01:25:00,053 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:25:00,053 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:25:00,053 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:25:00,053 Example #2\n", "2020-01-18 01:25:00,054 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:25:00,054 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:25:00,054 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jiri ahwo Egedhọ kpobi ; wha wo oghale .\n", "2020-01-18 01:25:00,054 Example #3\n", "2020-01-18 01:25:00,055 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:25:00,055 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:25:00,055 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:25:00,055 Validation result (greedy) at epoch 33, step 224000: bleu: 32.16, loss: 32331.9688, ppl: 3.4804, duration: 30.2926s\n", "2020-01-18 01:25:11,775 Epoch 33 Step: 224100 Batch Loss: 1.128115 Tokens per Sec: 18775, Lr: 0.000072\n", "2020-01-18 01:25:23,345 Epoch 33 Step: 224200 Batch Loss: 1.229844 Tokens per Sec: 19197, Lr: 0.000072\n", "2020-01-18 01:25:34,961 Epoch 33 Step: 224300 Batch Loss: 1.027113 Tokens per Sec: 19297, Lr: 0.000072\n", "2020-01-18 01:25:46,574 Epoch 33 Step: 224400 Batch Loss: 1.247756 Tokens per Sec: 19381, Lr: 0.000072\n", "2020-01-18 01:25:57,914 Epoch 33 Step: 224500 Batch Loss: 1.264403 Tokens per Sec: 19178, Lr: 0.000072\n", "2020-01-18 01:26:09,312 Epoch 33 Step: 224600 Batch Loss: 1.381401 Tokens per Sec: 18813, Lr: 0.000072\n", "2020-01-18 01:26:20,729 Epoch 33 Step: 224700 Batch Loss: 1.270968 Tokens per Sec: 19630, Lr: 0.000072\n", "2020-01-18 01:26:32,150 Epoch 33 Step: 224800 Batch Loss: 1.131922 Tokens per Sec: 19212, Lr: 0.000072\n", "2020-01-18 01:26:43,629 Epoch 33 Step: 224900 Batch Loss: 1.132361 Tokens per Sec: 18893, Lr: 0.000072\n", "2020-01-18 01:26:55,158 Epoch 33 Step: 225000 Batch Loss: 1.315496 Tokens per Sec: 19193, Lr: 0.000072\n", "2020-01-18 01:27:24,316 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:27:24,316 Saving new checkpoint.\n", "2020-01-18 01:27:25,554 Example #0\n", "2020-01-18 01:27:25,555 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:27:25,555 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:27:25,555 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:27:25,555 Example #1\n", "2020-01-18 01:27:25,555 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:27:25,556 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:27:25,556 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:27:25,556 Example #2\n", "2020-01-18 01:27:25,556 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:27:25,557 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:27:25,557 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jọ kẹ erẹwho kpobi ; re o wo oghale .\n", "2020-01-18 01:27:25,557 Example #3\n", "2020-01-18 01:27:25,557 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:27:25,558 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:27:25,558 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:27:25,558 Validation result (greedy) at epoch 33, step 225000: bleu: 32.18, loss: 32315.2344, ppl: 3.4781, duration: 30.3998s\n", "2020-01-18 01:27:37,199 Epoch 33 Step: 225100 Batch Loss: 1.059520 Tokens per Sec: 18897, Lr: 0.000072\n", "2020-01-18 01:27:48,800 Epoch 33 Step: 225200 Batch Loss: 1.339109 Tokens per Sec: 19607, Lr: 0.000072\n", "2020-01-18 01:28:00,238 Epoch 33 Step: 225300 Batch Loss: 0.967830 Tokens per Sec: 19130, Lr: 0.000072\n", "2020-01-18 01:28:11,698 Epoch 33 Step: 225400 Batch Loss: 1.311101 Tokens per Sec: 19004, Lr: 0.000072\n", "2020-01-18 01:28:13,902 Epoch 33: total training loss 2987.07\n", "2020-01-18 01:28:13,902 EPOCH 34\n", "2020-01-18 01:28:23,291 Epoch 34 Step: 225500 Batch Loss: 1.120255 Tokens per Sec: 18345, Lr: 0.000072\n", "2020-01-18 01:28:34,835 Epoch 34 Step: 225600 Batch Loss: 1.204295 Tokens per Sec: 19497, Lr: 0.000072\n", "2020-01-18 01:28:46,263 Epoch 34 Step: 225700 Batch Loss: 1.466729 Tokens per Sec: 19496, Lr: 0.000072\n", "2020-01-18 01:28:57,760 Epoch 34 Step: 225800 Batch Loss: 1.236129 Tokens per Sec: 19636, Lr: 0.000072\n", "2020-01-18 01:29:09,399 Epoch 34 Step: 225900 Batch Loss: 1.085923 Tokens per Sec: 19543, Lr: 0.000072\n", "2020-01-18 01:29:20,876 Epoch 34 Step: 226000 Batch Loss: 1.114246 Tokens per Sec: 19601, Lr: 0.000072\n", "2020-01-18 01:29:50,064 Example #0\n", "2020-01-18 01:29:50,064 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:29:50,065 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:29:50,065 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:29:50,065 Example #1\n", "2020-01-18 01:29:50,065 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:29:50,065 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:29:50,066 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:29:50,066 Example #2\n", "2020-01-18 01:29:50,066 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:29:50,066 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:29:50,066 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re erẹwho kpobi i wo oghale ; wha jọ uyere kẹ ae .\n", "2020-01-18 01:29:50,066 Example #3\n", "2020-01-18 01:29:50,067 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:29:50,067 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:29:50,067 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:29:50,067 Validation result (greedy) at epoch 34, step 226000: bleu: 32.08, loss: 32347.1387, ppl: 3.4824, duration: 29.1912s\n", "2020-01-18 01:30:01,641 Epoch 34 Step: 226100 Batch Loss: 1.008750 Tokens per Sec: 19266, Lr: 0.000072\n", "2020-01-18 01:30:13,257 Epoch 34 Step: 226200 Batch Loss: 1.247161 Tokens per Sec: 19577, Lr: 0.000072\n", "2020-01-18 01:30:24,847 Epoch 34 Step: 226300 Batch Loss: 1.214536 Tokens per Sec: 19667, Lr: 0.000072\n", "2020-01-18 01:30:36,296 Epoch 34 Step: 226400 Batch Loss: 1.344067 Tokens per Sec: 19266, Lr: 0.000072\n", "2020-01-18 01:30:47,710 Epoch 34 Step: 226500 Batch Loss: 1.181662 Tokens per Sec: 19142, Lr: 0.000072\n", "2020-01-18 01:30:59,153 Epoch 34 Step: 226600 Batch Loss: 0.690382 Tokens per Sec: 19154, Lr: 0.000072\n", "2020-01-18 01:31:10,674 Epoch 34 Step: 226700 Batch Loss: 1.295473 Tokens per Sec: 19184, Lr: 0.000072\n", "2020-01-18 01:31:21,994 Epoch 34 Step: 226800 Batch Loss: 0.953092 Tokens per Sec: 19412, Lr: 0.000072\n", "2020-01-18 01:31:33,584 Epoch 34 Step: 226900 Batch Loss: 1.162551 Tokens per Sec: 19676, Lr: 0.000072\n", "2020-01-18 01:31:45,071 Epoch 34 Step: 227000 Batch Loss: 1.061814 Tokens per Sec: 19403, Lr: 0.000072\n", "2020-01-18 01:32:14,236 Example #0\n", "2020-01-18 01:32:14,236 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:32:14,237 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:32:14,237 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:32:14,237 Example #1\n", "2020-01-18 01:32:14,238 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:32:14,238 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:32:14,238 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 01:32:14,238 Example #2\n", "2020-01-18 01:32:14,239 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:32:14,239 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:32:14,239 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te vihọ , re o wo oghale , re ọ ghale ae ; re erẹwho na kpobi i wo oghale .\n", "2020-01-18 01:32:14,239 Example #3\n", "2020-01-18 01:32:14,239 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:32:14,239 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:32:14,240 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:32:14,240 Validation result (greedy) at epoch 34, step 227000: bleu: 32.23, loss: 32336.6152, ppl: 3.4810, duration: 29.1678s\n", "2020-01-18 01:32:25,740 Epoch 34 Step: 227100 Batch Loss: 1.256965 Tokens per Sec: 19425, Lr: 0.000072\n", "2020-01-18 01:32:37,146 Epoch 34 Step: 227200 Batch Loss: 1.172816 Tokens per Sec: 19227, Lr: 0.000072\n", "2020-01-18 01:32:48,677 Epoch 34 Step: 227300 Batch Loss: 1.256194 Tokens per Sec: 19359, Lr: 0.000072\n", "2020-01-18 01:33:00,170 Epoch 34 Step: 227400 Batch Loss: 1.177886 Tokens per Sec: 19698, Lr: 0.000072\n", "2020-01-18 01:33:11,627 Epoch 34 Step: 227500 Batch Loss: 1.037908 Tokens per Sec: 18921, Lr: 0.000072\n", "2020-01-18 01:33:23,022 Epoch 34 Step: 227600 Batch Loss: 1.192304 Tokens per Sec: 19306, Lr: 0.000072\n", "2020-01-18 01:33:34,546 Epoch 34 Step: 227700 Batch Loss: 1.198653 Tokens per Sec: 19324, Lr: 0.000072\n", "2020-01-18 01:33:46,036 Epoch 34 Step: 227800 Batch Loss: 1.175970 Tokens per Sec: 19313, Lr: 0.000072\n", "2020-01-18 01:33:57,445 Epoch 34 Step: 227900 Batch Loss: 1.326285 Tokens per Sec: 19539, Lr: 0.000072\n", "2020-01-18 01:34:02,588 Epoch 34: total training loss 2972.98\n", "2020-01-18 01:34:02,589 EPOCH 35\n", "2020-01-18 01:34:09,245 Epoch 35 Step: 228000 Batch Loss: 1.401123 Tokens per Sec: 18110, Lr: 0.000072\n", "2020-01-18 01:34:38,381 Example #0\n", "2020-01-18 01:34:38,382 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:34:38,382 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:34:38,382 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:34:38,382 Example #1\n", "2020-01-18 01:34:38,383 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:34:38,383 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:34:38,383 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:34:38,383 Example #2\n", "2020-01-18 01:34:38,384 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:34:38,384 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:34:38,384 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re ọ jẹ ghale ae ; jọ ọ ghale ae ; wha kẹ ae re o wo oghale .\n", "2020-01-18 01:34:38,384 Example #3\n", "2020-01-18 01:34:38,384 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:34:38,384 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:34:38,385 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:34:38,385 Validation result (greedy) at epoch 35, step 228000: bleu: 32.01, loss: 32341.7031, ppl: 3.4817, duration: 29.1395s\n", "2020-01-18 01:34:49,935 Epoch 35 Step: 228100 Batch Loss: 1.197477 Tokens per Sec: 19194, Lr: 0.000072\n", "2020-01-18 01:35:01,546 Epoch 35 Step: 228200 Batch Loss: 1.277011 Tokens per Sec: 19620, Lr: 0.000072\n", "2020-01-18 01:35:13,280 Epoch 35 Step: 228300 Batch Loss: 1.181726 Tokens per Sec: 18976, Lr: 0.000072\n", "2020-01-18 01:35:24,744 Epoch 35 Step: 228400 Batch Loss: 1.216959 Tokens per Sec: 19468, Lr: 0.000072\n", "2020-01-18 01:35:36,172 Epoch 35 Step: 228500 Batch Loss: 1.199251 Tokens per Sec: 19713, Lr: 0.000072\n", "2020-01-18 01:35:47,648 Epoch 35 Step: 228600 Batch Loss: 1.499921 Tokens per Sec: 18964, Lr: 0.000072\n", "2020-01-18 01:35:59,116 Epoch 35 Step: 228700 Batch Loss: 1.242465 Tokens per Sec: 19201, Lr: 0.000072\n", "2020-01-18 01:36:10,702 Epoch 35 Step: 228800 Batch Loss: 1.423915 Tokens per Sec: 19231, Lr: 0.000072\n", "2020-01-18 01:36:22,229 Epoch 35 Step: 228900 Batch Loss: 1.322920 Tokens per Sec: 19634, Lr: 0.000072\n", "2020-01-18 01:36:33,766 Epoch 35 Step: 229000 Batch Loss: 1.156968 Tokens per Sec: 19489, Lr: 0.000072\n", "2020-01-18 01:37:02,801 Example #0\n", "2020-01-18 01:37:02,801 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:37:02,801 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:37:02,801 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:37:02,802 Example #1\n", "2020-01-18 01:37:02,802 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:37:02,802 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:37:02,802 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:37:02,803 Example #2\n", "2020-01-18 01:37:02,803 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:37:02,803 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:37:02,804 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ o jọ rọ kẹ erẹwho kpobi ; re o jọ rọ kẹ ae evawere .\n", "2020-01-18 01:37:02,804 Example #3\n", "2020-01-18 01:37:02,804 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:37:02,804 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:37:02,804 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:37:02,804 Validation result (greedy) at epoch 35, step 229000: bleu: 32.19, loss: 32335.5078, ppl: 3.4808, duration: 29.0383s\n", "2020-01-18 01:37:14,303 Epoch 35 Step: 229100 Batch Loss: 1.468629 Tokens per Sec: 18982, Lr: 0.000072\n", "2020-01-18 01:37:25,806 Epoch 35 Step: 229200 Batch Loss: 1.213004 Tokens per Sec: 19286, Lr: 0.000072\n", "2020-01-18 01:37:37,325 Epoch 35 Step: 229300 Batch Loss: 0.928927 Tokens per Sec: 19640, Lr: 0.000072\n", "2020-01-18 01:37:48,808 Epoch 35 Step: 229400 Batch Loss: 1.096699 Tokens per Sec: 19443, Lr: 0.000072\n", "2020-01-18 01:38:00,377 Epoch 35 Step: 229500 Batch Loss: 1.238749 Tokens per Sec: 19728, Lr: 0.000072\n", "2020-01-18 01:38:12,005 Epoch 35 Step: 229600 Batch Loss: 1.150131 Tokens per Sec: 19362, Lr: 0.000072\n", "2020-01-18 01:38:23,414 Epoch 35 Step: 229700 Batch Loss: 1.261060 Tokens per Sec: 19282, Lr: 0.000072\n", "2020-01-18 01:38:34,888 Epoch 35 Step: 229800 Batch Loss: 1.105652 Tokens per Sec: 19365, Lr: 0.000072\n", "2020-01-18 01:38:46,358 Epoch 35 Step: 229900 Batch Loss: 1.225060 Tokens per Sec: 19278, Lr: 0.000072\n", "2020-01-18 01:38:57,836 Epoch 35 Step: 230000 Batch Loss: 1.111792 Tokens per Sec: 19209, Lr: 0.000072\n", "2020-01-18 01:39:27,063 Example #0\n", "2020-01-18 01:39:27,064 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:39:27,064 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:39:27,065 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:39:27,065 Example #1\n", "2020-01-18 01:39:27,065 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:39:27,065 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:39:27,066 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:39:27,066 Example #2\n", "2020-01-18 01:39:27,066 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:39:27,066 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:39:27,067 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via , re ọ ghale ae ; jọ o jọ rọ kẹ erẹwho kpobi ; wha wo oghale .\n", "2020-01-18 01:39:27,067 Example #3\n", "2020-01-18 01:39:27,067 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:39:27,067 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:39:27,067 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:39:27,068 Validation result (greedy) at epoch 35, step 230000: bleu: 32.11, loss: 32321.1406, ppl: 3.4789, duration: 29.2309s\n", "2020-01-18 01:39:38,541 Epoch 35 Step: 230100 Batch Loss: 0.851419 Tokens per Sec: 19159, Lr: 0.000072\n", "2020-01-18 01:39:49,946 Epoch 35 Step: 230200 Batch Loss: 1.330287 Tokens per Sec: 19265, Lr: 0.000072\n", "2020-01-18 01:40:01,383 Epoch 35 Step: 230300 Batch Loss: 1.270630 Tokens per Sec: 19870, Lr: 0.000072\n", "2020-01-18 01:40:12,991 Epoch 35 Step: 230400 Batch Loss: 0.784012 Tokens per Sec: 19076, Lr: 0.000072\n", "2020-01-18 01:40:21,073 Epoch 35: total training loss 2973.15\n", "2020-01-18 01:40:21,073 EPOCH 36\n", "2020-01-18 01:40:24,782 Epoch 36 Step: 230500 Batch Loss: 1.330776 Tokens per Sec: 18462, Lr: 0.000072\n", "2020-01-18 01:40:36,211 Epoch 36 Step: 230600 Batch Loss: 0.883623 Tokens per Sec: 19208, Lr: 0.000072\n", "2020-01-18 01:40:47,680 Epoch 36 Step: 230700 Batch Loss: 1.276712 Tokens per Sec: 19402, Lr: 0.000072\n", "2020-01-18 01:40:59,078 Epoch 36 Step: 230800 Batch Loss: 1.178676 Tokens per Sec: 19467, Lr: 0.000072\n", "2020-01-18 01:41:10,704 Epoch 36 Step: 230900 Batch Loss: 1.179537 Tokens per Sec: 19396, Lr: 0.000072\n", "2020-01-18 01:41:22,106 Epoch 36 Step: 231000 Batch Loss: 1.164661 Tokens per Sec: 19274, Lr: 0.000072\n", "2020-01-18 01:41:51,229 Example #0\n", "2020-01-18 01:41:51,229 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:41:51,230 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:41:51,230 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:41:51,230 Example #1\n", "2020-01-18 01:41:51,230 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:41:51,230 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:41:51,231 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:41:51,231 Example #2\n", "2020-01-18 01:41:51,231 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:41:51,231 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:41:51,231 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo via eva riẹ , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 01:41:51,232 Example #3\n", "2020-01-18 01:41:51,232 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:41:51,232 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:41:51,232 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 01:41:51,232 Validation result (greedy) at epoch 36, step 231000: bleu: 31.98, loss: 32328.3906, ppl: 3.4799, duration: 29.1257s\n", "2020-01-18 01:42:02,755 Epoch 36 Step: 231100 Batch Loss: 0.985065 Tokens per Sec: 19548, Lr: 0.000050\n", "2020-01-18 01:42:14,316 Epoch 36 Step: 231200 Batch Loss: 1.266993 Tokens per Sec: 19162, Lr: 0.000050\n", "2020-01-18 01:42:25,850 Epoch 36 Step: 231300 Batch Loss: 1.063990 Tokens per Sec: 19654, Lr: 0.000050\n", "2020-01-18 01:42:37,279 Epoch 36 Step: 231400 Batch Loss: 1.224229 Tokens per Sec: 19072, Lr: 0.000050\n", "2020-01-18 01:42:48,729 Epoch 36 Step: 231500 Batch Loss: 1.031814 Tokens per Sec: 19492, Lr: 0.000050\n", "2020-01-18 01:43:00,277 Epoch 36 Step: 231600 Batch Loss: 1.156254 Tokens per Sec: 19392, Lr: 0.000050\n", "2020-01-18 01:43:11,792 Epoch 36 Step: 231700 Batch Loss: 1.277464 Tokens per Sec: 19293, Lr: 0.000050\n", "2020-01-18 01:43:23,176 Epoch 36 Step: 231800 Batch Loss: 1.218121 Tokens per Sec: 19057, Lr: 0.000050\n", "2020-01-18 01:43:34,695 Epoch 36 Step: 231900 Batch Loss: 1.122657 Tokens per Sec: 19456, Lr: 0.000050\n", "2020-01-18 01:43:46,082 Epoch 36 Step: 232000 Batch Loss: 1.247668 Tokens per Sec: 18869, Lr: 0.000050\n", "2020-01-18 01:44:15,208 Example #0\n", "2020-01-18 01:44:15,209 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:44:15,209 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:44:15,209 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:44:15,210 Example #1\n", "2020-01-18 01:44:15,210 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:44:15,210 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:44:15,210 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:44:15,210 Example #2\n", "2020-01-18 01:44:15,211 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:44:15,211 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:44:15,211 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ekuhọ ọrọ abọ urere ọrọ ole owoma nana nọ o rẹ sasa oma na : “ Jọ odẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo , re ahwo kpobi a wo oghale , re ahwo a wo oghale eva e were ae .\n", "2020-01-18 01:44:15,211 Example #3\n", "2020-01-18 01:44:15,212 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:44:15,212 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:44:15,212 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:44:15,212 Validation result (greedy) at epoch 36, step 232000: bleu: 32.25, loss: 32316.3945, ppl: 3.4783, duration: 29.1297s\n", "2020-01-18 01:44:26,697 Epoch 36 Step: 232100 Batch Loss: 1.249709 Tokens per Sec: 19317, Lr: 0.000050\n", "2020-01-18 01:44:38,226 Epoch 36 Step: 232200 Batch Loss: 1.049826 Tokens per Sec: 19249, Lr: 0.000050\n", "2020-01-18 01:44:49,845 Epoch 36 Step: 232300 Batch Loss: 1.353996 Tokens per Sec: 19357, Lr: 0.000050\n", "2020-01-18 01:45:01,356 Epoch 36 Step: 232400 Batch Loss: 1.437554 Tokens per Sec: 19901, Lr: 0.000050\n", "2020-01-18 01:45:12,930 Epoch 36 Step: 232500 Batch Loss: 1.125958 Tokens per Sec: 19346, Lr: 0.000050\n", "2020-01-18 01:45:24,493 Epoch 36 Step: 232600 Batch Loss: 1.183576 Tokens per Sec: 19618, Lr: 0.000050\n", "2020-01-18 01:45:36,167 Epoch 36 Step: 232700 Batch Loss: 1.260518 Tokens per Sec: 19392, Lr: 0.000050\n", "2020-01-18 01:45:47,755 Epoch 36 Step: 232800 Batch Loss: 1.239209 Tokens per Sec: 19295, Lr: 0.000050\n", "2020-01-18 01:45:59,076 Epoch 36 Step: 232900 Batch Loss: 1.091391 Tokens per Sec: 18797, Lr: 0.000050\n", "2020-01-18 01:46:10,415 Epoch 36: total training loss 2966.34\n", "2020-01-18 01:46:10,416 EPOCH 37\n", "2020-01-18 01:46:10,896 Epoch 37 Step: 233000 Batch Loss: 1.138633 Tokens per Sec: 8882, Lr: 0.000050\n", "2020-01-18 01:46:39,982 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:46:39,982 Saving new checkpoint.\n", "2020-01-18 01:46:41,292 Example #0\n", "2020-01-18 01:46:41,293 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:46:41,293 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:46:41,293 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:46:41,294 Example #1\n", "2020-01-18 01:46:41,294 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:46:41,295 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:46:41,295 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:46:41,295 Example #2\n", "2020-01-18 01:46:41,295 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:46:41,296 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:46:41,296 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oghale , re erẹwho kpobi i wo oghale ; wha kẹ ae uyere .\n", "2020-01-18 01:46:41,296 Example #3\n", "2020-01-18 01:46:41,296 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:46:41,297 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:46:41,297 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:46:41,297 Validation result (greedy) at epoch 37, step 233000: bleu: 32.18, loss: 32303.5859, ppl: 3.4765, duration: 30.4007s\n", "2020-01-18 01:46:52,867 Epoch 37 Step: 233100 Batch Loss: 1.263318 Tokens per Sec: 19135, Lr: 0.000050\n", "2020-01-18 01:47:04,430 Epoch 37 Step: 233200 Batch Loss: 1.061705 Tokens per Sec: 19019, Lr: 0.000050\n", "2020-01-18 01:47:15,970 Epoch 37 Step: 233300 Batch Loss: 1.220516 Tokens per Sec: 19285, Lr: 0.000050\n", "2020-01-18 01:47:27,485 Epoch 37 Step: 233400 Batch Loss: 1.390829 Tokens per Sec: 19453, Lr: 0.000050\n", "2020-01-18 01:47:38,936 Epoch 37 Step: 233500 Batch Loss: 1.320957 Tokens per Sec: 19627, Lr: 0.000050\n", "2020-01-18 01:47:50,342 Epoch 37 Step: 233600 Batch Loss: 1.119900 Tokens per Sec: 19176, Lr: 0.000050\n", "2020-01-18 01:48:01,846 Epoch 37 Step: 233700 Batch Loss: 1.035501 Tokens per Sec: 19458, Lr: 0.000050\n", "2020-01-18 01:48:13,425 Epoch 37 Step: 233800 Batch Loss: 0.999152 Tokens per Sec: 19307, Lr: 0.000050\n", "2020-01-18 01:48:24,946 Epoch 37 Step: 233900 Batch Loss: 1.367460 Tokens per Sec: 19263, Lr: 0.000050\n", "2020-01-18 01:48:36,354 Epoch 37 Step: 234000 Batch Loss: 1.102550 Tokens per Sec: 19322, Lr: 0.000050\n", "2020-01-18 01:49:05,476 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:49:05,477 Saving new checkpoint.\n", "2020-01-18 01:49:06,768 Example #0\n", "2020-01-18 01:49:06,769 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:49:06,770 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:49:06,770 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:49:06,770 Example #1\n", "2020-01-18 01:49:06,771 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:49:06,771 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:49:06,771 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:49:06,772 Example #2\n", "2020-01-18 01:49:06,779 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:49:06,779 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:49:06,780 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re erẹwho kpobi i wo oghale ; wha kẹ ae uyere .\n", "2020-01-18 01:49:06,780 Example #3\n", "2020-01-18 01:49:06,780 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:49:06,781 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:49:06,781 \tHypothesis: Ghele na , eme unu - uwou e rẹ kẹ ẹgba gaga nọ ma re ro ru udhedhẹ .\n", "2020-01-18 01:49:06,781 Validation result (greedy) at epoch 37, step 234000: bleu: 32.05, loss: 32249.6055, ppl: 3.4693, duration: 30.4265s\n", "2020-01-18 01:49:18,522 Epoch 37 Step: 234100 Batch Loss: 1.259529 Tokens per Sec: 19172, Lr: 0.000050\n", "2020-01-18 01:49:30,205 Epoch 37 Step: 234200 Batch Loss: 1.228032 Tokens per Sec: 19183, Lr: 0.000050\n", "2020-01-18 01:49:41,602 Epoch 37 Step: 234300 Batch Loss: 1.098232 Tokens per Sec: 18902, Lr: 0.000050\n", "2020-01-18 01:49:52,993 Epoch 37 Step: 234400 Batch Loss: 1.019199 Tokens per Sec: 19542, Lr: 0.000050\n", "2020-01-18 01:50:04,490 Epoch 37 Step: 234500 Batch Loss: 1.219249 Tokens per Sec: 19391, Lr: 0.000050\n", "2020-01-18 01:50:16,029 Epoch 37 Step: 234600 Batch Loss: 1.013298 Tokens per Sec: 19190, Lr: 0.000050\n", "2020-01-18 01:50:27,561 Epoch 37 Step: 234700 Batch Loss: 1.247661 Tokens per Sec: 19356, Lr: 0.000050\n", "2020-01-18 01:50:39,075 Epoch 37 Step: 234800 Batch Loss: 1.096608 Tokens per Sec: 19393, Lr: 0.000050\n", "2020-01-18 01:50:50,715 Epoch 37 Step: 234900 Batch Loss: 1.282461 Tokens per Sec: 19542, Lr: 0.000050\n", "2020-01-18 01:51:02,132 Epoch 37 Step: 235000 Batch Loss: 1.294714 Tokens per Sec: 19063, Lr: 0.000050\n", "2020-01-18 01:51:31,308 Example #0\n", "2020-01-18 01:51:31,309 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:51:31,309 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:51:31,309 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:51:31,309 Example #1\n", "2020-01-18 01:51:31,310 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:51:31,310 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:51:31,310 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:51:31,310 Example #2\n", "2020-01-18 01:51:31,311 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:51:31,311 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:51:31,311 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 01:51:31,311 Example #3\n", "2020-01-18 01:51:31,311 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:51:31,311 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:51:31,312 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:51:31,312 Validation result (greedy) at epoch 37, step 235000: bleu: 32.21, loss: 32258.8555, ppl: 3.4706, duration: 29.1790s\n", "2020-01-18 01:51:42,846 Epoch 37 Step: 235100 Batch Loss: 1.110759 Tokens per Sec: 19737, Lr: 0.000050\n", "2020-01-18 01:51:54,223 Epoch 37 Step: 235200 Batch Loss: 1.205421 Tokens per Sec: 19337, Lr: 0.000050\n", "2020-01-18 01:52:05,596 Epoch 37 Step: 235300 Batch Loss: 1.328942 Tokens per Sec: 19413, Lr: 0.000050\n", "2020-01-18 01:52:17,264 Epoch 37 Step: 235400 Batch Loss: 1.472899 Tokens per Sec: 19566, Lr: 0.000050\n", "2020-01-18 01:52:28,631 Epoch 37 Step: 235500 Batch Loss: 1.295959 Tokens per Sec: 18991, Lr: 0.000050\n", "2020-01-18 01:52:31,930 Epoch 37: total training loss 2962.69\n", "2020-01-18 01:52:31,930 EPOCH 38\n", "2020-01-18 01:52:40,317 Epoch 38 Step: 235600 Batch Loss: 0.735403 Tokens per Sec: 18884, Lr: 0.000050\n", "2020-01-18 01:52:51,851 Epoch 38 Step: 235700 Batch Loss: 1.252737 Tokens per Sec: 19548, Lr: 0.000050\n", "2020-01-18 01:53:03,331 Epoch 38 Step: 235800 Batch Loss: 1.237490 Tokens per Sec: 19817, Lr: 0.000050\n", "2020-01-18 01:53:14,789 Epoch 38 Step: 235900 Batch Loss: 1.247592 Tokens per Sec: 18949, Lr: 0.000050\n", "2020-01-18 01:53:26,354 Epoch 38 Step: 236000 Batch Loss: 1.054767 Tokens per Sec: 19541, Lr: 0.000050\n", "2020-01-18 01:53:55,437 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:53:55,437 Saving new checkpoint.\n", "2020-01-18 01:53:56,657 Example #0\n", "2020-01-18 01:53:56,657 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:53:56,658 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:53:56,658 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:53:56,658 Example #1\n", "2020-01-18 01:53:56,658 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:53:56,658 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:53:56,659 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 01:53:56,659 Example #2\n", "2020-01-18 01:53:56,659 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:53:56,659 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:53:56,660 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jọ rọ kẹ erẹwho kpobi ; wha wo oghale .\n", "2020-01-18 01:53:56,660 Example #3\n", "2020-01-18 01:53:56,660 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:53:56,660 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:53:56,660 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:53:56,661 Validation result (greedy) at epoch 38, step 236000: bleu: 32.25, loss: 32247.7676, ppl: 3.4691, duration: 30.3062s\n", "2020-01-18 01:54:08,292 Epoch 38 Step: 236100 Batch Loss: 1.164144 Tokens per Sec: 19053, Lr: 0.000050\n", "2020-01-18 01:54:19,925 Epoch 38 Step: 236200 Batch Loss: 1.116882 Tokens per Sec: 19595, Lr: 0.000050\n", "2020-01-18 01:54:31,295 Epoch 38 Step: 236300 Batch Loss: 0.975022 Tokens per Sec: 19151, Lr: 0.000050\n", "2020-01-18 01:54:42,810 Epoch 38 Step: 236400 Batch Loss: 1.276349 Tokens per Sec: 19444, Lr: 0.000050\n", "2020-01-18 01:54:54,244 Epoch 38 Step: 236500 Batch Loss: 1.285388 Tokens per Sec: 19140, Lr: 0.000050\n", "2020-01-18 01:55:05,789 Epoch 38 Step: 236600 Batch Loss: 1.109603 Tokens per Sec: 19456, Lr: 0.000050\n", "2020-01-18 01:55:17,320 Epoch 38 Step: 236700 Batch Loss: 1.063654 Tokens per Sec: 18994, Lr: 0.000050\n", "2020-01-18 01:55:28,835 Epoch 38 Step: 236800 Batch Loss: 1.013579 Tokens per Sec: 19705, Lr: 0.000050\n", "2020-01-18 01:55:40,227 Epoch 38 Step: 236900 Batch Loss: 1.146821 Tokens per Sec: 19420, Lr: 0.000050\n", "2020-01-18 01:55:51,820 Epoch 38 Step: 237000 Batch Loss: 1.121358 Tokens per Sec: 19223, Lr: 0.000050\n", "2020-01-18 01:56:21,013 Hooray! New best validation result [ppl]!\n", "2020-01-18 01:56:21,013 Saving new checkpoint.\n", "2020-01-18 01:56:22,246 Example #0\n", "2020-01-18 01:56:22,247 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:56:22,247 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:56:22,248 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:56:22,248 Example #1\n", "2020-01-18 01:56:22,248 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:56:22,248 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:56:22,249 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:56:22,249 Example #2\n", "2020-01-18 01:56:22,249 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:56:22,250 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:56:22,250 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba abọ urere ọrọ ole nana nọ o wo erru gbe omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; wha jọ ahwo kpobi a wo oghale .\n", "2020-01-18 01:56:22,250 Example #3\n", "2020-01-18 01:56:22,250 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:56:22,250 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:56:22,251 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:56:22,251 Validation result (greedy) at epoch 38, step 237000: bleu: 32.26, loss: 32213.2402, ppl: 3.4645, duration: 30.4307s\n", "2020-01-18 01:56:33,733 Epoch 38 Step: 237100 Batch Loss: 0.874742 Tokens per Sec: 19266, Lr: 0.000050\n", "2020-01-18 01:56:45,373 Epoch 38 Step: 237200 Batch Loss: 1.173848 Tokens per Sec: 18570, Lr: 0.000050\n", "2020-01-18 01:56:56,790 Epoch 38 Step: 237300 Batch Loss: 1.295986 Tokens per Sec: 19252, Lr: 0.000050\n", "2020-01-18 01:57:08,337 Epoch 38 Step: 237400 Batch Loss: 1.244942 Tokens per Sec: 19415, Lr: 0.000050\n", "2020-01-18 01:57:19,814 Epoch 38 Step: 237500 Batch Loss: 1.181847 Tokens per Sec: 19503, Lr: 0.000050\n", "2020-01-18 01:57:31,245 Epoch 38 Step: 237600 Batch Loss: 1.166375 Tokens per Sec: 19123, Lr: 0.000050\n", "2020-01-18 01:57:42,748 Epoch 38 Step: 237700 Batch Loss: 0.980125 Tokens per Sec: 18872, Lr: 0.000050\n", "2020-01-18 01:57:54,259 Epoch 38 Step: 237800 Batch Loss: 1.177585 Tokens per Sec: 20025, Lr: 0.000050\n", "2020-01-18 01:58:05,594 Epoch 38 Step: 237900 Batch Loss: 0.973674 Tokens per Sec: 19216, Lr: 0.000050\n", "2020-01-18 01:58:17,069 Epoch 38 Step: 238000 Batch Loss: 1.118855 Tokens per Sec: 19081, Lr: 0.000050\n", "2020-01-18 01:58:46,179 Example #0\n", "2020-01-18 01:58:46,180 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 01:58:46,180 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 01:58:46,180 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 01:58:46,180 Example #1\n", "2020-01-18 01:58:46,180 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 01:58:46,181 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 01:58:46,181 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 01:58:46,181 Example #2\n", "2020-01-18 01:58:46,181 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 01:58:46,181 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 01:58:46,182 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , jọ odẹ riẹ o rro , re o jọ ole riẹ ole evawere kẹ ae ; jọ o jọ kẹ erẹwho kpobi ; re o wo oghale .\n", "2020-01-18 01:58:46,182 Example #3\n", "2020-01-18 01:58:46,182 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 01:58:46,182 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 01:58:46,182 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 01:58:46,182 Validation result (greedy) at epoch 38, step 238000: bleu: 32.19, loss: 32247.8730, ppl: 3.4691, duration: 29.1126s\n", "2020-01-18 01:58:53,129 Epoch 38: total training loss 2960.71\n", "2020-01-18 01:58:53,130 EPOCH 39\n", "2020-01-18 01:58:57,916 Epoch 39 Step: 238100 Batch Loss: 0.897929 Tokens per Sec: 18267, Lr: 0.000050\n", "2020-01-18 01:59:09,325 Epoch 39 Step: 238200 Batch Loss: 0.992671 Tokens per Sec: 18740, Lr: 0.000050\n", "2020-01-18 01:59:20,693 Epoch 39 Step: 238300 Batch Loss: 1.042414 Tokens per Sec: 19084, Lr: 0.000050\n", "2020-01-18 01:59:32,195 Epoch 39 Step: 238400 Batch Loss: 1.008591 Tokens per Sec: 19291, Lr: 0.000050\n", "2020-01-18 01:59:43,743 Epoch 39 Step: 238500 Batch Loss: 1.109202 Tokens per Sec: 19604, Lr: 0.000050\n", "2020-01-18 01:59:55,294 Epoch 39 Step: 238600 Batch Loss: 1.190957 Tokens per Sec: 19870, Lr: 0.000050\n", "2020-01-18 02:00:06,646 Epoch 39 Step: 238700 Batch Loss: 1.284367 Tokens per Sec: 19103, Lr: 0.000050\n", "2020-01-18 02:00:18,115 Epoch 39 Step: 238800 Batch Loss: 1.127748 Tokens per Sec: 19252, Lr: 0.000050\n", "2020-01-18 02:00:29,720 Epoch 39 Step: 238900 Batch Loss: 1.413830 Tokens per Sec: 19308, Lr: 0.000050\n", "2020-01-18 02:00:41,291 Epoch 39 Step: 239000 Batch Loss: 1.016789 Tokens per Sec: 19424, Lr: 0.000050\n", "2020-01-18 02:01:10,453 Example #0\n", "2020-01-18 02:01:10,454 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:01:10,454 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:01:10,454 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:01:10,455 Example #1\n", "2020-01-18 02:01:10,455 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:01:10,455 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:01:10,455 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 02:01:10,455 Example #2\n", "2020-01-18 02:01:10,456 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:01:10,456 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:01:10,456 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹjiroro urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 02:01:10,456 Example #3\n", "2020-01-18 02:01:10,457 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:01:10,457 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:01:10,457 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:01:10,457 Validation result (greedy) at epoch 39, step 239000: bleu: 32.10, loss: 32267.7227, ppl: 3.4717, duration: 29.1657s\n", "2020-01-18 02:01:21,870 Epoch 39 Step: 239100 Batch Loss: 1.218065 Tokens per Sec: 19341, Lr: 0.000050\n", "2020-01-18 02:01:33,471 Epoch 39 Step: 239200 Batch Loss: 1.201277 Tokens per Sec: 19458, Lr: 0.000050\n", "2020-01-18 02:01:45,012 Epoch 39 Step: 239300 Batch Loss: 1.082435 Tokens per Sec: 19041, Lr: 0.000050\n", "2020-01-18 02:01:56,384 Epoch 39 Step: 239400 Batch Loss: 1.261711 Tokens per Sec: 19538, Lr: 0.000050\n", "2020-01-18 02:02:07,967 Epoch 39 Step: 239500 Batch Loss: 1.161107 Tokens per Sec: 19363, Lr: 0.000050\n", "2020-01-18 02:02:19,505 Epoch 39 Step: 239600 Batch Loss: 1.004398 Tokens per Sec: 19685, Lr: 0.000050\n", "2020-01-18 02:02:30,994 Epoch 39 Step: 239700 Batch Loss: 1.120540 Tokens per Sec: 19228, Lr: 0.000050\n", "2020-01-18 02:02:42,597 Epoch 39 Step: 239800 Batch Loss: 1.316023 Tokens per Sec: 19490, Lr: 0.000050\n", "2020-01-18 02:02:54,012 Epoch 39 Step: 239900 Batch Loss: 1.014099 Tokens per Sec: 19450, Lr: 0.000050\n", "2020-01-18 02:03:05,439 Epoch 39 Step: 240000 Batch Loss: 1.253639 Tokens per Sec: 19177, Lr: 0.000050\n", "2020-01-18 02:03:34,554 Example #0\n", "2020-01-18 02:03:34,555 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:03:34,555 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:03:34,556 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:03:34,556 Example #1\n", "2020-01-18 02:03:34,556 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:03:34,556 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:03:34,556 \tHypothesis: A te lele Usi Uvie na Ijo .\n", "2020-01-18 02:03:34,556 Example #2\n", "2020-01-18 02:03:34,557 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:03:34,557 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:03:34,557 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 02:03:34,557 Example #3\n", "2020-01-18 02:03:34,558 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:03:34,558 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:03:34,558 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:03:34,558 Validation result (greedy) at epoch 39, step 240000: bleu: 32.11, loss: 32232.0645, ppl: 3.4670, duration: 29.1183s\n", "2020-01-18 02:03:46,002 Epoch 39 Step: 240100 Batch Loss: 1.170485 Tokens per Sec: 19062, Lr: 0.000050\n", "2020-01-18 02:03:57,475 Epoch 39 Step: 240200 Batch Loss: 1.331491 Tokens per Sec: 19817, Lr: 0.000050\n", "2020-01-18 02:04:08,918 Epoch 39 Step: 240300 Batch Loss: 1.121390 Tokens per Sec: 19419, Lr: 0.000050\n", "2020-01-18 02:04:20,268 Epoch 39 Step: 240400 Batch Loss: 1.421413 Tokens per Sec: 19409, Lr: 0.000050\n", "2020-01-18 02:04:31,819 Epoch 39 Step: 240500 Batch Loss: 1.232381 Tokens per Sec: 19320, Lr: 0.000050\n", "2020-01-18 02:04:42,291 Epoch 39: total training loss 2957.49\n", "2020-01-18 02:04:42,292 EPOCH 40\n", "2020-01-18 02:04:43,557 Epoch 40 Step: 240600 Batch Loss: 1.094226 Tokens per Sec: 12602, Lr: 0.000050\n", "2020-01-18 02:04:54,984 Epoch 40 Step: 240700 Batch Loss: 1.148433 Tokens per Sec: 19632, Lr: 0.000050\n", "2020-01-18 02:05:06,507 Epoch 40 Step: 240800 Batch Loss: 1.423242 Tokens per Sec: 19745, Lr: 0.000050\n", "2020-01-18 02:05:18,024 Epoch 40 Step: 240900 Batch Loss: 1.132883 Tokens per Sec: 19269, Lr: 0.000050\n", "2020-01-18 02:05:29,518 Epoch 40 Step: 241000 Batch Loss: 1.204192 Tokens per Sec: 19564, Lr: 0.000050\n", "2020-01-18 02:05:58,646 Example #0\n", "2020-01-18 02:05:58,647 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:05:58,647 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:05:58,647 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:05:58,647 Example #1\n", "2020-01-18 02:05:58,647 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:05:58,647 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:05:58,647 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 02:05:58,648 Example #2\n", "2020-01-18 02:05:58,648 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:05:58,648 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:05:58,648 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba uvou mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ o jọ ahwo kpobi a wo oghale ; wha kẹ ae udhedhẹ .\n", "2020-01-18 02:05:58,648 Example #3\n", "2020-01-18 02:05:58,649 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:05:58,649 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:05:58,649 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 02:05:58,649 Validation result (greedy) at epoch 40, step 241000: bleu: 32.03, loss: 32303.0566, ppl: 3.4765, duration: 29.1302s\n", "2020-01-18 02:06:10,299 Epoch 40 Step: 241100 Batch Loss: 1.308165 Tokens per Sec: 18975, Lr: 0.000050\n", "2020-01-18 02:06:21,671 Epoch 40 Step: 241200 Batch Loss: 0.952999 Tokens per Sec: 19400, Lr: 0.000050\n", "2020-01-18 02:06:33,226 Epoch 40 Step: 241300 Batch Loss: 1.292979 Tokens per Sec: 19314, Lr: 0.000050\n", "2020-01-18 02:06:44,630 Epoch 40 Step: 241400 Batch Loss: 1.222087 Tokens per Sec: 18846, Lr: 0.000050\n", "2020-01-18 02:06:56,141 Epoch 40 Step: 241500 Batch Loss: 1.108967 Tokens per Sec: 19630, Lr: 0.000050\n", "2020-01-18 02:07:07,742 Epoch 40 Step: 241600 Batch Loss: 1.089116 Tokens per Sec: 19342, Lr: 0.000050\n", "2020-01-18 02:07:19,190 Epoch 40 Step: 241700 Batch Loss: 1.220403 Tokens per Sec: 19472, Lr: 0.000050\n", "2020-01-18 02:07:30,655 Epoch 40 Step: 241800 Batch Loss: 1.248234 Tokens per Sec: 19406, Lr: 0.000050\n", "2020-01-18 02:07:42,118 Epoch 40 Step: 241900 Batch Loss: 1.281147 Tokens per Sec: 19479, Lr: 0.000050\n", "2020-01-18 02:07:53,625 Epoch 40 Step: 242000 Batch Loss: 0.959452 Tokens per Sec: 19829, Lr: 0.000050\n", "2020-01-18 02:08:22,816 Example #0\n", "2020-01-18 02:08:22,817 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:08:22,817 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:08:22,817 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:08:22,818 Example #1\n", "2020-01-18 02:08:22,818 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:08:22,818 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:08:22,818 \tHypothesis: Ijo , a te lele Usi Uvie na .\n", "2020-01-18 02:08:22,818 Example #2\n", "2020-01-18 02:08:22,819 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:08:22,819 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:08:22,819 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ o jọ ahwo kpobi a wo oghale .\n", "2020-01-18 02:08:22,819 Example #3\n", "2020-01-18 02:08:22,820 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:08:22,820 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:08:22,820 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:08:22,820 Validation result (greedy) at epoch 40, step 242000: bleu: 32.23, loss: 32235.7031, ppl: 3.4675, duration: 29.1952s\n", "2020-01-18 02:08:34,218 Epoch 40 Step: 242100 Batch Loss: 1.189097 Tokens per Sec: 18660, Lr: 0.000050\n", "2020-01-18 02:08:45,665 Epoch 40 Step: 242200 Batch Loss: 1.197644 Tokens per Sec: 19337, Lr: 0.000050\n", "2020-01-18 02:08:57,123 Epoch 40 Step: 242300 Batch Loss: 1.253239 Tokens per Sec: 19258, Lr: 0.000050\n", "2020-01-18 02:09:08,594 Epoch 40 Step: 242400 Batch Loss: 1.202692 Tokens per Sec: 19396, Lr: 0.000050\n", "2020-01-18 02:09:19,961 Epoch 40 Step: 242500 Batch Loss: 1.097577 Tokens per Sec: 19559, Lr: 0.000050\n", "2020-01-18 02:09:31,464 Epoch 40 Step: 242600 Batch Loss: 1.254805 Tokens per Sec: 19212, Lr: 0.000050\n", "2020-01-18 02:09:42,963 Epoch 40 Step: 242700 Batch Loss: 1.369373 Tokens per Sec: 19509, Lr: 0.000050\n", "2020-01-18 02:09:54,449 Epoch 40 Step: 242800 Batch Loss: 1.080658 Tokens per Sec: 19409, Lr: 0.000050\n", "2020-01-18 02:10:05,911 Epoch 40 Step: 242900 Batch Loss: 1.172418 Tokens per Sec: 19680, Lr: 0.000050\n", "2020-01-18 02:10:17,460 Epoch 40 Step: 243000 Batch Loss: 1.025468 Tokens per Sec: 19264, Lr: 0.000050\n", "2020-01-18 02:10:46,604 Example #0\n", "2020-01-18 02:10:46,605 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:10:46,605 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:10:46,605 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:10:46,605 Example #1\n", "2020-01-18 02:10:46,605 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:10:46,606 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:10:46,606 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:10:46,606 Example #2\n", "2020-01-18 02:10:46,606 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:10:46,606 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:10:46,606 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana , onọ o rẹ kẹ omosasọ gbe omosasọ : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo via eva riẹ , re ọ ghale ae ; jọ ahwo kpobi a wo oghale .\n", "2020-01-18 02:10:46,607 Example #3\n", "2020-01-18 02:10:46,607 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:10:46,607 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:10:46,607 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re ma ru udhedhẹ .\n", "2020-01-18 02:10:46,607 Validation result (greedy) at epoch 40, step 243000: bleu: 32.22, loss: 32241.7090, ppl: 3.4683, duration: 29.1468s\n", "2020-01-18 02:10:58,126 Epoch 40 Step: 243100 Batch Loss: 1.206459 Tokens per Sec: 19805, Lr: 0.000035\n", "2020-01-18 02:11:00,105 Epoch 40: total training loss 2948.82\n", "2020-01-18 02:11:00,105 EPOCH 41\n", "2020-01-18 02:11:09,953 Epoch 41 Step: 243200 Batch Loss: 1.109141 Tokens per Sec: 18668, Lr: 0.000035\n", "2020-01-18 02:11:21,332 Epoch 41 Step: 243300 Batch Loss: 1.292137 Tokens per Sec: 19169, Lr: 0.000035\n", "2020-01-18 02:11:33,027 Epoch 41 Step: 243400 Batch Loss: 1.138479 Tokens per Sec: 19330, Lr: 0.000035\n", "2020-01-18 02:11:44,559 Epoch 41 Step: 243500 Batch Loss: 0.952829 Tokens per Sec: 19551, Lr: 0.000035\n", "2020-01-18 02:11:55,929 Epoch 41 Step: 243600 Batch Loss: 1.025196 Tokens per Sec: 19238, Lr: 0.000035\n", "2020-01-18 02:12:07,486 Epoch 41 Step: 243700 Batch Loss: 1.167634 Tokens per Sec: 19412, Lr: 0.000035\n", "2020-01-18 02:12:18,930 Epoch 41 Step: 243800 Batch Loss: 1.198332 Tokens per Sec: 19147, Lr: 0.000035\n", "2020-01-18 02:12:30,489 Epoch 41 Step: 243900 Batch Loss: 1.254098 Tokens per Sec: 19395, Lr: 0.000035\n", "2020-01-18 02:12:42,010 Epoch 41 Step: 244000 Batch Loss: 0.991856 Tokens per Sec: 19252, Lr: 0.000035\n", "2020-01-18 02:13:11,130 Hooray! New best validation result [ppl]!\n", "2020-01-18 02:13:11,130 Saving new checkpoint.\n", "2020-01-18 02:13:12,306 Example #0\n", "2020-01-18 02:13:12,307 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:13:12,307 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:13:12,307 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:13:12,307 Example #1\n", "2020-01-18 02:13:12,308 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:13:12,308 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:13:12,308 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:13:12,308 Example #2\n", "2020-01-18 02:13:12,309 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:13:12,309 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:13:12,309 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o te lo via eva riẹ , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 02:13:12,309 Example #3\n", "2020-01-18 02:13:12,309 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:13:12,310 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:13:12,310 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:13:12,310 Validation result (greedy) at epoch 41, step 244000: bleu: 32.26, loss: 32189.1309, ppl: 3.4612, duration: 30.2993s\n", "2020-01-18 02:13:23,692 Epoch 41 Step: 244100 Batch Loss: 1.277950 Tokens per Sec: 19384, Lr: 0.000035\n", "2020-01-18 02:13:35,456 Epoch 41 Step: 244200 Batch Loss: 1.362723 Tokens per Sec: 18830, Lr: 0.000035\n", "2020-01-18 02:13:46,930 Epoch 41 Step: 244300 Batch Loss: 1.146554 Tokens per Sec: 18948, Lr: 0.000035\n", "2020-01-18 02:13:58,416 Epoch 41 Step: 244400 Batch Loss: 1.172782 Tokens per Sec: 19805, Lr: 0.000035\n", "2020-01-18 02:14:09,879 Epoch 41 Step: 244500 Batch Loss: 1.185487 Tokens per Sec: 19432, Lr: 0.000035\n", "2020-01-18 02:14:21,361 Epoch 41 Step: 244600 Batch Loss: 1.278113 Tokens per Sec: 19733, Lr: 0.000035\n", "2020-01-18 02:14:32,883 Epoch 41 Step: 244700 Batch Loss: 1.104972 Tokens per Sec: 19548, Lr: 0.000035\n", "2020-01-18 02:14:44,320 Epoch 41 Step: 244800 Batch Loss: 1.371010 Tokens per Sec: 18918, Lr: 0.000035\n", "2020-01-18 02:14:55,708 Epoch 41 Step: 244900 Batch Loss: 0.939816 Tokens per Sec: 19256, Lr: 0.000035\n", "2020-01-18 02:15:07,179 Epoch 41 Step: 245000 Batch Loss: 1.373656 Tokens per Sec: 19116, Lr: 0.000035\n", "2020-01-18 02:15:36,279 Example #0\n", "2020-01-18 02:15:36,280 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:15:36,280 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:15:36,280 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:15:36,280 Example #1\n", "2020-01-18 02:15:36,281 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:15:36,281 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:15:36,281 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:15:36,281 Example #2\n", "2020-01-18 02:15:36,281 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:15:36,282 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:15:36,282 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹjiroro mai rite abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale ; wha kẹ odẹ riẹ ; wha jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 02:15:36,282 Example #3\n", "2020-01-18 02:15:36,282 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:15:36,282 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:15:36,282 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:15:36,282 Validation result (greedy) at epoch 41, step 245000: bleu: 32.26, loss: 32210.9512, ppl: 3.4641, duration: 29.1031s\n", "2020-01-18 02:15:47,780 Epoch 41 Step: 245100 Batch Loss: 1.078676 Tokens per Sec: 19359, Lr: 0.000035\n", "2020-01-18 02:15:59,271 Epoch 41 Step: 245200 Batch Loss: 1.384629 Tokens per Sec: 19808, Lr: 0.000035\n", "2020-01-18 02:16:10,723 Epoch 41 Step: 245300 Batch Loss: 1.196513 Tokens per Sec: 19202, Lr: 0.000035\n", "2020-01-18 02:16:22,155 Epoch 41 Step: 245400 Batch Loss: 0.897974 Tokens per Sec: 19285, Lr: 0.000035\n", "2020-01-18 02:16:33,678 Epoch 41 Step: 245500 Batch Loss: 1.247525 Tokens per Sec: 19316, Lr: 0.000035\n", "2020-01-18 02:16:45,295 Epoch 41 Step: 245600 Batch Loss: 1.216055 Tokens per Sec: 19445, Lr: 0.000035\n", "2020-01-18 02:16:50,638 Epoch 41: total training loss 2945.15\n", "2020-01-18 02:16:50,639 EPOCH 42\n", "2020-01-18 02:16:57,038 Epoch 42 Step: 245700 Batch Loss: 0.846151 Tokens per Sec: 18641, Lr: 0.000035\n", "2020-01-18 02:17:08,503 Epoch 42 Step: 245800 Batch Loss: 1.377835 Tokens per Sec: 19296, Lr: 0.000035\n", "2020-01-18 02:17:19,951 Epoch 42 Step: 245900 Batch Loss: 1.276254 Tokens per Sec: 19719, Lr: 0.000035\n", "2020-01-18 02:17:31,368 Epoch 42 Step: 246000 Batch Loss: 1.175824 Tokens per Sec: 19262, Lr: 0.000035\n", "2020-01-18 02:18:00,452 Hooray! New best validation result [ppl]!\n", "2020-01-18 02:18:00,452 Saving new checkpoint.\n", "2020-01-18 02:18:01,664 Example #0\n", "2020-01-18 02:18:01,665 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:18:01,665 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:18:01,665 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:18:01,665 Example #1\n", "2020-01-18 02:18:01,665 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:18:01,666 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:18:01,666 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:18:01,666 Example #2\n", "2020-01-18 02:18:01,666 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:18:01,666 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:18:01,667 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba eme urere erọ ole owoma nana , onọ o te kẹ omosasọ gbe omosasọ : “ Jọ odẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo odẹ riẹ , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale ; wha wo oghale .\n", "2020-01-18 02:18:01,667 Example #3\n", "2020-01-18 02:18:01,667 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:18:01,667 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:18:01,668 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:18:01,668 Validation result (greedy) at epoch 42, step 246000: bleu: 32.17, loss: 32175.2422, ppl: 3.4594, duration: 30.2997s\n", "2020-01-18 02:18:13,558 Epoch 42 Step: 246100 Batch Loss: 1.251633 Tokens per Sec: 18996, Lr: 0.000035\n", "2020-01-18 02:18:25,116 Epoch 42 Step: 246200 Batch Loss: 1.176641 Tokens per Sec: 19459, Lr: 0.000035\n", "2020-01-18 02:18:36,627 Epoch 42 Step: 246300 Batch Loss: 1.284332 Tokens per Sec: 19921, Lr: 0.000035\n", "2020-01-18 02:18:48,115 Epoch 42 Step: 246400 Batch Loss: 1.159731 Tokens per Sec: 19493, Lr: 0.000035\n", "2020-01-18 02:18:59,620 Epoch 42 Step: 246500 Batch Loss: 1.099183 Tokens per Sec: 19408, Lr: 0.000035\n", "2020-01-18 02:19:11,144 Epoch 42 Step: 246600 Batch Loss: 1.328899 Tokens per Sec: 19014, Lr: 0.000035\n", "2020-01-18 02:19:22,561 Epoch 42 Step: 246700 Batch Loss: 1.299481 Tokens per Sec: 19541, Lr: 0.000035\n", "2020-01-18 02:19:34,039 Epoch 42 Step: 246800 Batch Loss: 1.004118 Tokens per Sec: 19316, Lr: 0.000035\n", "2020-01-18 02:19:45,631 Epoch 42 Step: 246900 Batch Loss: 1.199076 Tokens per Sec: 19951, Lr: 0.000035\n", "2020-01-18 02:19:57,041 Epoch 42 Step: 247000 Batch Loss: 1.383438 Tokens per Sec: 19382, Lr: 0.000035\n", "2020-01-18 02:20:26,139 Example #0\n", "2020-01-18 02:20:26,139 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:20:26,139 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:20:26,140 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:20:26,140 Example #1\n", "2020-01-18 02:20:26,140 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:20:26,140 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:20:26,140 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:20:26,140 Example #2\n", "2020-01-18 02:20:26,141 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:20:26,141 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:20:26,141 \tHypothesis: Koyehọ , mai kpobi ma te rọ oghọghọ fiba ẹjiroro urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oruaro , re ọ ghale ae ; wha jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 02:20:26,141 Example #3\n", "2020-01-18 02:20:26,142 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:20:26,142 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:20:26,142 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:20:26,142 Validation result (greedy) at epoch 42, step 247000: bleu: 32.35, loss: 32237.3379, ppl: 3.4677, duration: 29.1008s\n", "2020-01-18 02:20:37,642 Epoch 42 Step: 247100 Batch Loss: 1.015739 Tokens per Sec: 19427, Lr: 0.000035\n", "2020-01-18 02:20:49,138 Epoch 42 Step: 247200 Batch Loss: 1.034066 Tokens per Sec: 19603, Lr: 0.000035\n", "2020-01-18 02:21:00,528 Epoch 42 Step: 247300 Batch Loss: 1.239831 Tokens per Sec: 19402, Lr: 0.000035\n", "2020-01-18 02:21:12,037 Epoch 42 Step: 247400 Batch Loss: 1.291089 Tokens per Sec: 19350, Lr: 0.000035\n", "2020-01-18 02:21:23,355 Epoch 42 Step: 247500 Batch Loss: 0.919419 Tokens per Sec: 19165, Lr: 0.000035\n", "2020-01-18 02:21:34,825 Epoch 42 Step: 247600 Batch Loss: 1.265297 Tokens per Sec: 19283, Lr: 0.000035\n", "2020-01-18 02:21:46,337 Epoch 42 Step: 247700 Batch Loss: 1.257320 Tokens per Sec: 18661, Lr: 0.000035\n", "2020-01-18 02:21:57,715 Epoch 42 Step: 247800 Batch Loss: 1.239878 Tokens per Sec: 19455, Lr: 0.000035\n", "2020-01-18 02:22:09,305 Epoch 42 Step: 247900 Batch Loss: 1.032873 Tokens per Sec: 19357, Lr: 0.000035\n", "2020-01-18 02:22:20,562 Epoch 42 Step: 248000 Batch Loss: 1.254526 Tokens per Sec: 19256, Lr: 0.000035\n", "2020-01-18 02:22:49,635 Example #0\n", "2020-01-18 02:22:49,636 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:22:49,636 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:22:49,637 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:22:49,637 Example #1\n", "2020-01-18 02:22:49,637 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:22:49,637 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:22:49,637 \tHypothesis: A te je lele Usi Uvie Na No .\n", "2020-01-18 02:22:49,638 Example #2\n", "2020-01-18 02:22:49,638 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:22:49,638 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:22:49,638 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oruaro , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale ; wha wo oghale .\n", "2020-01-18 02:22:49,638 Example #3\n", "2020-01-18 02:22:49,639 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:22:49,639 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:22:49,639 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:22:49,639 Validation result (greedy) at epoch 42, step 248000: bleu: 32.18, loss: 32201.8086, ppl: 3.4629, duration: 29.0771s\n", "2020-01-18 02:23:01,169 Epoch 42 Step: 248100 Batch Loss: 1.083421 Tokens per Sec: 19614, Lr: 0.000035\n", "2020-01-18 02:23:09,549 Epoch 42: total training loss 2936.00\n", "2020-01-18 02:23:09,549 EPOCH 43\n", "2020-01-18 02:23:13,003 Epoch 43 Step: 248200 Batch Loss: 1.277343 Tokens per Sec: 18672, Lr: 0.000035\n", "2020-01-18 02:23:24,419 Epoch 43 Step: 248300 Batch Loss: 0.841956 Tokens per Sec: 19513, Lr: 0.000035\n", "2020-01-18 02:23:35,844 Epoch 43 Step: 248400 Batch Loss: 1.355963 Tokens per Sec: 19323, Lr: 0.000035\n", "2020-01-18 02:23:47,344 Epoch 43 Step: 248500 Batch Loss: 1.130870 Tokens per Sec: 19350, Lr: 0.000035\n", "2020-01-18 02:23:58,668 Epoch 43 Step: 248600 Batch Loss: 0.983818 Tokens per Sec: 19499, Lr: 0.000035\n", "2020-01-18 02:24:10,229 Epoch 43 Step: 248700 Batch Loss: 0.983561 Tokens per Sec: 19331, Lr: 0.000035\n", "2020-01-18 02:24:21,562 Epoch 43 Step: 248800 Batch Loss: 1.165510 Tokens per Sec: 19245, Lr: 0.000035\n", "2020-01-18 02:24:33,072 Epoch 43 Step: 248900 Batch Loss: 1.257036 Tokens per Sec: 19199, Lr: 0.000035\n", "2020-01-18 02:24:44,615 Epoch 43 Step: 249000 Batch Loss: 1.270260 Tokens per Sec: 19808, Lr: 0.000035\n", "2020-01-18 02:25:13,747 Example #0\n", "2020-01-18 02:25:13,748 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:25:13,748 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:25:13,748 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:25:13,748 Example #1\n", "2020-01-18 02:25:13,749 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:25:13,749 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:25:13,749 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:25:13,749 Example #2\n", "2020-01-18 02:25:13,749 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:25:13,749 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:25:13,750 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo , re o wo oruaro , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 02:25:13,750 Example #3\n", "2020-01-18 02:25:13,750 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:25:13,750 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:25:13,750 \tHypothesis: Ghele na , eme unu - uwou e rẹ jọ gaga re a ru udhedhẹ .\n", "2020-01-18 02:25:13,750 Validation result (greedy) at epoch 43, step 249000: bleu: 32.27, loss: 32189.3125, ppl: 3.4613, duration: 29.1348s\n", "2020-01-18 02:25:25,246 Epoch 43 Step: 249100 Batch Loss: 1.055531 Tokens per Sec: 19467, Lr: 0.000035\n", "2020-01-18 02:25:36,597 Epoch 43 Step: 249200 Batch Loss: 1.112487 Tokens per Sec: 19286, Lr: 0.000035\n", "2020-01-18 02:25:48,033 Epoch 43 Step: 249300 Batch Loss: 1.161858 Tokens per Sec: 19128, Lr: 0.000035\n", "2020-01-18 02:25:59,473 Epoch 43 Step: 249400 Batch Loss: 1.159937 Tokens per Sec: 19532, Lr: 0.000035\n", "2020-01-18 02:26:11,019 Epoch 43 Step: 249500 Batch Loss: 0.999176 Tokens per Sec: 19760, Lr: 0.000035\n", "2020-01-18 02:26:22,321 Epoch 43 Step: 249600 Batch Loss: 1.035299 Tokens per Sec: 19273, Lr: 0.000035\n", "2020-01-18 02:26:33,789 Epoch 43 Step: 249700 Batch Loss: 1.262206 Tokens per Sec: 19538, Lr: 0.000035\n", "2020-01-18 02:26:45,356 Epoch 43 Step: 249800 Batch Loss: 1.293918 Tokens per Sec: 19735, Lr: 0.000035\n", "2020-01-18 02:26:56,800 Epoch 43 Step: 249900 Batch Loss: 1.217475 Tokens per Sec: 19096, Lr: 0.000035\n", "2020-01-18 02:27:08,408 Epoch 43 Step: 250000 Batch Loss: 1.281499 Tokens per Sec: 19322, Lr: 0.000035\n", "2020-01-18 02:27:37,461 Hooray! New best validation result [ppl]!\n", "2020-01-18 02:27:37,461 Saving new checkpoint.\n", "2020-01-18 02:27:38,680 Example #0\n", "2020-01-18 02:27:38,681 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:27:38,681 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:27:38,681 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:27:38,681 Example #1\n", "2020-01-18 02:27:38,682 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:27:38,682 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:27:38,682 \tHypothesis: A te je lele Usi Uvie Na No .\n", "2020-01-18 02:27:38,682 Example #2\n", "2020-01-18 02:27:38,683 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:27:38,683 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:27:38,683 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba ẹjiroro urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re ọ ghale ae ; jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 02:27:38,683 Example #3\n", "2020-01-18 02:27:38,684 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:27:38,684 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:27:38,684 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:27:38,684 Validation result (greedy) at epoch 43, step 250000: bleu: 32.01, loss: 32149.1309, ppl: 3.4559, duration: 30.2757s\n", "2020-01-18 02:27:50,705 Epoch 43 Step: 250100 Batch Loss: 0.973700 Tokens per Sec: 18597, Lr: 0.000035\n", "2020-01-18 02:28:02,226 Epoch 43 Step: 250200 Batch Loss: 1.243881 Tokens per Sec: 19414, Lr: 0.000035\n", "2020-01-18 02:28:13,688 Epoch 43 Step: 250300 Batch Loss: 1.165339 Tokens per Sec: 18970, Lr: 0.000035\n", "2020-01-18 02:28:25,170 Epoch 43 Step: 250400 Batch Loss: 0.997764 Tokens per Sec: 19120, Lr: 0.000035\n", "2020-01-18 02:28:36,548 Epoch 43 Step: 250500 Batch Loss: 1.233254 Tokens per Sec: 19022, Lr: 0.000035\n", "2020-01-18 02:28:48,110 Epoch 43 Step: 250600 Batch Loss: 1.222584 Tokens per Sec: 19740, Lr: 0.000035\n", "2020-01-18 02:28:59,563 Epoch 43 Step: 250700 Batch Loss: 1.007386 Tokens per Sec: 19636, Lr: 0.000035\n", "2020-01-18 02:28:59,570 Epoch 43: total training loss 2938.04\n", "2020-01-18 02:28:59,570 EPOCH 44\n", "2020-01-18 02:29:11,254 Epoch 44 Step: 250800 Batch Loss: 1.185394 Tokens per Sec: 18896, Lr: 0.000035\n", "2020-01-18 02:29:22,642 Epoch 44 Step: 250900 Batch Loss: 1.217009 Tokens per Sec: 19382, Lr: 0.000035\n", "2020-01-18 02:29:34,106 Epoch 44 Step: 251000 Batch Loss: 1.257790 Tokens per Sec: 19607, Lr: 0.000035\n", "2020-01-18 02:30:03,161 Example #0\n", "2020-01-18 02:30:03,161 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:30:03,161 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:30:03,162 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:30:03,162 Example #1\n", "2020-01-18 02:30:03,162 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:30:03,162 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:30:03,162 \tHypothesis: A te je ru onana avọ Usi Uvie Na .\n", "2020-01-18 02:30:03,162 Example #2\n", "2020-01-18 02:30:03,163 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:30:03,163 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:30:03,163 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oghale , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 02:30:03,163 Example #3\n", "2020-01-18 02:30:03,164 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:30:03,164 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:30:03,164 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:30:03,164 Validation result (greedy) at epoch 44, step 251000: bleu: 32.08, loss: 32180.5977, ppl: 3.4601, duration: 29.0579s\n", "2020-01-18 02:30:14,723 Epoch 44 Step: 251100 Batch Loss: 1.243477 Tokens per Sec: 19465, Lr: 0.000035\n", "2020-01-18 02:30:26,150 Epoch 44 Step: 251200 Batch Loss: 0.974293 Tokens per Sec: 19517, Lr: 0.000035\n", "2020-01-18 02:30:37,632 Epoch 44 Step: 251300 Batch Loss: 1.232686 Tokens per Sec: 19692, Lr: 0.000035\n", "2020-01-18 02:30:49,134 Epoch 44 Step: 251400 Batch Loss: 0.961486 Tokens per Sec: 19644, Lr: 0.000035\n", "2020-01-18 02:31:00,639 Epoch 44 Step: 251500 Batch Loss: 1.331664 Tokens per Sec: 19138, Lr: 0.000035\n", "2020-01-18 02:31:12,220 Epoch 44 Step: 251600 Batch Loss: 0.958449 Tokens per Sec: 19246, Lr: 0.000035\n", "2020-01-18 02:31:23,665 Epoch 44 Step: 251700 Batch Loss: 1.227149 Tokens per Sec: 19424, Lr: 0.000035\n", "2020-01-18 02:31:35,100 Epoch 44 Step: 251800 Batch Loss: 0.888747 Tokens per Sec: 19210, Lr: 0.000035\n", "2020-01-18 02:31:46,637 Epoch 44 Step: 251900 Batch Loss: 1.011638 Tokens per Sec: 19465, Lr: 0.000035\n", "2020-01-18 02:31:58,217 Epoch 44 Step: 252000 Batch Loss: 1.169026 Tokens per Sec: 19691, Lr: 0.000035\n", "2020-01-18 02:32:27,387 Example #0\n", "2020-01-18 02:32:27,388 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:32:27,388 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:32:27,389 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:32:27,389 Example #1\n", "2020-01-18 02:32:27,389 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:32:27,389 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:32:27,390 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:32:27,390 Example #2\n", "2020-01-18 02:32:27,390 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:32:27,390 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:32:27,391 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ erẹwho kpobi i wo oghale .\n", "2020-01-18 02:32:27,391 Example #3\n", "2020-01-18 02:32:27,391 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:32:27,391 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:32:27,391 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:32:27,392 Validation result (greedy) at epoch 44, step 252000: bleu: 32.36, loss: 32209.5000, ppl: 3.4640, duration: 29.1741s\n", "2020-01-18 02:32:38,802 Epoch 44 Step: 252100 Batch Loss: 1.254270 Tokens per Sec: 19111, Lr: 0.000035\n", "2020-01-18 02:32:50,138 Epoch 44 Step: 252200 Batch Loss: 1.125267 Tokens per Sec: 19389, Lr: 0.000035\n", "2020-01-18 02:33:01,608 Epoch 44 Step: 252300 Batch Loss: 1.029905 Tokens per Sec: 19524, Lr: 0.000035\n", "2020-01-18 02:33:13,000 Epoch 44 Step: 252400 Batch Loss: 0.892837 Tokens per Sec: 19098, Lr: 0.000035\n", "2020-01-18 02:33:24,462 Epoch 44 Step: 252500 Batch Loss: 1.168761 Tokens per Sec: 19512, Lr: 0.000035\n", "2020-01-18 02:33:35,909 Epoch 44 Step: 252600 Batch Loss: 1.150535 Tokens per Sec: 19365, Lr: 0.000035\n", "2020-01-18 02:33:47,469 Epoch 44 Step: 252700 Batch Loss: 1.117001 Tokens per Sec: 19253, Lr: 0.000035\n", "2020-01-18 02:33:58,887 Epoch 44 Step: 252800 Batch Loss: 1.093339 Tokens per Sec: 19824, Lr: 0.000035\n", "2020-01-18 02:34:10,444 Epoch 44 Step: 252900 Batch Loss: 1.044813 Tokens per Sec: 19163, Lr: 0.000035\n", "2020-01-18 02:34:21,828 Epoch 44 Step: 253000 Batch Loss: 0.952957 Tokens per Sec: 19833, Lr: 0.000035\n", "2020-01-18 02:34:50,932 Example #0\n", "2020-01-18 02:34:50,933 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:34:50,933 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:34:50,933 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:34:50,934 Example #1\n", "2020-01-18 02:34:50,934 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:34:50,934 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:34:50,934 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:34:50,934 Example #2\n", "2020-01-18 02:34:50,935 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:34:50,935 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:34:50,935 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana nọ o rẹ kẹ omosasọ na : “ Jọ odẹ [ Ovie Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o wo oruaro , re ọ ghale ae ; jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 02:34:50,935 Example #3\n", "2020-01-18 02:34:50,936 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:34:50,936 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:34:50,936 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:34:50,936 Validation result (greedy) at epoch 44, step 253000: bleu: 32.48, loss: 32154.1035, ppl: 3.4566, duration: 29.1079s\n", "2020-01-18 02:35:02,410 Epoch 44 Step: 253100 Batch Loss: 1.214936 Tokens per Sec: 19462, Lr: 0.000035\n", "2020-01-18 02:35:13,970 Epoch 44 Step: 253200 Batch Loss: 1.321404 Tokens per Sec: 19139, Lr: 0.000035\n", "2020-01-18 02:35:16,626 Epoch 44: total training loss 2928.10\n", "2020-01-18 02:35:16,627 EPOCH 45\n", "2020-01-18 02:35:25,720 Epoch 45 Step: 253300 Batch Loss: 1.352574 Tokens per Sec: 19106, Lr: 0.000035\n", "2020-01-18 02:35:37,161 Epoch 45 Step: 253400 Batch Loss: 0.942734 Tokens per Sec: 19236, Lr: 0.000035\n", "2020-01-18 02:35:48,609 Epoch 45 Step: 253500 Batch Loss: 1.129594 Tokens per Sec: 19441, Lr: 0.000035\n", "2020-01-18 02:36:00,056 Epoch 45 Step: 253600 Batch Loss: 1.345874 Tokens per Sec: 19471, Lr: 0.000035\n", "2020-01-18 02:36:11,600 Epoch 45 Step: 253700 Batch Loss: 1.223620 Tokens per Sec: 19214, Lr: 0.000035\n", "2020-01-18 02:36:23,039 Epoch 45 Step: 253800 Batch Loss: 0.840550 Tokens per Sec: 19372, Lr: 0.000035\n", "2020-01-18 02:36:34,479 Epoch 45 Step: 253900 Batch Loss: 1.117557 Tokens per Sec: 19305, Lr: 0.000035\n", "2020-01-18 02:36:45,929 Epoch 45 Step: 254000 Batch Loss: 1.031609 Tokens per Sec: 19364, Lr: 0.000035\n", "2020-01-18 02:37:15,183 Example #0\n", "2020-01-18 02:37:15,184 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:37:15,184 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:37:15,184 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:37:15,184 Example #1\n", "2020-01-18 02:37:15,185 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:37:15,185 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:37:15,185 \tHypothesis: A te lele Usi Uvie Na No .\n", "2020-01-18 02:37:15,185 Example #2\n", "2020-01-18 02:37:15,186 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:37:15,186 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:37:15,186 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; wha jọ ahwo Egedhọ kpobi a wo oghale .\n", "2020-01-18 02:37:15,187 Example #3\n", "2020-01-18 02:37:15,187 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:37:15,187 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:37:15,187 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ologbo nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:37:15,188 Validation result (greedy) at epoch 45, step 254000: bleu: 32.27, loss: 32184.1719, ppl: 3.4606, duration: 29.2582s\n", "2020-01-18 02:37:26,705 Epoch 45 Step: 254100 Batch Loss: 1.052296 Tokens per Sec: 19440, Lr: 0.000035\n", "2020-01-18 02:37:38,212 Epoch 45 Step: 254200 Batch Loss: 1.152094 Tokens per Sec: 19774, Lr: 0.000035\n", "2020-01-18 02:37:49,592 Epoch 45 Step: 254300 Batch Loss: 1.232656 Tokens per Sec: 19106, Lr: 0.000035\n", "2020-01-18 02:38:01,070 Epoch 45 Step: 254400 Batch Loss: 1.035370 Tokens per Sec: 19591, Lr: 0.000035\n", "2020-01-18 02:38:12,602 Epoch 45 Step: 254500 Batch Loss: 1.016091 Tokens per Sec: 19411, Lr: 0.000035\n", "2020-01-18 02:38:24,090 Epoch 45 Step: 254600 Batch Loss: 0.992785 Tokens per Sec: 19632, Lr: 0.000035\n", "2020-01-18 02:38:35,506 Epoch 45 Step: 254700 Batch Loss: 1.239884 Tokens per Sec: 19460, Lr: 0.000035\n", "2020-01-18 02:38:47,014 Epoch 45 Step: 254800 Batch Loss: 1.181622 Tokens per Sec: 19542, Lr: 0.000035\n", "2020-01-18 02:38:58,445 Epoch 45 Step: 254900 Batch Loss: 1.003713 Tokens per Sec: 19397, Lr: 0.000035\n", "2020-01-18 02:39:10,010 Epoch 45 Step: 255000 Batch Loss: 1.267494 Tokens per Sec: 19004, Lr: 0.000035\n", "2020-01-18 02:39:39,084 Example #0\n", "2020-01-18 02:39:39,085 \tSource: We can even ask God to ‘ create in us a pure heart . ’\n", "2020-01-18 02:39:39,085 \tReference: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ kẹ omai eva efuafo . ’\n", "2020-01-18 02:39:39,085 \tHypothesis: Ma rẹ sae tubẹ yare Ọghẹnẹ re ọ ‘ ma omai evaọ eva efuafo . ’\n", "2020-01-18 02:39:39,085 Example #1\n", "2020-01-18 02:39:39,086 \tSource: This was followed by Kingdom News No .\n", "2020-01-18 02:39:39,086 \tReference: U no ere no , a te siobọno obe usi ofa , Kingdom News No .\n", "2020-01-18 02:39:39,086 \tHypothesis: A te je lele Usi Uvie Na No .\n", "2020-01-18 02:39:39,086 Example #2\n", "2020-01-18 02:39:39,087 \tSource: In effect , all of us will then joyfully add our voices to the concluding portion of this beautiful and heartwarming song : “ Let his name [ that of the King Jesus Christ ] prove to be to time indefinite ; before the sun let his name have increase , and by means of him let them bless themselves ; let all nations pronounce him happy .\n", "2020-01-18 02:39:39,087 \tReference: Mai kpobi ma te rọ oghọghọ ku irru mai gbe so abọ urere ọrọ ole omosasọ nana : “ Jọ odẹ riẹ [ ọrọ Jesu Kristi Ovie na ] o te jọ bẹdẹ bẹdẹ , re usi riẹ u ti do bẹse nọ akpọ ọ bẹoviẹ !\n", "2020-01-18 02:39:39,087 \tHypothesis: Koyehọ mai kpobi ma te rọ oghọghọ fiba urru mai te abọ urere ọrọ ole owoma nana : “ Jọ odẹ riẹ [ Ovie na Jesu Kristi ] o jọ bẹdẹ bẹdẹ ; re ọre o te lo via , re o te lo via eva riẹ , re ọ ghale ae ; jọ o jọ rọ kẹ erẹwho kpobi ; wha wo oghale .\n", "2020-01-18 02:39:39,087 Example #3\n", "2020-01-18 02:39:39,087 \tSource: Still , words of apology are a strong force toward making peace .\n", "2020-01-18 02:39:39,087 \tReference: Ghele na , eme unu - uwou u re fi obọ họ gaga evaọ eruo udhedhẹ .\n", "2020-01-18 02:39:39,088 \tHypothesis: Ghele na , eme unu - uwou yọ ẹgba ọgaga nọ a re ro ru udhedhẹ .\n", "2020-01-18 02:39:39,088 Validation result (greedy) at epoch 45, step 255000: bleu: 32.04, loss: 32182.6582, ppl: 3.4604, duration: 29.0773s\n", "2020-01-18 02:39:50,622 Epoch 45 Step: 255100 Batch Loss: 1.044141 Tokens per Sec: 19443, Lr: 0.000035\n", "2020-01-18 02:40:01,987 Epoch 45 Step: 255200 Batch Loss: 1.198003 Tokens per Sec: 19303, Lr: 0.000035\n", "2020-01-18 02:40:13,578 Epoch 45 Step: 255300 Batch Loss: 1.094482 Tokens per Sec: 19394, Lr: 0.000035\n", "2020-01-18 02:40:25,022 Epoch 45 Step: 255400 Batch Loss: 1.229972 Tokens per Sec: 19496, Lr: 0.000035\n", "2020-01-18 02:40:36,499 Epoch 45 Step: 255500 Batch Loss: 1.026478 Tokens per Sec: 19334, Lr: 0.000035\n", "2020-01-18 02:40:47,848 Epoch 45 Step: 255600 Batch Loss: 1.316891 Tokens per Sec: 18914, Lr: 0.000035\n", "2020-01-18 02:40:59,207 Epoch 45 Step: 255700 Batch Loss: 1.071252 Tokens per Sec: 19136, Lr: 0.000035\n", "2020-01-18 02:41:05,349 Epoch 45: total training loss 2939.65\n", "2020-01-18 02:41:05,350 Training ended after 45 epochs.\n", "2020-01-18 02:41:05,350 Best validation result (greedy) at step 250000: 3.46 ppl.\n", "2020-01-18 02:41:27,216 dev bleu: 32.58 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-01-18 02:41:27,221 Translations saved to: /content/drive/My Drive/masakhane/en-iso-baseline/models/eniso_transformer/00250000.hyps.dev\n", "2020-01-18 02:42:02,073 test bleu: 38.05 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-01-18 02:42:02,080 Translations saved to: /content/drive/My Drive/masakhane/en-iso-baseline/models/eniso_transformer/00250000.hyps.test\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "MBoDS09JM807", "colab": {} }, "source": [ "# Copy the created models from the notebook storage to google drive for persistant storage \n", "# !cp -r joeynmt/models/${src}${tgt}_transformer/* \"$gdrive_path/models/${src}${tgt}_transformer/\"" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "n94wlrCjVc17", "outputId": "280ab882-55a7-40b6-fb19-3087275ac87b", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Output our validation accuracy epoch 1-30\n", "! cat \"$gdrive_path/models/${src}${tgt}_transformer/validations.txt\"" ], "execution_count": 21, "outputs": [ { "output_type": "stream", "text": [ "Steps: 1000\tLoss: 94234.79688\tPPL: 37.89809\tbleu: 2.08377\tLR: 0.00030000\t*\n", "Steps: 2000\tLoss: 80098.55469\tPPL: 21.96886\tbleu: 4.41429\tLR: 0.00030000\t*\n", "Steps: 3000\tLoss: 71809.64062\tPPL: 15.95703\tbleu: 6.13988\tLR: 0.00030000\t*\n", "Steps: 4000\tLoss: 66469.93750\tPPL: 12.98678\tbleu: 9.18670\tLR: 0.00030000\t*\n", "Steps: 5000\tLoss: 62164.83594\tPPL: 10.99975\tbleu: 11.60847\tLR: 0.00030000\t*\n", "Steps: 6000\tLoss: 59310.51953\tPPL: 9.85297\tbleu: 13.32222\tLR: 0.00030000\t*\n", "Steps: 7000\tLoss: 56697.83594\tPPL: 8.90840\tbleu: 14.61969\tLR: 0.00030000\t*\n", "Steps: 8000\tLoss: 55065.73047\tPPL: 8.36486\tbleu: 15.46040\tLR: 0.00030000\t*\n", "Steps: 9000\tLoss: 53112.52734\tPPL: 7.75780\tbleu: 17.31818\tLR: 0.00030000\t*\n", "Steps: 10000\tLoss: 51696.43750\tPPL: 7.34542\tbleu: 17.20387\tLR: 0.00030000\t*\n", "Steps: 11000\tLoss: 50569.53906\tPPL: 7.03297\tbleu: 17.98495\tLR: 0.00030000\t*\n", "Steps: 12000\tLoss: 49303.01953\tPPL: 6.69764\tbleu: 18.85888\tLR: 0.00030000\t*\n", "Steps: 13000\tLoss: 48429.05469\tPPL: 6.47562\tbleu: 19.24016\tLR: 0.00030000\t*\n", "Steps: 14000\tLoss: 47421.25000\tPPL: 6.22872\tbleu: 20.29620\tLR: 0.00030000\t*\n", "Steps: 15000\tLoss: 46672.74609\tPPL: 6.05145\tbleu: 20.52968\tLR: 0.00030000\t*\n", "Steps: 16000\tLoss: 46095.49219\tPPL: 5.91820\tbleu: 20.91999\tLR: 0.00030000\t*\n", "Steps: 17000\tLoss: 45379.99219\tPPL: 5.75710\tbleu: 21.96850\tLR: 0.00030000\t*\n", "Steps: 18000\tLoss: 44758.84766\tPPL: 5.62080\tbleu: 21.40537\tLR: 0.00030000\t*\n", "Steps: 19000\tLoss: 44534.36719\tPPL: 5.57234\tbleu: 22.14367\tLR: 0.00030000\t*\n", "Steps: 20000\tLoss: 44845.13672\tPPL: 5.63954\tbleu: 20.87363\tLR: 0.00030000\t\n", "Steps: 21000\tLoss: 43438.25781\tPPL: 5.34165\tbleu: 23.31405\tLR: 0.00030000\t*\n", "Steps: 22000\tLoss: 43037.99219\tPPL: 5.25982\tbleu: 22.98467\tLR: 0.00030000\t*\n", "Steps: 23000\tLoss: 42851.05859\tPPL: 5.22203\tbleu: 23.75610\tLR: 0.00030000\t*\n", "Steps: 24000\tLoss: 42324.09766\tPPL: 5.11695\tbleu: 23.37798\tLR: 0.00030000\t*\n", "Steps: 25000\tLoss: 41840.85938\tPPL: 5.02246\tbleu: 24.11398\tLR: 0.00030000\t*\n", "Steps: 26000\tLoss: 41932.92578\tPPL: 5.04033\tbleu: 24.44848\tLR: 0.00030000\t\n", "Steps: 27000\tLoss: 41288.82031\tPPL: 4.91664\tbleu: 24.48488\tLR: 0.00030000\t*\n", "Steps: 28000\tLoss: 41143.61719\tPPL: 4.88918\tbleu: 24.75412\tLR: 0.00030000\t*\n", "Steps: 29000\tLoss: 40905.48828\tPPL: 4.84448\tbleu: 24.61773\tLR: 0.00030000\t*\n", "Steps: 30000\tLoss: 40556.39062\tPPL: 4.77968\tbleu: 25.65801\tLR: 0.00030000\t*\n", "Steps: 31000\tLoss: 40280.01172\tPPL: 4.72900\tbleu: 25.36187\tLR: 0.00030000\t*\n", "Steps: 32000\tLoss: 40043.01172\tPPL: 4.68596\tbleu: 25.48974\tLR: 0.00030000\t*\n", "Steps: 33000\tLoss: 39784.09766\tPPL: 4.63940\tbleu: 25.54002\tLR: 0.00030000\t*\n", "Steps: 34000\tLoss: 39677.82031\tPPL: 4.62042\tbleu: 25.62489\tLR: 0.00030000\t*\n", "Steps: 35000\tLoss: 39490.33203\tPPL: 4.58712\tbleu: 26.49471\tLR: 0.00030000\t*\n", "Steps: 36000\tLoss: 39363.20312\tPPL: 4.56468\tbleu: 25.94072\tLR: 0.00030000\t*\n", "Steps: 37000\tLoss: 39259.39453\tPPL: 4.54644\tbleu: 26.27984\tLR: 0.00030000\t*\n", "Steps: 38000\tLoss: 39367.76172\tPPL: 4.56549\tbleu: 25.91601\tLR: 0.00030000\t\n", "Steps: 39000\tLoss: 38852.83203\tPPL: 4.47570\tbleu: 26.36728\tLR: 0.00030000\t*\n", "Steps: 40000\tLoss: 38837.37500\tPPL: 4.47303\tbleu: 27.19716\tLR: 0.00030000\t*\n", "Steps: 41000\tLoss: 38758.82031\tPPL: 4.45950\tbleu: 27.08113\tLR: 0.00030000\t*\n", "Steps: 42000\tLoss: 38634.75391\tPPL: 4.43821\tbleu: 26.10681\tLR: 0.00030000\t*\n", "Steps: 43000\tLoss: 38215.01562\tPPL: 4.36693\tbleu: 26.85208\tLR: 0.00030000\t*\n", "Steps: 44000\tLoss: 38140.56250\tPPL: 4.35441\tbleu: 26.70836\tLR: 0.00030000\t*\n", "Steps: 45000\tLoss: 37830.87891\tPPL: 4.30270\tbleu: 26.83892\tLR: 0.00030000\t*\n", "Steps: 46000\tLoss: 37947.87109\tPPL: 4.32216\tbleu: 27.22322\tLR: 0.00030000\t\n", "Steps: 47000\tLoss: 37837.01953\tPPL: 4.30372\tbleu: 26.90088\tLR: 0.00030000\t\n", "Steps: 48000\tLoss: 37429.32031\tPPL: 4.23657\tbleu: 26.76920\tLR: 0.00030000\t*\n", "Steps: 49000\tLoss: 37535.80859\tPPL: 4.25401\tbleu: 27.91024\tLR: 0.00030000\t\n", "Steps: 50000\tLoss: 37384.20312\tPPL: 4.22920\tbleu: 27.38686\tLR: 0.00030000\t*\n", "Steps: 51000\tLoss: 37488.26562\tPPL: 4.24621\tbleu: 27.30332\tLR: 0.00030000\t\n", "Steps: 52000\tLoss: 37249.16797\tPPL: 4.20723\tbleu: 27.73932\tLR: 0.00030000\t*\n", "Steps: 53000\tLoss: 37049.71484\tPPL: 4.17499\tbleu: 27.67015\tLR: 0.00030000\t*\n", "Steps: 54000\tLoss: 37027.88672\tPPL: 4.17147\tbleu: 28.06041\tLR: 0.00030000\t*\n", "Steps: 55000\tLoss: 36923.14453\tPPL: 4.15465\tbleu: 28.32371\tLR: 0.00030000\t*\n", "Steps: 56000\tLoss: 36901.88672\tPPL: 4.15125\tbleu: 28.38773\tLR: 0.00030000\t*\n", "Steps: 57000\tLoss: 36688.31641\tPPL: 4.11719\tbleu: 27.99866\tLR: 0.00030000\t*\n", "Steps: 58000\tLoss: 36623.31641\tPPL: 4.10688\tbleu: 28.00368\tLR: 0.00030000\t*\n", "Steps: 59000\tLoss: 36544.61328\tPPL: 4.09443\tbleu: 28.65796\tLR: 0.00030000\t*\n", "Steps: 60000\tLoss: 36513.17578\tPPL: 4.08947\tbleu: 28.05014\tLR: 0.00030000\t*\n", "Steps: 61000\tLoss: 36508.17578\tPPL: 4.08868\tbleu: 28.72723\tLR: 0.00030000\t*\n", "Steps: 62000\tLoss: 36356.16016\tPPL: 4.06478\tbleu: 28.56549\tLR: 0.00030000\t*\n", "Steps: 63000\tLoss: 36310.91016\tPPL: 4.05769\tbleu: 28.21350\tLR: 0.00030000\t*\n", "Steps: 64000\tLoss: 36172.55078\tPPL: 4.03609\tbleu: 28.46607\tLR: 0.00030000\t*\n", "Steps: 65000\tLoss: 36344.39453\tPPL: 4.06293\tbleu: 28.36017\tLR: 0.00030000\t\n", "Steps: 66000\tLoss: 36147.22656\tPPL: 4.03215\tbleu: 29.02760\tLR: 0.00030000\t*\n", "Steps: 67000\tLoss: 36021.07422\tPPL: 4.01258\tbleu: 28.62494\tLR: 0.00030000\t*\n", "Steps: 68000\tLoss: 35972.80469\tPPL: 4.00511\tbleu: 28.98547\tLR: 0.00030000\t*\n", "Steps: 69000\tLoss: 36014.33203\tPPL: 4.01153\tbleu: 28.50999\tLR: 0.00030000\t\n", "Steps: 70000\tLoss: 35944.46484\tPPL: 4.00074\tbleu: 28.83174\tLR: 0.00030000\t*\n", "Steps: 71000\tLoss: 36056.96875\tPPL: 4.01814\tbleu: 28.60269\tLR: 0.00030000\t\n", "Steps: 72000\tLoss: 35998.14844\tPPL: 4.00903\tbleu: 28.93575\tLR: 0.00030000\t\n", "Steps: 73000\tLoss: 35760.00000\tPPL: 3.97237\tbleu: 28.95383\tLR: 0.00030000\t*\n", "Steps: 74000\tLoss: 35690.31250\tPPL: 3.96171\tbleu: 28.99425\tLR: 0.00030000\t*\n", "Steps: 75000\tLoss: 35568.84375\tPPL: 3.94319\tbleu: 29.37997\tLR: 0.00030000\t*\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "P5C8PYXRj6D5", "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "outputId": "46e3a1bb-acde-4ec0-a79f-50088997adee" }, "source": [ "# Output our validation accuracy last 45 epochs (upto ~100, step 250000)\n", "! cat \"$gdrive_path/models/${src}${tgt}_transformer/validations.txt\"" ], "execution_count": 10, "outputs": [ { "output_type": "stream", "text": [ "Steps: 143000\tLoss: 33366.60938\tPPL: 3.62206\tbleu: 31.11651\tLR: 0.00021000\t\n", "Steps: 144000\tLoss: 33235.20312\tPPL: 3.60375\tbleu: 30.60829\tLR: 0.00021000\t*\n", "Steps: 145000\tLoss: 33330.00781\tPPL: 3.61695\tbleu: 31.30943\tLR: 0.00021000\t\n", "Steps: 146000\tLoss: 33293.15234\tPPL: 3.61181\tbleu: 31.12491\tLR: 0.00021000\t\n", "Steps: 147000\tLoss: 33180.60156\tPPL: 3.59617\tbleu: 31.44658\tLR: 0.00021000\t*\n", "Steps: 148000\tLoss: 33217.98828\tPPL: 3.60136\tbleu: 31.32405\tLR: 0.00021000\t\n", "Steps: 149000\tLoss: 33266.59766\tPPL: 3.60812\tbleu: 31.32715\tLR: 0.00021000\t\n", "Steps: 150000\tLoss: 33267.26172\tPPL: 3.60821\tbleu: 30.70815\tLR: 0.00021000\t\n", "Steps: 151000\tLoss: 33168.08984\tPPL: 3.59443\tbleu: 31.32964\tLR: 0.00021000\t*\n", "Steps: 152000\tLoss: 33182.56641\tPPL: 3.59644\tbleu: 30.97980\tLR: 0.00021000\t\n", "Steps: 153000\tLoss: 33219.05859\tPPL: 3.60151\tbleu: 31.11813\tLR: 0.00021000\t\n", "Steps: 154000\tLoss: 33167.32812\tPPL: 3.59433\tbleu: 31.04881\tLR: 0.00021000\t*\n", "Steps: 155000\tLoss: 33235.98828\tPPL: 3.60386\tbleu: 31.13271\tLR: 0.00021000\t\n", "Steps: 156000\tLoss: 33227.29297\tPPL: 3.60265\tbleu: 31.53557\tLR: 0.00021000\t\n", "Steps: 157000\tLoss: 33150.13672\tPPL: 3.59194\tbleu: 31.35000\tLR: 0.00021000\t*\n", "Steps: 158000\tLoss: 33310.57031\tPPL: 3.61424\tbleu: 30.74298\tLR: 0.00021000\t\n", "Steps: 159000\tLoss: 33128.37891\tPPL: 3.58893\tbleu: 31.32122\tLR: 0.00021000\t*\n", "Steps: 160000\tLoss: 33135.18750\tPPL: 3.58987\tbleu: 31.46437\tLR: 0.00021000\t\n", "Steps: 161000\tLoss: 33146.13281\tPPL: 3.59139\tbleu: 31.55068\tLR: 0.00021000\t\n", "Steps: 162000\tLoss: 33080.03125\tPPL: 3.58224\tbleu: 31.54937\tLR: 0.00021000\t*\n", "Steps: 163000\tLoss: 33079.60938\tPPL: 3.58219\tbleu: 31.65488\tLR: 0.00021000\t*\n", "Steps: 164000\tLoss: 33093.65234\tPPL: 3.58413\tbleu: 31.52681\tLR: 0.00021000\t\n", "Steps: 165000\tLoss: 33284.89844\tPPL: 3.61066\tbleu: 31.67963\tLR: 0.00021000\t\n", "Steps: 166000\tLoss: 33037.52344\tPPL: 3.57638\tbleu: 31.04570\tLR: 0.00021000\t*\n", "Steps: 167000\tLoss: 33014.28516\tPPL: 3.57317\tbleu: 31.33825\tLR: 0.00021000\t*\n", "Steps: 168000\tLoss: 33147.79297\tPPL: 3.59162\tbleu: 31.43172\tLR: 0.00021000\t\n", "Steps: 169000\tLoss: 33050.36328\tPPL: 3.57815\tbleu: 31.51004\tLR: 0.00021000\t\n", "Steps: 170000\tLoss: 33142.08984\tPPL: 3.59083\tbleu: 31.28554\tLR: 0.00021000\t\n", "Steps: 171000\tLoss: 33071.93750\tPPL: 3.58113\tbleu: 31.31454\tLR: 0.00021000\t\n", "Steps: 172000\tLoss: 33148.04297\tPPL: 3.59165\tbleu: 31.30697\tLR: 0.00021000\t\n", "Steps: 173000\tLoss: 32963.51953\tPPL: 3.56618\tbleu: 31.67199\tLR: 0.00021000\t*\n", "Steps: 174000\tLoss: 33111.25781\tPPL: 3.58656\tbleu: 31.16471\tLR: 0.00021000\t\n", "Steps: 175000\tLoss: 33074.26953\tPPL: 3.58145\tbleu: 31.41755\tLR: 0.00021000\t\n", "Steps: 176000\tLoss: 33076.17188\tPPL: 3.58171\tbleu: 31.50669\tLR: 0.00021000\t\n", "Steps: 177000\tLoss: 33007.60547\tPPL: 3.57225\tbleu: 31.14142\tLR: 0.00021000\t\n", "Steps: 178000\tLoss: 33101.43359\tPPL: 3.58520\tbleu: 31.26915\tLR: 0.00021000\t\n", "Steps: 179000\tLoss: 32988.21875\tPPL: 3.56958\tbleu: 31.59069\tLR: 0.00014700\t\n", "Steps: 180000\tLoss: 33004.18750\tPPL: 3.57178\tbleu: 31.01125\tLR: 0.00014700\t\n", "Steps: 181000\tLoss: 32881.06250\tPPL: 3.55486\tbleu: 31.42336\tLR: 0.00014700\t*\n", "Steps: 182000\tLoss: 32908.01562\tPPL: 3.55855\tbleu: 31.23916\tLR: 0.00014700\t\n", "Steps: 183000\tLoss: 32924.68750\tPPL: 3.56084\tbleu: 31.49475\tLR: 0.00014700\t\n", "Steps: 184000\tLoss: 32812.78516\tPPL: 3.54551\tbleu: 31.20760\tLR: 0.00014700\t*\n", "Steps: 185000\tLoss: 32739.67969\tPPL: 3.53552\tbleu: 31.32815\tLR: 0.00014700\t*\n", "Steps: 186000\tLoss: 32805.17188\tPPL: 3.54447\tbleu: 31.48863\tLR: 0.00014700\t\n", "Steps: 187000\tLoss: 32724.63086\tPPL: 3.53347\tbleu: 31.44920\tLR: 0.00014700\t*\n", "Steps: 188000\tLoss: 32821.34766\tPPL: 3.54668\tbleu: 31.76145\tLR: 0.00014700\t\n", "Steps: 189000\tLoss: 32760.18555\tPPL: 3.53832\tbleu: 31.40587\tLR: 0.00014700\t\n", "Steps: 190000\tLoss: 32747.72461\tPPL: 3.53662\tbleu: 31.81834\tLR: 0.00014700\t\n", "Steps: 191000\tLoss: 32657.43359\tPPL: 3.52432\tbleu: 31.88885\tLR: 0.00014700\t*\n", "Steps: 192000\tLoss: 32732.07812\tPPL: 3.53449\tbleu: 31.19510\tLR: 0.00014700\t\n", "Steps: 193000\tLoss: 32709.13867\tPPL: 3.53136\tbleu: 31.56380\tLR: 0.00014700\t\n", "Steps: 194000\tLoss: 32802.26172\tPPL: 3.54407\tbleu: 31.59507\tLR: 0.00014700\t\n", "Steps: 195000\tLoss: 32665.66797\tPPL: 3.52544\tbleu: 31.61228\tLR: 0.00014700\t\n", "Steps: 196000\tLoss: 32666.77539\tPPL: 3.52559\tbleu: 31.68255\tLR: 0.00014700\t\n", "Steps: 197000\tLoss: 32716.40234\tPPL: 3.53235\tbleu: 32.00604\tLR: 0.00010290\t\n", "Steps: 198000\tLoss: 32703.61914\tPPL: 3.53061\tbleu: 31.86405\tLR: 0.00010290\t\n", "Steps: 199000\tLoss: 32634.68750\tPPL: 3.52123\tbleu: 31.53987\tLR: 0.00010290\t*\n", "Steps: 200000\tLoss: 32606.65039\tPPL: 3.51743\tbleu: 32.01149\tLR: 0.00010290\t*\n", "Steps: 201000\tLoss: 32624.09961\tPPL: 3.51980\tbleu: 31.92480\tLR: 0.00010290\t\n", "Steps: 202000\tLoss: 32595.18555\tPPL: 3.51587\tbleu: 32.11004\tLR: 0.00010290\t*\n", "Steps: 203000\tLoss: 32705.66406\tPPL: 3.53089\tbleu: 31.75624\tLR: 0.00010290\t\n", "Steps: 204000\tLoss: 32633.04492\tPPL: 3.52101\tbleu: 31.77189\tLR: 0.00010290\t\n", "Steps: 205000\tLoss: 32577.72266\tPPL: 3.51351\tbleu: 31.64475\tLR: 0.00010290\t*\n", "Steps: 206000\tLoss: 32588.16602\tPPL: 3.51492\tbleu: 31.77710\tLR: 0.00010290\t\n", "Steps: 207000\tLoss: 32693.01172\tPPL: 3.52916\tbleu: 31.81219\tLR: 0.00010290\t\n", "Steps: 208000\tLoss: 32622.94727\tPPL: 3.51964\tbleu: 32.09540\tLR: 0.00010290\t\n", "Steps: 209000\tLoss: 32566.51953\tPPL: 3.51199\tbleu: 32.00634\tLR: 0.00010290\t*\n", "Steps: 210000\tLoss: 32484.92578\tPPL: 3.50095\tbleu: 31.81163\tLR: 0.00010290\t*\n", "Steps: 211000\tLoss: 32594.23828\tPPL: 3.51574\tbleu: 31.97931\tLR: 0.00010290\t\n", "Steps: 212000\tLoss: 32526.58008\tPPL: 3.50658\tbleu: 31.63183\tLR: 0.00010290\t\n", "Steps: 213000\tLoss: 32552.91211\tPPL: 3.51014\tbleu: 31.88180\tLR: 0.00010290\t\n", "Steps: 214000\tLoss: 32582.36914\tPPL: 3.51413\tbleu: 32.05458\tLR: 0.00010290\t\n", "Steps: 215000\tLoss: 32548.73047\tPPL: 3.50958\tbleu: 32.09711\tLR: 0.00010290\t\n", "Steps: 216000\tLoss: 32515.74414\tPPL: 3.50512\tbleu: 31.68310\tLR: 0.00007203\t\n", "Steps: 217000\tLoss: 32480.00195\tPPL: 3.50029\tbleu: 31.80098\tLR: 0.00007203\t*\n", "Steps: 218000\tLoss: 32434.72852\tPPL: 3.49418\tbleu: 31.83858\tLR: 0.00007203\t*\n", "Steps: 219000\tLoss: 32366.59766\tPPL: 3.48501\tbleu: 32.29820\tLR: 0.00007203\t*\n", "Steps: 220000\tLoss: 32395.39453\tPPL: 3.48888\tbleu: 32.04569\tLR: 0.00007203\t\n", "Steps: 221000\tLoss: 32392.70312\tPPL: 3.48852\tbleu: 31.80917\tLR: 0.00007203\t\n", "Steps: 222000\tLoss: 32367.43359\tPPL: 3.48512\tbleu: 32.10322\tLR: 0.00007203\t\n", "Steps: 223000\tLoss: 32454.59570\tPPL: 3.49686\tbleu: 31.98180\tLR: 0.00007203\t\n", "Steps: 224000\tLoss: 32331.96875\tPPL: 3.48036\tbleu: 32.16459\tLR: 0.00007203\t*\n", "Steps: 225000\tLoss: 32315.23438\tPPL: 3.47811\tbleu: 32.17856\tLR: 0.00007203\t*\n", "Steps: 226000\tLoss: 32347.13867\tPPL: 3.48239\tbleu: 32.08161\tLR: 0.00007203\t\n", "Steps: 227000\tLoss: 32336.61523\tPPL: 3.48098\tbleu: 32.22578\tLR: 0.00007203\t\n", "Steps: 228000\tLoss: 32341.70312\tPPL: 3.48166\tbleu: 32.01166\tLR: 0.00007203\t\n", "Steps: 229000\tLoss: 32335.50781\tPPL: 3.48083\tbleu: 32.18935\tLR: 0.00007203\t\n", "Steps: 230000\tLoss: 32321.14062\tPPL: 3.47890\tbleu: 32.10579\tLR: 0.00007203\t\n", "Steps: 231000\tLoss: 32328.39062\tPPL: 3.47988\tbleu: 31.97715\tLR: 0.00005042\t\n", "Steps: 232000\tLoss: 32316.39453\tPPL: 3.47827\tbleu: 32.25027\tLR: 0.00005042\t\n", "Steps: 233000\tLoss: 32303.58594\tPPL: 3.47655\tbleu: 32.18177\tLR: 0.00005042\t*\n", "Steps: 234000\tLoss: 32249.60547\tPPL: 3.46932\tbleu: 32.04732\tLR: 0.00005042\t*\n", "Steps: 235000\tLoss: 32258.85547\tPPL: 3.47056\tbleu: 32.20643\tLR: 0.00005042\t\n", "Steps: 236000\tLoss: 32247.76758\tPPL: 3.46907\tbleu: 32.24648\tLR: 0.00005042\t*\n", "Steps: 237000\tLoss: 32213.24023\tPPL: 3.46445\tbleu: 32.25782\tLR: 0.00005042\t*\n", "Steps: 238000\tLoss: 32247.87305\tPPL: 3.46909\tbleu: 32.19444\tLR: 0.00005042\t\n", "Steps: 239000\tLoss: 32267.72266\tPPL: 3.47174\tbleu: 32.10197\tLR: 0.00005042\t\n", "Steps: 240000\tLoss: 32232.06445\tPPL: 3.46697\tbleu: 32.10608\tLR: 0.00005042\t\n", "Steps: 241000\tLoss: 32303.05664\tPPL: 3.47648\tbleu: 32.03237\tLR: 0.00005042\t\n", "Steps: 242000\tLoss: 32235.70312\tPPL: 3.46746\tbleu: 32.22747\tLR: 0.00005042\t\n", "Steps: 243000\tLoss: 32241.70898\tPPL: 3.46826\tbleu: 32.21955\tLR: 0.00003529\t\n", "Steps: 244000\tLoss: 32189.13086\tPPL: 3.46123\tbleu: 32.26463\tLR: 0.00003529\t*\n", "Steps: 245000\tLoss: 32210.95117\tPPL: 3.46415\tbleu: 32.26037\tLR: 0.00003529\t\n", "Steps: 246000\tLoss: 32175.24219\tPPL: 3.45938\tbleu: 32.17156\tLR: 0.00003529\t*\n", "Steps: 247000\tLoss: 32237.33789\tPPL: 3.46768\tbleu: 32.35194\tLR: 0.00003529\t\n", "Steps: 248000\tLoss: 32201.80859\tPPL: 3.46293\tbleu: 32.18360\tLR: 0.00003529\t\n", "Steps: 249000\tLoss: 32189.31250\tPPL: 3.46126\tbleu: 32.26904\tLR: 0.00003529\t\n", "Steps: 250000\tLoss: 32149.13086\tPPL: 3.45590\tbleu: 32.00540\tLR: 0.00003529\t*\n", "Steps: 251000\tLoss: 32180.59766\tPPL: 3.46009\tbleu: 32.07722\tLR: 0.00003529\t\n", "Steps: 252000\tLoss: 32209.50000\tPPL: 3.46395\tbleu: 32.35816\tLR: 0.00003529\t\n", "Steps: 253000\tLoss: 32154.10352\tPPL: 3.45656\tbleu: 32.47964\tLR: 0.00003529\t\n", "Steps: 254000\tLoss: 32184.17188\tPPL: 3.46057\tbleu: 32.27139\tLR: 0.00003529\t\n", "Steps: 255000\tLoss: 32182.65820\tPPL: 3.46037\tbleu: 32.03828\tLR: 0.00003529\t\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "id": "66WhRE9lIhoD", "outputId": "3ad07c70-8750-44f9-c47e-8763e250ed61", "colab": { "base_uri": "https://localhost:8080/", "height": 68 } }, "source": [ "# Test our model\n", "! cd joeynmt; python3 -m joeynmt test \"$gdrive_path/models/${src}${tgt}_transformer/config.yaml\"" ], "execution_count": 11, "outputs": [ { "output_type": "stream", "text": [ "2020-01-18 02:53:15,890 Hello! This is Joey-NMT.\n", "2020-01-18 02:53:41,222 dev bleu: 32.58 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-01-18 02:54:16,562 test bleu: 38.05 [Beam search decoding with beam size = 5 and alpha = 1.0]\n" ], "name": "stdout" } ] } ] }