{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "starter_notebook.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.6"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Igc5itf-xMGj"
      },
      "source": [
        "# Masakhane - Machine Translation for African Languages (Using JoeyNMT)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "x4fXCKCf36IK"
      },
      "source": [
        "## Note before beginning:\n",
        "### - The idea is that you should be able to make minimal changes to this in order to get SOME result for your own translation corpus. \n",
        "\n",
        "### - The tl;dr: Go to the **\"TODO\"** comments which will tell you what to update to get up and running\n",
        "\n",
        "### - If you actually want to have a clue what you're doing, read the text and peek at the links\n",
        "\n",
        "### - With 100 epochs, it should take around 7 hours to run in Google Colab\n",
        "\n",
        "### - Once you've gotten a result for your language, please attach and email your notebook that generated it to masakhanetranslation@gmail.com\n",
        "\n",
        "### - If you care enough and get a chance, doing a brief background on your language would be amazing. See examples in  [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "l929HimrxS0a"
      },
      "source": [
        "## Retrieve your data & make a parallel corpus\n",
        "\n",
        "If you are wanting to use the JW300 data referenced on the Masakhane website or in our GitHub repo, you can use `opus-tools` to convert the data into a convenient format. `opus_read` from that package provides a convenient tool for reading the native aligned XML files and to convert them to TMX format. The tool can also be used to fetch relevant files from OPUS on the fly and to filter the data as necessary. [Read the documentation](https://pypi.org/project/opustools-pkg/) for more details.\n",
        "\n",
        "Once you have your corpus files in TMX format (an xml structure which will include the sentences in your target language and your source language in a single file), we recommend reading them into a pandas dataframe. Thankfully, Jade wrote a silly `tmx2dataframe` package which converts your tmx file to a pandas dataframe. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "oGRmDELn7Az0",
        "outputId": "d82d9a36-9bd4-43c9-fcaf-966eacf72e2e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Cn3tgQLzUxwn",
        "colab": {}
      },
      "source": [
        "# TODO: Set your source and target languages. Keep in mind, these traditionally use language codes as found here:\n",
        "# These will also become the suffix's of all vocab and corpus files used throughout\n",
        "import os\n",
        "source_language = \"en\"\n",
        "target_language = \"bin\" \n",
        "lc = False  # If True, lowercase the data.\n",
        "seed = 42  # Random seed for shuffling.\n",
        "tag = \"baseline\" # Give a unique name to your folder - this is to ensure you don't rewrite any models you've already submitted\n",
        "\n",
        "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n",
        "os.environ[\"tgt\"] = target_language\n",
        "os.environ[\"tag\"] = tag\n",
        "\n",
        "# This will save it to a folder in our gdrive instead!\n",
        "!mkdir -p \"/content/drive/My Drive/masakhane/$src-$tgt-$tag\"\n",
        "os.environ[\"gdrive_path\"] = \"/content/drive/My Drive/masakhane/%s-%s-%s\" % (source_language, target_language, tag)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kBSgJHEw7Nvx",
        "outputId": "4d99ba0e-6b19-4652-9b6d-96e57d891f5b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "!echo $gdrive_path"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/My Drive/masakhane/en-bin-baseline\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "gA75Fs9ys8Y9",
        "outputId": "89beb74b-5d6c-4bf5-e89e-56e35670c582",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Install opus-tools\n",
        "! pip install opustools-pkg"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting opustools-pkg\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/6c/9f/e829a0cceccc603450cd18e1ff80807b6237a88d9a8df2c0bb320796e900/opustools_pkg-0.0.52-py3-none-any.whl (80kB)\n",
            "\r\u001b[K     |████                            | 10kB 22.2MB/s eta 0:00:01\r\u001b[K     |████████                        | 20kB 7.3MB/s eta 0:00:01\r\u001b[K     |████████████▏                   | 30kB 8.3MB/s eta 0:00:01\r\u001b[K     |████████████████▏               | 40kB 6.3MB/s eta 0:00:01\r\u001b[K     |████████████████████▎           | 51kB 7.6MB/s eta 0:00:01\r\u001b[K     |████████████████████████▎       | 61kB 9.0MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▎   | 71kB 9.2MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 81kB 5.3MB/s \n",
            "\u001b[?25hInstalling collected packages: opustools-pkg\n",
            "Successfully installed opustools-pkg-0.0.52\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "xq-tDZVks7ZD",
        "outputId": "36fd305c-ceb5-495e-9466-a1e0dd642ece",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Downloading our corpus\n",
        "! opus_read -d JW300 -s $src -t $tgt -wm moses -w jw300.$src jw300.$tgt -q\n",
        "\n",
        "# extract the corpus file\n",
        "! gunzip JW300_latest_xml_$src-$tgt.xml.gz"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Alignment file /proj/nlpl/data/OPUS/JW300/latest/xml/bin-en.xml.gz not found. The following files are available for downloading:\n",
            "\n",
            "Unable to retrieve the data.\n",
            "./JW300_latest_xml_bin-en.xml.gz ... 100% of 132 KB\n",
            "./JW300_latest_xml_bin.zip ... 100% of 1 MB\n",
            "./JW300_latest_xml_en.zip ... 100% of 263 MB\n",
            "gzip: JW300_latest_xml_en-bin.xml.gz: No such file or directory\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n48GDRnP8y2G",
        "colab_type": "code",
        "outputId": "5a12afd0-f784-4f58-ff94-303bf8afe263",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 578
        }
      },
      "source": [
        "# Download the global test set.\n",
        "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n",
        "  \n",
        "# And the specific test set for this language pair.\n",
        "os.environ[\"trg\"] = target_language \n",
        "os.environ[\"src\"] = source_language \n",
        "\n",
        "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.en \n",
        "! mv test.en-$trg.en test.en\n",
        "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.$trg \n",
        "! mv test.en-$trg.$trg test.$trg"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-01-25 23:42:05--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 277791 (271K) [text/plain]\n",
            "Saving to: ‘test.en-any.en’\n",
            "\n",
            "test.en-any.en      100%[===================>] 271.28K  --.-KB/s    in 0.02s   \n",
            "\n",
            "2020-01-25 23:42:06 (14.9 MB/s) - ‘test.en-any.en’ saved [277791/277791]\n",
            "\n",
            "--2020-01-25 23:42:06--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-bin.en\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 62018 (61K) [text/plain]\n",
            "Saving to: ‘test.en-bin.en’\n",
            "\n",
            "test.en-bin.en      100%[===================>]  60.56K  --.-KB/s    in 0.008s  \n",
            "\n",
            "2020-01-25 23:42:07 (7.86 MB/s) - ‘test.en-bin.en’ saved [62018/62018]\n",
            "\n",
            "--2020-01-25 23:42:09--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-bin.bin\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 78356 (77K) [text/plain]\n",
            "Saving to: ‘test.en-bin.bin’\n",
            "\n",
            "test.en-bin.bin     100%[===================>]  76.52K  --.-KB/s    in 0.008s  \n",
            "\n",
            "2020-01-25 23:42:09 (9.96 MB/s) - ‘test.en-bin.bin’ saved [78356/78356]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NqDG-CI28y2L",
        "colab_type": "code",
        "outputId": "560d583f-601f-4276-98df-265025a6e11f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Read the test data to filter from train and dev splits.\n",
        "# Store english portion in set for quick filtering checks.\n",
        "en_test_sents = set()\n",
        "filter_test_sents = \"test.en-any.en\"\n",
        "j = 0\n",
        "with open(filter_test_sents) as f:\n",
        "  for line in f:\n",
        "    en_test_sents.add(line.strip())\n",
        "    j += 1\n",
        "print('Loaded {} global test sentences to filter from the training/dev data.'.format(j))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loaded 3571 global test sentences to filter from the training/dev data.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "3CNdwLBCfSIl",
        "outputId": "7713cfa4-c7f1-4186-c74b-dffa310e5b3c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        }
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "# TMX file to dataframe\n",
        "source_file = 'jw300.' + source_language\n",
        "target_file = 'jw300.' + target_language\n",
        "\n",
        "source = []\n",
        "target = []\n",
        "skip_lines = []  # Collect the line numbers of the source portion to skip the same lines for the target portion.\n",
        "with open(source_file) as f:\n",
        "    for i, line in enumerate(f):\n",
        "        # Skip sentences that are contained in the test set.\n",
        "        if line.strip() not in en_test_sents:\n",
        "            source.append(line.strip())\n",
        "        else:\n",
        "            skip_lines.append(i)             \n",
        "with open(target_file) as f:\n",
        "    for j, line in enumerate(f):\n",
        "        # Only add to corpus if corresponding source was not skipped.\n",
        "        if j not in skip_lines:\n",
        "            target.append(line.strip())\n",
        "    \n",
        "print('Loaded data and skipped {}/{} lines since contained in test set.'.format(len(skip_lines), i))\n",
        "    \n",
        "df = pd.DataFrame(zip(source, target), columns=['source_sentence', 'target_sentence'])\n",
        "# if you get TypeError: data argument can't be an iterator is because of your zip version run this below\n",
        "#df = pd.DataFrame(list(zip(source, target)), columns=['source_sentence', 'target_sentence'])\n",
        "df.head(3)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loaded data and skipped 1074/13416 lines since contained in test set.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>source_sentence</th>\n",
              "      <th>target_sentence</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3 Do You Remember ?</td>\n",
              "      <td>3 U Yerre Ra ?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>4 Jehovah Provides Comfort in All Our Trials</td>\n",
              "      <td>4 E Jehova Ẹre Ọ Rhie Ifuẹko Ne Ima Vbuwe Ọlọg...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Though we all face trials and tribulations , J...</td>\n",
              "      <td>Ma hia werriẹ aro daa edanmwẹ kevbe ọlọghọmwa ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                     source_sentence                                    target_sentence\n",
              "0                                3 Do You Remember ?                                     3 U Yerre Ra ?\n",
              "1       4 Jehovah Provides Comfort in All Our Trials  4 E Jehova Ẹre Ọ Rhie Ifuẹko Ne Ima Vbuwe Ọlọg...\n",
              "2  Though we all face trials and tribulations , J...  Ma hia werriẹ aro daa edanmwẹ kevbe ọlọghọmwa ..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YkuK3B4p2AkN"
      },
      "source": [
        "## Pre-processing and export\n",
        "\n",
        "It is generally a good idea to remove duplicate translations and conflicting translations from the corpus. In practice, these public corpora include some number of these that need to be cleaned.\n",
        "\n",
        "In addition we will split our data into dev/test/train and export to the filesystem."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "M_2ouEOH1_1q",
        "outputId": "6bcf1274-f533-43f6-8dff-2da2756c43e6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "# drop duplicate translations\n",
        "df_pp = df.drop_duplicates()\n",
        "\n",
        "# drop conflicting translations\n",
        "# (this is optional and something that you might want to comment out \n",
        "# depending on the size of your corpus)\n",
        "df_pp.drop_duplicates(subset='source_sentence', inplace=True)\n",
        "df_pp.drop_duplicates(subset='target_sentence', inplace=True)\n",
        "\n",
        "# Shuffle the data to remove bias in dev set selection.\n",
        "df_pp = df_pp.sample(frac=1, random_state=seed).reset_index(drop=True)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:6: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame\n",
            "\n",
            "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  \n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:7: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame\n",
            "\n",
            "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  import sys\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z_1BwAApEtMk",
        "colab_type": "code",
        "outputId": "3e6aec03-6c35-4b8f-9347-22424a38cc9d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 530
        }
      },
      "source": [
        "# Install fuzzy wuzzy to remove \"almost duplicate\" sentences in the\n",
        "# test and training sets.\n",
        "! pip install fuzzywuzzy\n",
        "! pip install python-Levenshtein\n",
        "import time\n",
        "from fuzzywuzzy import process\n",
        "import numpy as np\n",
        "\n",
        "# reset the index of the training set after previous filtering\n",
        "df_pp.reset_index(drop=False, inplace=True)\n",
        "\n",
        "# Remove samples from the training data set if they \"almost overlap\" with the\n",
        "# samples in the test set.\n",
        "\n",
        "# Filtering function. Adjust pad to narrow down the candidate matches to\n",
        "# within a certain length of characters of the given sample.\n",
        "def fuzzfilter(sample, candidates, pad):\n",
        "  candidates = [x for x in candidates if len(x) <= len(sample)+pad and len(x) >= len(sample)-pad] \n",
        "  if len(candidates) > 0:\n",
        "    return process.extractOne(sample, candidates)[1]\n",
        "  else:\n",
        "    return np.nan\n",
        "\n",
        "# NOTE - This might run slow depending on the size of your training set. We are\n",
        "# printing some information to help you track how long it would take. \n",
        "scores = []\n",
        "start_time = time.time()\n",
        "for idx, row in df_pp.iterrows():\n",
        "  scores.append(fuzzfilter(row['source_sentence'], list(en_test_sents), 5))\n",
        "  if idx % 1000 == 0:\n",
        "    hours, rem = divmod(time.time() - start_time, 3600)\n",
        "    minutes, seconds = divmod(rem, 60)\n",
        "    print(\"{:0>2}:{:0>2}:{:05.2f}\".format(int(hours),int(minutes),seconds), \"%0.2f percent complete\" % (100.0*float(idx)/float(len(df_pp))))\n",
        "\n",
        "# Filter out \"almost overlapping samples\"\n",
        "df_pp['scores'] = scores\n",
        "df_pp = df_pp[df_pp['scores'] < 95]"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting fuzzywuzzy\n",
            "  Downloading https://files.pythonhosted.org/packages/d8/f1/5a267addb30ab7eaa1beab2b9323073815da4551076554ecc890a3595ec9/fuzzywuzzy-0.17.0-py2.py3-none-any.whl\n",
            "Installing collected packages: fuzzywuzzy\n",
            "Successfully installed fuzzywuzzy-0.17.0\n",
            "Collecting python-Levenshtein\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/42/a9/d1785c85ebf9b7dfacd08938dd028209c34a0ea3b1bcdb895208bd40a67d/python-Levenshtein-0.12.0.tar.gz (48kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 4.0MB/s \n",
            "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from python-Levenshtein) (42.0.2)\n",
            "Building wheels for collected packages: python-Levenshtein\n",
            "  Building wheel for python-Levenshtein (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for python-Levenshtein: filename=python_Levenshtein-0.12.0-cp36-cp36m-linux_x86_64.whl size=144668 sha256=2da979c686bdebfac68b0ff1161355d66695f0d2a13ed1c95a2d031663d8bd5e\n",
            "  Stored in directory: /root/.cache/pip/wheels/de/c2/93/660fd5f7559049268ad2dc6d81c4e39e9e36518766eaf7e342\n",
            "Successfully built python-Levenshtein\n",
            "Installing collected packages: python-Levenshtein\n",
            "Successfully installed python-Levenshtein-0.12.0\n",
            "00:00:00.02 0.00 percent complete\n",
            "00:00:20.43 8.91 percent complete\n",
            "00:00:41.42 17.82 percent complete\n",
            "00:01:01.58 26.73 percent complete\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "00:01:21.94 35.64 percent complete\n",
            "00:01:42.10 44.55 percent complete\n",
            "00:02:02.18 53.46 percent complete\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '*']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "00:02:22.60 62.37 percent complete\n",
            "00:02:43.41 71.28 percent complete\n",
            "00:03:03.81 80.19 percent complete\n",
            "00:03:24.06 89.09 percent complete\n",
            "00:03:44.29 98.00 percent complete\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "hxxBOCA-xXhy",
        "outputId": "47ca1742-a447-4f69-8fd5-d41f572eb641",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 799
        }
      },
      "source": [
        "# This section does the split between train/dev for the parallel corpora then saves them as separate files\n",
        "# We use 1000 dev test and the given test set.\n",
        "import csv\n",
        "\n",
        "# Do the split between dev/train and create parallel corpora\n",
        "num_dev_patterns = 1000\n",
        "\n",
        "# Optional: lower case the corpora - this will make it easier to generalize, but without proper casing.\n",
        "if lc:  # Julia: making lowercasing optional\n",
        "    df_pp[\"source_sentence\"] = df_pp[\"source_sentence\"].str.lower()\n",
        "    df_pp[\"target_sentence\"] = df_pp[\"target_sentence\"].str.lower()\n",
        "\n",
        "# Julia: test sets are already generated\n",
        "dev = df_pp.tail(num_dev_patterns) # Herman: Error in original\n",
        "stripped = df_pp.drop(df_pp.tail(num_dev_patterns).index)\n",
        "\n",
        "with open(\"train.\"+source_language, \"w\") as src_file, open(\"train.\"+target_language, \"w\") as trg_file:\n",
        "  for index, row in stripped.iterrows():\n",
        "    src_file.write(row[\"source_sentence\"]+\"\\n\")\n",
        "    trg_file.write(row[\"target_sentence\"]+\"\\n\")\n",
        "    \n",
        "with open(\"dev.\"+source_language, \"w\") as src_file, open(\"dev.\"+target_language, \"w\") as trg_file:\n",
        "  for index, row in dev.iterrows():\n",
        "    src_file.write(row[\"source_sentence\"]+\"\\n\")\n",
        "    trg_file.write(row[\"target_sentence\"]+\"\\n\")\n",
        "\n",
        "#stripped[[\"source_sentence\"]].to_csv(\"train.\"+source_language, header=False, index=False)  # Herman: Added `header=False` everywhere\n",
        "#stripped[[\"target_sentence\"]].to_csv(\"train.\"+target_language, header=False, index=False)  # Julia: Problematic handling of quotation marks.\n",
        "\n",
        "#dev[[\"source_sentence\"]].to_csv(\"dev.\"+source_language, header=False, index=False)\n",
        "#dev[[\"target_sentence\"]].to_csv(\"dev.\"+target_language, header=False, index=False)\n",
        "\n",
        "# Doublecheck the format below. There should be no extra quotation marks or weird characters.\n",
        "! head train.*\n",
        "! head dev.*"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==> train.bin <==\n",
            "12 : 11 ) Usun iwinna nọ bi ye ediọn izabọ ọre ne iran gha rhie igiọdu ne emwa ni ro iro fi uyinmwẹ werriẹ , ne iran ghẹ gha gbe ọsumẹ segbe .\n",
            "Iran na hannọ ikpia ihinrọn ni gbegba ne iran gha ru iwinna evbare ne a ghae vbe odẹ nọ khẹke .\n",
            "Esa i rrọọ ne ima yaro yọ wẹẹ Jesu ra Erhae gha ru emwi ọyunnuan vberriọ ne ima vbe ẹdẹnẹrẹ .\n",
            "Te ọ wa gha rre ima orhiọn ne ima ya ga vbe Abotu vbe igiogbẹ .\n",
            "3 U Gha Mwẹ Ẹbo Izinegbe Ra ?\n",
            "17 - 19 .\n",
            "Emwi ne I ruẹ re vbe Owebe Gilead keghi ya mwẹ sikẹ e Jehova sayọ .\n",
            "Asọn nii , ne a do Ugie Ayere , ẹre ọ wa khẹke ne ima gele muẹn roro emwi ne eka ne a ma gua kevbe ayọn nọ baa ne a loo vbe Ugie Ayere demu .\n",
            "Arriọba na ghi fan emwa ni rre iwe fua nẹ , nibun vbọ na gha die Kyrgyzstan . Eso vbọ ẹre ọ ghi ya iyẹn nọ maan sẹ otọ ẹvbo ima .\n",
            "De emwi ne u gha ru ne u mieke na gha mwẹ alaghodaro vbe odẹ ne u ya dia egbuẹ ?\n",
            "\n",
            "==> train.en <==\n",
            "Third , elders are commissioned and trained to reassure repentant sinners , helping to ease their pain and guilt .\n",
            "The apostles quickly rectified matters by appointing qualified men to handle the food distribution .\n",
            "We have no basis to expect that Jesus or his Father will now perform such a miracle for us .\n",
            "We enjoyed pioneering together .\n",
            "3 Are You Willing to Wait Patiently ?\n",
            "17 - 19 .\n",
            "The training at Gilead definitely helped me to improve my spirituality .\n",
            "Prior to that special night ​ — and especially on that night — ​ give serious thought to the significance of the unleavened bread and the red wine .\n",
            "When these exiles were released , many came to Kyrgyzstan , and some of them brought the truth with them .\n",
            "Consider this real - life situation .\n",
            "==> dev.bin <==\n",
            "De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "3 : 3 ) Ne ọmwa nọ ru orukhọ mieke na miẹn ayabọ , te ọ khẹke nọ gele fiwerriẹ .\n",
            "A sẹtin vbe miẹn wẹẹ , ọmwa ru emwi nọ sọnnọ ruẹ vbe ẹghẹ eso nọ gberra , sokpan ọ ma fo vbe ekhọe ruẹ .\n",
            "Ọna keghi re odẹ ọvbehe ne Erha mwa nọ mwẹ ahoẹmwọmwa ya ye Ẹmwẹ ọnrẹn rhie ifuẹko ne emwa ni rre ibavbaro .\n",
            "Ọyẹnmwẹ na do gha sẹ ima . ”\n",
            "Ẹmwẹ eso na ka loo ghi do lughaẹn ne vbene a te ya loo ẹre . Ughaghe urhuẹvbo ne u zẹ vbe fiwerriẹ nẹ .\n",
            "De emwa ni khian rhiọ kpaegbe vbe ẹghẹ urremwẹ ọghe Kristi ? Vbọ khian ya sunu hẹ ?\n",
            "\n",
            "==> dev.en <==\n",
            "Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "What are the rewards for being humble ?\n",
            "In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "A sinner must be repentant if he is to receive mercy .\n",
            "To improve relationships , even with enemies , the Bible recommends hospitality .\n",
            "This is another way that our loving Father , through his Word , offers support and comfort to those experiencing tribulation .\n",
            "What benefits come from being hospitable ?\n",
            "Perhaps you can think of examples of how a language that you speak has changed .\n",
            "What does the Bible indicate would happen regarding the resurrection during Christ’s presence ?\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "epeCydmCyS8X"
      },
      "source": [
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "## Installation of JoeyNMT\n",
        "\n",
        "JoeyNMT is a simple, minimalist NMT package which is useful for learning and teaching. Check out the documentation for JoeyNMT [here](https://joeynmt.readthedocs.io)  "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iBRMm4kMxZ8L",
        "outputId": "c0200130-b6a3-403a-d87b-f141524d7af0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "# Install JoeyNMT\n",
        "! git clone https://github.com/joeynmt/joeynmt.git\n",
        "! cd joeynmt; pip3 install ."
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'joeynmt'...\n",
            "remote: Enumerating objects: 61, done.\u001b[K\n",
            "remote: Counting objects: 100% (61/61), done.\u001b[K\n",
            "remote: Compressing objects: 100% (39/39), done.\u001b[K\n",
            "remote: Total 2245 (delta 34), reused 34 (delta 22), pack-reused 2184\u001b[K\n",
            "Receiving objects: 100% (2245/2245), 2.63 MiB | 4.97 MiB/s, done.\n",
            "Resolving deltas: 100% (1555/1555), done.\n",
            "Processing /content/joeynmt\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.16.0)\n",
            "Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (6.2.2)\n",
            "Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.17.5)\n",
            "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (42.0.2)\n",
            "Requirement already satisfied: torch>=1.1 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.3.1)\n",
            "Requirement already satisfied: tensorflow>=1.14 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.15.0)\n",
            "Requirement already satisfied: torchtext in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.3.1)\n",
            "Collecting sacrebleu>=1.3.6\n",
            "  Downloading https://files.pythonhosted.org/packages/45/31/1a135b964c169984b27fb2f7a50280fa7f8e6d9d404d8a9e596180487fd1/sacrebleu-1.4.3-py3-none-any.whl\n",
            "Collecting subword-nmt\n",
            "  Downloading https://files.pythonhosted.org/packages/74/60/6600a7bc09e7ab38bc53a48a20d8cae49b837f93f5842a41fe513a694912/subword_nmt-0.3.7-py2.py3-none-any.whl\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (3.1.2)\n",
            "Requirement already satisfied: seaborn in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.9.0)\n",
            "Collecting pyyaml>=5.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/3d/d9/ea9816aea31beeadccd03f1f8b625ecf8f645bd66744484d162d84803ce5/PyYAML-5.3.tar.gz (268kB)\n",
            "\u001b[K     |████████████████████████████████| 276kB 23.6MB/s \n",
            "\u001b[?25hCollecting pylint\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl (302kB)\n",
            "\u001b[K     |████████████████████████████████| 307kB 48.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: six==1.12 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.12.0)\n",
            "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.2.2)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.9.0)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.1.0)\n",
            "Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.1)\n",
            "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.33.6)\n",
            "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.10.0)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.8.1)\n",
            "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.0.8)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.11.2)\n",
            "Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n",
            "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.1.8)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (4.28.1)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (2.21.0)\n",
            "Requirement already satisfied: typing in /usr/local/lib/python3.6/dist-packages (from sacrebleu>=1.3.6->joeynmt==0.0.1) (3.6.6)\n",
            "Collecting portalocker\n",
            "  Downloading https://files.pythonhosted.org/packages/91/db/7bc703c0760df726839e0699b7f78a4d8217fdc9c7fcb1b51b39c5a22a4e/portalocker-1.5.2-py2.py3-none-any.whl\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.6.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (0.10.0)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.4.6)\n",
            "Requirement already satisfied: scipy>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (1.4.1)\n",
            "Requirement already satisfied: pandas>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (0.25.3)\n",
            "Collecting isort<5,>=4.2.5\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl (42kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 7.4MB/s \n",
            "\u001b[?25hCollecting astroid<2.4,>=2.3.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ad/ae/86734823047962e7b8c8529186a1ac4a7ca19aaf1aa0c7713c022ef593fd/astroid-2.3.3-py3-none-any.whl (205kB)\n",
            "\u001b[K     |████████████████████████████████| 215kB 53.3MB/s \n",
            "\u001b[?25hCollecting mccabe<0.7,>=0.6\n",
            "  Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl\n",
            "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow>=1.14->joeynmt==0.0.1) (2.8.0)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (3.1.1)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (0.16.0)\n",
            "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2.8)\n",
            "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2019.11.28)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (3.0.4)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.15.2->seaborn->joeynmt==0.0.1) (2018.9)\n",
            "Collecting typed-ast<1.5,>=1.4.0; implementation_name == \"cpython\" and python_version < \"3.8\"\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/90/ed/5459080d95eb87a02fe860d447197be63b6e2b5e9ff73c2b0a85622994f4/typed_ast-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (737kB)\n",
            "\u001b[K     |████████████████████████████████| 747kB 58.0MB/s \n",
            "\u001b[?25hCollecting lazy-object-proxy==1.4.*\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/0b/dd/b1e3407e9e6913cf178e506cd0dee818e58694d9a5cd1984e3f6a8b9a10f/lazy_object_proxy-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (55kB)\n",
            "\u001b[K     |████████████████████████████████| 61kB 9.2MB/s \n",
            "\u001b[?25hBuilding wheels for collected packages: joeynmt, pyyaml\n",
            "  Building wheel for joeynmt (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for joeynmt: filename=joeynmt-0.0.1-cp36-none-any.whl size=72136 sha256=3035711b84697c36039ebe81ee275b52e3211e1540a20026cc1a354a046d5387\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-djb2_2h_/wheels/db/01/db/751cc9f3e7f6faec127c43644ba250a3ea7ad200594aeda70a\n",
            "  Building wheel for pyyaml (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pyyaml: filename=PyYAML-5.3-cp36-cp36m-linux_x86_64.whl size=44229 sha256=60517bbbc7bf233c0b0a57e9f8f7e1be5fb7e95c178b7106dff1e132fc0387b3\n",
            "  Stored in directory: /root/.cache/pip/wheels/e4/76/4d/a95b8dd7b452b69e8ed4f68b69e1b55e12c9c9624dd962b191\n",
            "Successfully built joeynmt pyyaml\n",
            "Installing collected packages: portalocker, sacrebleu, subword-nmt, pyyaml, isort, typed-ast, lazy-object-proxy, astroid, mccabe, pylint, joeynmt\n",
            "  Found existing installation: PyYAML 3.13\n",
            "    Uninstalling PyYAML-3.13:\n",
            "      Successfully uninstalled PyYAML-3.13\n",
            "Successfully installed astroid-2.3.3 isort-4.3.21 joeynmt-0.0.1 lazy-object-proxy-1.4.3 mccabe-0.6.1 portalocker-1.5.2 pylint-2.4.4 pyyaml-5.3 sacrebleu-1.4.3 subword-nmt-0.3.7 typed-ast-1.4.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "AaE77Tcppex9"
      },
      "source": [
        "# Preprocessing the Data into Subword BPE Tokens\n",
        "\n",
        "- One of the most powerful improvements for agglutinative languages (a feature of most Bantu languages) is using BPE tokenization [ (Sennrich, 2015) ](https://arxiv.org/abs/1508.07909).\n",
        "\n",
        "- It was also shown that by optimizing the umber of BPE codes we significantly improve results for low-resourced languages [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021) [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)\n",
        "\n",
        "- Below we have the scripts for doing BPE tokenization of our data. We use 4000 tokens as recommended by [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021). You do not need to change anything. Simply running the below will be suitable. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "H-TyjtmXB1mL",
        "outputId": "bf1448b9-2b0a-45ee-d7a7-7a3d6a28e607",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 428
        }
      },
      "source": [
        "# One of the huge boosts in NMT performance was to use a different method of tokenizing. \n",
        "# Usually, NMT would tokenize by words. However, using a method called BPE gave amazing boosts to performance\n",
        "\n",
        "# Do subword NMT\n",
        "from os import path\n",
        "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n",
        "os.environ[\"tgt\"] = target_language\n",
        "\n",
        "# Learn BPEs on the training data.\n",
        "os.environ[\"data_path\"] = path.join(\"joeynmt\", \"data\", source_language + target_language) # Herman! \n",
        "! subword-nmt learn-joint-bpe-and-vocab --input train.$src train.$tgt -s 4000 -o bpe.codes.4000 --write-vocabulary vocab.$src vocab.$tgt\n",
        "\n",
        "# Apply BPE splits to the development and test data.\n",
        "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < train.$src > train.bpe.$src\n",
        "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < train.$tgt > train.bpe.$tgt\n",
        "\n",
        "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < dev.$src > dev.bpe.$src\n",
        "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < dev.$tgt > dev.bpe.$tgt\n",
        "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < test.$src > test.bpe.$src\n",
        "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt\n",
        "\n",
        "# Create directory, move everyone we care about to the correct location\n",
        "! mkdir -p $data_path\n",
        "! cp train.* $data_path\n",
        "! cp test.* $data_path\n",
        "! cp dev.* $data_path\n",
        "! cp bpe.codes.4000 $data_path\n",
        "! ls $data_path\n",
        "\n",
        "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n",
        "! cp train.* \"$gdrive_path\"\n",
        "! cp test.* \"$gdrive_path\"\n",
        "! cp dev.* \"$gdrive_path\"\n",
        "! cp bpe.codes.4000 \"$gdrive_path\"\n",
        "! ls \"$gdrive_path\"\n",
        "\n",
        "# Create that vocab using build_vocab\n",
        "! sudo chmod 777 joeynmt/scripts/build_vocab.py\n",
        "! joeynmt/scripts/build_vocab.py joeynmt/data/$src$tgt/train.bpe.$src joeynmt/data/$src$tgt/train.bpe.$tgt --output_path joeynmt/data/$src$tgt/vocab.txt\n",
        "\n",
        "# Some output\n",
        "! echo \"BPE Ẹ̀dó Sentences\"\n",
        "! tail -n 5 test.bpe.$tgt\n",
        "! echo \"Combined BPE Vocab\"\n",
        "! tail -n 10 joeynmt/data/$src$tgt/vocab.txt  # Herman\n",
        "! cp joeynmt/data/$src$tgt/vocab.txt \"$gdrive_path\""
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "bpe.codes.4000\tdev.bpe.en  test.bpe.bin  test.en-any.en  train.bpe.en\n",
            "dev.bin\t\tdev.en\t    test.bpe.en   train.bin\t  train.en\n",
            "dev.bpe.bin\ttest.bin    test.en\t  train.bpe.bin\n",
            "bpe.codes.4000\tdev.bpe.en  test.bpe.bin  test.en-any.en  train.bpe.en\n",
            "dev.bin\t\tdev.en\t    test.bpe.en   train.bin\t  train.en\n",
            "dev.bpe.bin\ttest.bin    test.en\t  train.bpe.bin\n",
            "BPE Ẹ̀dó Sentences\n",
            "I keghi do bẹghe ẹre wẹẹ , emwa wa danmwehọ ẹsẹ@@ sẹ@@ mw@@ ẹse vbe iran gha tobọ iran bẹghe ẹre wẹẹ , e Baibol ẹre u mobọ loo kevbe wẹẹ , ekhọe hia ẹre u ya hoo ne u ru iyobọ ne iran . ”\n",
            "U@@ mozo ọghe orhiọn ( Ghee okhuẹn 19 - 20 )\n",
            "Vbe ẹtin e Jehova , ma gha sẹtin khọnmiotọ yan ogh@@ ian ima !\n",
            "( b ) De emwi ne ima khian ziro yan vbe ako iruẹmwi nọ ghi lele ọna ?\n",
            "( Tie Hibru 11 : 24 - 27 . )\n",
            "Combined BPE Vocab\n",
            "satisfac@@\n",
            "gy@@\n",
            "ọrhirhighay@@\n",
            "circu@@\n",
            "ọnnọ\n",
            "hunmw@@\n",
            "congrega@@\n",
            "vu@@\n",
            "regu@@\n",
            "Vbe@@\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "IlMitUHR8Qy-",
        "outputId": "b8efc6d0-eb4f-4308-8b48-4e5d75b4d356",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n",
        "! cp train.* \"$gdrive_path\"\n",
        "! cp test.* \"$gdrive_path\"\n",
        "! cp dev.* \"$gdrive_path\"\n",
        "! cp bpe.codes.4000 \"$gdrive_path\"\n",
        "! ls \"$gdrive_path\""
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "bpe.codes.4000\tdev.bpe.en  test.bpe.bin  test.en-any.en  train.bpe.en\n",
            "dev.bin\t\tdev.en\t    test.bpe.en   train.bin\t  train.en\n",
            "dev.bpe.bin\ttest.bin    test.en\t  train.bpe.bin   vocab.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Ixmzi60WsUZ8"
      },
      "source": [
        "# Creating the JoeyNMT Config\n",
        "\n",
        "JoeyNMT requires a yaml config. We provide a template below. We've also set a number of defaults with it, that you may play with!\n",
        "\n",
        "- We used Transformer architecture \n",
        "- We set our dropout to reasonably high: 0.3 (recommended in  [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021))\n",
        "\n",
        "Things worth playing with:\n",
        "- The batch size (also recommended to change for low-resourced languages)\n",
        "- The number of epochs (we've set it at 30 just so it runs in about an hour, for testing purposes)\n",
        "- The decoder options (beam_size, alpha)\n",
        "- Evaluation metrics (BLEU versus Crhf4)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "PIs1lY2hxMsl",
        "colab": {}
      },
      "source": [
        "# This creates the config file for our JoeyNMT system. It might seem overwhelming so we've provided a couple of useful parameters you'll need to update\n",
        "# (You can of course play with all the parameters if you'd like!)\n",
        "\n",
        "name = '%s%s' % (source_language, target_language)\n",
        "gdrive_path = os.environ[\"gdrive_path\"]\n",
        "\n",
        "# Create the config\n",
        "config = \"\"\"\n",
        "name: \"{name}_transformer\"\n",
        "\n",
        "data:\n",
        "    src: \"{source_language}\"\n",
        "    trg: \"{target_language}\"\n",
        "    train: \"{gdrive_path}/train.bpe\"\n",
        "    dev:   \"{gdrive_path}/dev.bpe\"\n",
        "    test:  \"{gdrive_path}/test.bpe\"\n",
        "    level: \"bpe\"\n",
        "    lowercase: False\n",
        "    max_sent_length: 100\n",
        "    src_vocab: \"{gdrive_path}/vocab.txt\"\n",
        "    trg_vocab: \"{gdrive_path}/vocab.txt\"\n",
        "\n",
        "testing:\n",
        "    beam_size: 5\n",
        "    alpha: 1.0\n",
        "\n",
        "training:\n",
        "    # load_model: \"{gdrive_path}/models/{name}_transformer_orig/142000.ckpt\" # if uncommented, load a pre-trained model from this checkpoint\n",
        "    random_seed: 42\n",
        "    optimizer: \"adam\"\n",
        "    normalization: \"tokens\"\n",
        "    adam_betas: [0.9, 0.999] \n",
        "    scheduling: \"plateau\"           # TODO: try switching from plateau to Noam scheduling\n",
        "    patience: 5                     # For plateau: decrease learning rate by decrease_factor if validation score has not improved for this many validation rounds.\n",
        "    learning_rate_factor: 0.5       # factor for Noam scheduler (used with Transformer)\n",
        "    learning_rate_warmup: 1000      # warmup steps for Noam scheduler (used with Transformer)\n",
        "    decrease_factor: 0.7\n",
        "    loss: \"crossentropy\"\n",
        "    learning_rate: 0.0003\n",
        "    learning_rate_min: 0.00000001\n",
        "    weight_decay: 0.0\n",
        "    label_smoothing: 0.1\n",
        "    batch_size: 4096\n",
        "    batch_type: \"token\"\n",
        "    eval_batch_size: 3600\n",
        "    eval_batch_type: \"token\"\n",
        "    batch_multiplier: 1\n",
        "    early_stopping_metric: \"ppl\"\n",
        "    epochs: 400                     # TODO: Decrease for when playing around and checking of working. Around 30 is sufficient to check if its working at all\n",
        "    validation_freq: 100          # TODO: Set to at least once per epoch.\n",
        "    logging_freq: 100\n",
        "    eval_metric: \"bleu\"\n",
        "    model_dir: \"{gdrive_path}/models/{name}_transformer\"\n",
        "    overwrite: True               # TODO: Set to True if you want to overwrite possibly existing models. \n",
        "    shuffle: True\n",
        "    use_cuda: True\n",
        "    max_output_length: 100\n",
        "    print_valid_sents: [0, 1, 2, 3]\n",
        "    keep_last_ckpts: 3\n",
        "\n",
        "model:\n",
        "    initializer: \"xavier\"\n",
        "    bias_initializer: \"zeros\"\n",
        "    init_gain: 1.0\n",
        "    embed_initializer: \"xavier\"\n",
        "    embed_init_gain: 1.0\n",
        "    tied_embeddings: True\n",
        "    tied_softmax: True\n",
        "    encoder:\n",
        "        type: \"transformer\"\n",
        "        num_layers: 6\n",
        "        num_heads: 4             # TODO: Increase to 8 for larger data.\n",
        "        embeddings:\n",
        "            embedding_dim: 256   # TODO: Increase to 512 for larger data.\n",
        "            scale: True\n",
        "            dropout: 0.2\n",
        "        # typically ff_size = 4 x hidden_size\n",
        "        hidden_size: 256         # TODO: Increase to 512 for larger data.\n",
        "        ff_size: 1024            # TODO: Increase to 2048 for larger data.\n",
        "        dropout: 0.3\n",
        "    decoder:\n",
        "        type: \"transformer\"\n",
        "        num_layers: 6\n",
        "        num_heads: 4              # TODO: Increase to 8 for larger data.\n",
        "        embeddings:\n",
        "            embedding_dim: 256    # TODO: Increase to 512 for larger data.\n",
        "            scale: True\n",
        "            dropout: 0.2\n",
        "        # typically ff_size = 4 x hidden_size\n",
        "        hidden_size: 256         # TODO: Increase to 512 for larger data.\n",
        "        ff_size: 1024            # TODO: Increase to 2048 for larger data.\n",
        "        dropout: 0.3\n",
        "\"\"\".format(name=name, gdrive_path=os.environ[\"gdrive_path\"], source_language=source_language, target_language=target_language)\n",
        "with open(\"joeynmt/configs/transformer_{name}.yaml\".format(name=name),'w') as f:\n",
        "    f.write(config)\n",
        "\n",
        "! cp joeynmt/configs/transformer_$src$tgt.yaml \"$gdrive_path\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "pIifxE3Qzuvs"
      },
      "source": [
        "# Train the Model\n",
        "\n",
        "This single line of joeynmt runs the training using the config we made above"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "6ZBPFwT94WpI",
        "outputId": "072da8e6-646b-4887-e6cc-02672af443c3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "# Train the model\n",
        "# You can press Ctrl-C to stop. And then run the next cell to save your checkpoints! \n",
        "# !cd joeynmt; python3 -m joeynmt train configs/transformer_$src$tgt.yaml\n",
        "!python3 -m joeynmt train \"$gdrive_path/transformer_$src$tgt.yaml\""
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2020-01-26 00:38:10,917 Hello! This is Joey-NMT.\n",
            "2020-01-26 00:38:12,214 Total params: 12093440\n",
            "2020-01-26 00:38:12,216 Trainable parameters: ['decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.dec_layer_norm.bias', 'decoder.layers.0.dec_layer_norm.weight', 'decoder.layers.0.feed_forward.layer_norm.bias', 'decoder.layers.0.feed_forward.layer_norm.weight', 'decoder.layers.0.feed_forward.pwff_layer.0.bias', 'decoder.layers.0.feed_forward.pwff_layer.0.weight', 'decoder.layers.0.feed_forward.pwff_layer.3.bias', 'decoder.layers.0.feed_forward.pwff_layer.3.weight', 'decoder.layers.0.src_trg_att.k_layer.bias', 'decoder.layers.0.src_trg_att.k_layer.weight', 'decoder.layers.0.src_trg_att.output_layer.bias', 'decoder.layers.0.src_trg_att.output_layer.weight', 'decoder.layers.0.src_trg_att.q_layer.bias', 'decoder.layers.0.src_trg_att.q_layer.weight', 'decoder.layers.0.src_trg_att.v_layer.bias', 'decoder.layers.0.src_trg_att.v_layer.weight', 'decoder.layers.0.trg_trg_att.k_layer.bias', 'decoder.layers.0.trg_trg_att.k_layer.weight', 'decoder.layers.0.trg_trg_att.output_layer.bias', 'decoder.layers.0.trg_trg_att.output_layer.weight', 'decoder.layers.0.trg_trg_att.q_layer.bias', 'decoder.layers.0.trg_trg_att.q_layer.weight', 'decoder.layers.0.trg_trg_att.v_layer.bias', 'decoder.layers.0.trg_trg_att.v_layer.weight', 'decoder.layers.0.x_layer_norm.bias', 'decoder.layers.0.x_layer_norm.weight', 'decoder.layers.1.dec_layer_norm.bias', 'decoder.layers.1.dec_layer_norm.weight', 'decoder.layers.1.feed_forward.layer_norm.bias', 'decoder.layers.1.feed_forward.layer_norm.weight', 'decoder.layers.1.feed_forward.pwff_layer.0.bias', 'decoder.layers.1.feed_forward.pwff_layer.0.weight', 'decoder.layers.1.feed_forward.pwff_layer.3.bias', 'decoder.layers.1.feed_forward.pwff_layer.3.weight', 'decoder.layers.1.src_trg_att.k_layer.bias', 'decoder.layers.1.src_trg_att.k_layer.weight', 'decoder.layers.1.src_trg_att.output_layer.bias', 'decoder.layers.1.src_trg_att.output_layer.weight', 'decoder.layers.1.src_trg_att.q_layer.bias', 'decoder.layers.1.src_trg_att.q_layer.weight', 'decoder.layers.1.src_trg_att.v_layer.bias', 'decoder.layers.1.src_trg_att.v_layer.weight', 'decoder.layers.1.trg_trg_att.k_layer.bias', 'decoder.layers.1.trg_trg_att.k_layer.weight', 'decoder.layers.1.trg_trg_att.output_layer.bias', 'decoder.layers.1.trg_trg_att.output_layer.weight', 'decoder.layers.1.trg_trg_att.q_layer.bias', 'decoder.layers.1.trg_trg_att.q_layer.weight', 'decoder.layers.1.trg_trg_att.v_layer.bias', 'decoder.layers.1.trg_trg_att.v_layer.weight', 'decoder.layers.1.x_layer_norm.bias', 'decoder.layers.1.x_layer_norm.weight', 'decoder.layers.2.dec_layer_norm.bias', 'decoder.layers.2.dec_layer_norm.weight', 'decoder.layers.2.feed_forward.layer_norm.bias', 'decoder.layers.2.feed_forward.layer_norm.weight', 'decoder.layers.2.feed_forward.pwff_layer.0.bias', 'decoder.layers.2.feed_forward.pwff_layer.0.weight', 'decoder.layers.2.feed_forward.pwff_layer.3.bias', 'decoder.layers.2.feed_forward.pwff_layer.3.weight', 'decoder.layers.2.src_trg_att.k_layer.bias', 'decoder.layers.2.src_trg_att.k_layer.weight', 'decoder.layers.2.src_trg_att.output_layer.bias', 'decoder.layers.2.src_trg_att.output_layer.weight', 'decoder.layers.2.src_trg_att.q_layer.bias', 'decoder.layers.2.src_trg_att.q_layer.weight', 'decoder.layers.2.src_trg_att.v_layer.bias', 'decoder.layers.2.src_trg_att.v_layer.weight', 'decoder.layers.2.trg_trg_att.k_layer.bias', 'decoder.layers.2.trg_trg_att.k_layer.weight', 'decoder.layers.2.trg_trg_att.output_layer.bias', 'decoder.layers.2.trg_trg_att.output_layer.weight', 'decoder.layers.2.trg_trg_att.q_layer.bias', 'decoder.layers.2.trg_trg_att.q_layer.weight', 'decoder.layers.2.trg_trg_att.v_layer.bias', 'decoder.layers.2.trg_trg_att.v_layer.weight', 'decoder.layers.2.x_layer_norm.bias', 'decoder.layers.2.x_layer_norm.weight', 'decoder.layers.3.dec_layer_norm.bias', 'decoder.layers.3.dec_layer_norm.weight', 'decoder.layers.3.feed_forward.layer_norm.bias', 'decoder.layers.3.feed_forward.layer_norm.weight', 'decoder.layers.3.feed_forward.pwff_layer.0.bias', 'decoder.layers.3.feed_forward.pwff_layer.0.weight', 'decoder.layers.3.feed_forward.pwff_layer.3.bias', 'decoder.layers.3.feed_forward.pwff_layer.3.weight', 'decoder.layers.3.src_trg_att.k_layer.bias', 'decoder.layers.3.src_trg_att.k_layer.weight', 'decoder.layers.3.src_trg_att.output_layer.bias', 'decoder.layers.3.src_trg_att.output_layer.weight', 'decoder.layers.3.src_trg_att.q_layer.bias', 'decoder.layers.3.src_trg_att.q_layer.weight', 'decoder.layers.3.src_trg_att.v_layer.bias', 'decoder.layers.3.src_trg_att.v_layer.weight', 'decoder.layers.3.trg_trg_att.k_layer.bias', 'decoder.layers.3.trg_trg_att.k_layer.weight', 'decoder.layers.3.trg_trg_att.output_layer.bias', 'decoder.layers.3.trg_trg_att.output_layer.weight', 'decoder.layers.3.trg_trg_att.q_layer.bias', 'decoder.layers.3.trg_trg_att.q_layer.weight', 'decoder.layers.3.trg_trg_att.v_layer.bias', 'decoder.layers.3.trg_trg_att.v_layer.weight', 'decoder.layers.3.x_layer_norm.bias', 'decoder.layers.3.x_layer_norm.weight', 'decoder.layers.4.dec_layer_norm.bias', 'decoder.layers.4.dec_layer_norm.weight', 'decoder.layers.4.feed_forward.layer_norm.bias', 'decoder.layers.4.feed_forward.layer_norm.weight', 'decoder.layers.4.feed_forward.pwff_layer.0.bias', 'decoder.layers.4.feed_forward.pwff_layer.0.weight', 'decoder.layers.4.feed_forward.pwff_layer.3.bias', 'decoder.layers.4.feed_forward.pwff_layer.3.weight', 'decoder.layers.4.src_trg_att.k_layer.bias', 'decoder.layers.4.src_trg_att.k_layer.weight', 'decoder.layers.4.src_trg_att.output_layer.bias', 'decoder.layers.4.src_trg_att.output_layer.weight', 'decoder.layers.4.src_trg_att.q_layer.bias', 'decoder.layers.4.src_trg_att.q_layer.weight', 'decoder.layers.4.src_trg_att.v_layer.bias', 'decoder.layers.4.src_trg_att.v_layer.weight', 'decoder.layers.4.trg_trg_att.k_layer.bias', 'decoder.layers.4.trg_trg_att.k_layer.weight', 'decoder.layers.4.trg_trg_att.output_layer.bias', 'decoder.layers.4.trg_trg_att.output_layer.weight', 'decoder.layers.4.trg_trg_att.q_layer.bias', 'decoder.layers.4.trg_trg_att.q_layer.weight', 'decoder.layers.4.trg_trg_att.v_layer.bias', 'decoder.layers.4.trg_trg_att.v_layer.weight', 'decoder.layers.4.x_layer_norm.bias', 'decoder.layers.4.x_layer_norm.weight', 'decoder.layers.5.dec_layer_norm.bias', 'decoder.layers.5.dec_layer_norm.weight', 'decoder.layers.5.feed_forward.layer_norm.bias', 'decoder.layers.5.feed_forward.layer_norm.weight', 'decoder.layers.5.feed_forward.pwff_layer.0.bias', 'decoder.layers.5.feed_forward.pwff_layer.0.weight', 'decoder.layers.5.feed_forward.pwff_layer.3.bias', 'decoder.layers.5.feed_forward.pwff_layer.3.weight', 'decoder.layers.5.src_trg_att.k_layer.bias', 'decoder.layers.5.src_trg_att.k_layer.weight', 'decoder.layers.5.src_trg_att.output_layer.bias', 'decoder.layers.5.src_trg_att.output_layer.weight', 'decoder.layers.5.src_trg_att.q_layer.bias', 'decoder.layers.5.src_trg_att.q_layer.weight', 'decoder.layers.5.src_trg_att.v_layer.bias', 'decoder.layers.5.src_trg_att.v_layer.weight', 'decoder.layers.5.trg_trg_att.k_layer.bias', 'decoder.layers.5.trg_trg_att.k_layer.weight', 'decoder.layers.5.trg_trg_att.output_layer.bias', 'decoder.layers.5.trg_trg_att.output_layer.weight', 'decoder.layers.5.trg_trg_att.q_layer.bias', 'decoder.layers.5.trg_trg_att.q_layer.weight', 'decoder.layers.5.trg_trg_att.v_layer.bias', 'decoder.layers.5.trg_trg_att.v_layer.weight', 'decoder.layers.5.x_layer_norm.bias', 'decoder.layers.5.x_layer_norm.weight', 'encoder.layer_norm.bias', 'encoder.layer_norm.weight', 'encoder.layers.0.feed_forward.layer_norm.bias', 'encoder.layers.0.feed_forward.layer_norm.weight', 'encoder.layers.0.feed_forward.pwff_layer.0.bias', 'encoder.layers.0.feed_forward.pwff_layer.0.weight', 'encoder.layers.0.feed_forward.pwff_layer.3.bias', 'encoder.layers.0.feed_forward.pwff_layer.3.weight', 'encoder.layers.0.layer_norm.bias', 'encoder.layers.0.layer_norm.weight', 'encoder.layers.0.src_src_att.k_layer.bias', 'encoder.layers.0.src_src_att.k_layer.weight', 'encoder.layers.0.src_src_att.output_layer.bias', 'encoder.layers.0.src_src_att.output_layer.weight', 'encoder.layers.0.src_src_att.q_layer.bias', 'encoder.layers.0.src_src_att.q_layer.weight', 'encoder.layers.0.src_src_att.v_layer.bias', 'encoder.layers.0.src_src_att.v_layer.weight', 'encoder.layers.1.feed_forward.layer_norm.bias', 'encoder.layers.1.feed_forward.layer_norm.weight', 'encoder.layers.1.feed_forward.pwff_layer.0.bias', 'encoder.layers.1.feed_forward.pwff_layer.0.weight', 'encoder.layers.1.feed_forward.pwff_layer.3.bias', 'encoder.layers.1.feed_forward.pwff_layer.3.weight', 'encoder.layers.1.layer_norm.bias', 'encoder.layers.1.layer_norm.weight', 'encoder.layers.1.src_src_att.k_layer.bias', 'encoder.layers.1.src_src_att.k_layer.weight', 'encoder.layers.1.src_src_att.output_layer.bias', 'encoder.layers.1.src_src_att.output_layer.weight', 'encoder.layers.1.src_src_att.q_layer.bias', 'encoder.layers.1.src_src_att.q_layer.weight', 'encoder.layers.1.src_src_att.v_layer.bias', 'encoder.layers.1.src_src_att.v_layer.weight', 'encoder.layers.2.feed_forward.layer_norm.bias', 'encoder.layers.2.feed_forward.layer_norm.weight', 'encoder.layers.2.feed_forward.pwff_layer.0.bias', 'encoder.layers.2.feed_forward.pwff_layer.0.weight', 'encoder.layers.2.feed_forward.pwff_layer.3.bias', 'encoder.layers.2.feed_forward.pwff_layer.3.weight', 'encoder.layers.2.layer_norm.bias', 'encoder.layers.2.layer_norm.weight', 'encoder.layers.2.src_src_att.k_layer.bias', 'encoder.layers.2.src_src_att.k_layer.weight', 'encoder.layers.2.src_src_att.output_layer.bias', 'encoder.layers.2.src_src_att.output_layer.weight', 'encoder.layers.2.src_src_att.q_layer.bias', 'encoder.layers.2.src_src_att.q_layer.weight', 'encoder.layers.2.src_src_att.v_layer.bias', 'encoder.layers.2.src_src_att.v_layer.weight', 'encoder.layers.3.feed_forward.layer_norm.bias', 'encoder.layers.3.feed_forward.layer_norm.weight', 'encoder.layers.3.feed_forward.pwff_layer.0.bias', 'encoder.layers.3.feed_forward.pwff_layer.0.weight', 'encoder.layers.3.feed_forward.pwff_layer.3.bias', 'encoder.layers.3.feed_forward.pwff_layer.3.weight', 'encoder.layers.3.layer_norm.bias', 'encoder.layers.3.layer_norm.weight', 'encoder.layers.3.src_src_att.k_layer.bias', 'encoder.layers.3.src_src_att.k_layer.weight', 'encoder.layers.3.src_src_att.output_layer.bias', 'encoder.layers.3.src_src_att.output_layer.weight', 'encoder.layers.3.src_src_att.q_layer.bias', 'encoder.layers.3.src_src_att.q_layer.weight', 'encoder.layers.3.src_src_att.v_layer.bias', 'encoder.layers.3.src_src_att.v_layer.weight', 'encoder.layers.4.feed_forward.layer_norm.bias', 'encoder.layers.4.feed_forward.layer_norm.weight', 'encoder.layers.4.feed_forward.pwff_layer.0.bias', 'encoder.layers.4.feed_forward.pwff_layer.0.weight', 'encoder.layers.4.feed_forward.pwff_layer.3.bias', 'encoder.layers.4.feed_forward.pwff_layer.3.weight', 'encoder.layers.4.layer_norm.bias', 'encoder.layers.4.layer_norm.weight', 'encoder.layers.4.src_src_att.k_layer.bias', 'encoder.layers.4.src_src_att.k_layer.weight', 'encoder.layers.4.src_src_att.output_layer.bias', 'encoder.layers.4.src_src_att.output_layer.weight', 'encoder.layers.4.src_src_att.q_layer.bias', 'encoder.layers.4.src_src_att.q_layer.weight', 'encoder.layers.4.src_src_att.v_layer.bias', 'encoder.layers.4.src_src_att.v_layer.weight', 'encoder.layers.5.feed_forward.layer_norm.bias', 'encoder.layers.5.feed_forward.layer_norm.weight', 'encoder.layers.5.feed_forward.pwff_layer.0.bias', 'encoder.layers.5.feed_forward.pwff_layer.0.weight', 'encoder.layers.5.feed_forward.pwff_layer.3.bias', 'encoder.layers.5.feed_forward.pwff_layer.3.weight', 'encoder.layers.5.layer_norm.bias', 'encoder.layers.5.layer_norm.weight', 'encoder.layers.5.src_src_att.k_layer.bias', 'encoder.layers.5.src_src_att.k_layer.weight', 'encoder.layers.5.src_src_att.output_layer.bias', 'encoder.layers.5.src_src_att.output_layer.weight', 'encoder.layers.5.src_src_att.q_layer.bias', 'encoder.layers.5.src_src_att.q_layer.weight', 'encoder.layers.5.src_src_att.v_layer.bias', 'encoder.layers.5.src_src_att.v_layer.weight', 'src_embed.lut.weight']\n",
            "2020-01-26 00:38:16,262 cfg.name                           : enbin_transformer\n",
            "2020-01-26 00:38:16,263 cfg.data.src                       : en\n",
            "2020-01-26 00:38:16,263 cfg.data.trg                       : bin\n",
            "2020-01-26 00:38:16,263 cfg.data.train                     : /content/drive/My Drive/masakhane/en-bin-baseline/train.bpe\n",
            "2020-01-26 00:38:16,263 cfg.data.dev                       : /content/drive/My Drive/masakhane/en-bin-baseline/dev.bpe\n",
            "2020-01-26 00:38:16,263 cfg.data.test                      : /content/drive/My Drive/masakhane/en-bin-baseline/test.bpe\n",
            "2020-01-26 00:38:16,264 cfg.data.level                     : bpe\n",
            "2020-01-26 00:38:16,264 cfg.data.lowercase                 : False\n",
            "2020-01-26 00:38:16,264 cfg.data.max_sent_length           : 100\n",
            "2020-01-26 00:38:16,264 cfg.data.src_vocab                 : /content/drive/My Drive/masakhane/en-bin-baseline/vocab.txt\n",
            "2020-01-26 00:38:16,264 cfg.data.trg_vocab                 : /content/drive/My Drive/masakhane/en-bin-baseline/vocab.txt\n",
            "2020-01-26 00:38:16,264 cfg.testing.beam_size              : 5\n",
            "2020-01-26 00:38:16,264 cfg.testing.alpha                  : 1.0\n",
            "2020-01-26 00:38:16,265 cfg.training.random_seed           : 42\n",
            "2020-01-26 00:38:16,265 cfg.training.optimizer             : adam\n",
            "2020-01-26 00:38:16,265 cfg.training.normalization         : tokens\n",
            "2020-01-26 00:38:16,265 cfg.training.adam_betas            : [0.9, 0.999]\n",
            "2020-01-26 00:38:16,265 cfg.training.scheduling            : plateau\n",
            "2020-01-26 00:38:16,265 cfg.training.patience              : 5\n",
            "2020-01-26 00:38:16,265 cfg.training.learning_rate_factor  : 0.5\n",
            "2020-01-26 00:38:16,266 cfg.training.learning_rate_warmup  : 1000\n",
            "2020-01-26 00:38:16,266 cfg.training.decrease_factor       : 0.7\n",
            "2020-01-26 00:38:16,266 cfg.training.loss                  : crossentropy\n",
            "2020-01-26 00:38:16,266 cfg.training.learning_rate         : 0.0003\n",
            "2020-01-26 00:38:16,266 cfg.training.learning_rate_min     : 1e-08\n",
            "2020-01-26 00:38:16,267 cfg.training.weight_decay          : 0.0\n",
            "2020-01-26 00:38:16,267 cfg.training.label_smoothing       : 0.1\n",
            "2020-01-26 00:38:16,267 cfg.training.batch_size            : 4096\n",
            "2020-01-26 00:38:16,267 cfg.training.batch_type            : token\n",
            "2020-01-26 00:38:16,267 cfg.training.eval_batch_size       : 3600\n",
            "2020-01-26 00:38:16,267 cfg.training.eval_batch_type       : token\n",
            "2020-01-26 00:38:16,268 cfg.training.batch_multiplier      : 1\n",
            "2020-01-26 00:38:16,268 cfg.training.early_stopping_metric : ppl\n",
            "2020-01-26 00:38:16,268 cfg.training.epochs                : 400\n",
            "2020-01-26 00:38:16,268 cfg.training.validation_freq       : 100\n",
            "2020-01-26 00:38:16,268 cfg.training.logging_freq          : 100\n",
            "2020-01-26 00:38:16,268 cfg.training.eval_metric           : bleu\n",
            "2020-01-26 00:38:16,268 cfg.training.model_dir             : /content/drive/My Drive/masakhane/en-bin-baseline/models/enbin_transformer\n",
            "2020-01-26 00:38:16,269 cfg.training.overwrite             : True\n",
            "2020-01-26 00:38:16,269 cfg.training.shuffle               : True\n",
            "2020-01-26 00:38:16,269 cfg.training.use_cuda              : True\n",
            "2020-01-26 00:38:16,269 cfg.training.max_output_length     : 100\n",
            "2020-01-26 00:38:16,269 cfg.training.print_valid_sents     : [0, 1, 2, 3]\n",
            "2020-01-26 00:38:16,269 cfg.training.keep_last_ckpts       : 3\n",
            "2020-01-26 00:38:16,269 cfg.model.initializer              : xavier\n",
            "2020-01-26 00:38:16,270 cfg.model.bias_initializer         : zeros\n",
            "2020-01-26 00:38:16,270 cfg.model.init_gain                : 1.0\n",
            "2020-01-26 00:38:16,270 cfg.model.embed_initializer        : xavier\n",
            "2020-01-26 00:38:16,270 cfg.model.embed_init_gain          : 1.0\n",
            "2020-01-26 00:38:16,270 cfg.model.tied_embeddings          : True\n",
            "2020-01-26 00:38:16,270 cfg.model.tied_softmax             : True\n",
            "2020-01-26 00:38:16,270 cfg.model.encoder.type             : transformer\n",
            "2020-01-26 00:38:16,271 cfg.model.encoder.num_layers       : 6\n",
            "2020-01-26 00:38:16,271 cfg.model.encoder.num_heads        : 4\n",
            "2020-01-26 00:38:16,271 cfg.model.encoder.embeddings.embedding_dim : 256\n",
            "2020-01-26 00:38:16,271 cfg.model.encoder.embeddings.scale : True\n",
            "2020-01-26 00:38:16,271 cfg.model.encoder.embeddings.dropout : 0.2\n",
            "2020-01-26 00:38:16,271 cfg.model.encoder.hidden_size      : 256\n",
            "2020-01-26 00:38:16,272 cfg.model.encoder.ff_size          : 1024\n",
            "2020-01-26 00:38:16,272 cfg.model.encoder.dropout          : 0.3\n",
            "2020-01-26 00:38:16,272 cfg.model.decoder.type             : transformer\n",
            "2020-01-26 00:38:16,272 cfg.model.decoder.num_layers       : 6\n",
            "2020-01-26 00:38:16,272 cfg.model.decoder.num_heads        : 4\n",
            "2020-01-26 00:38:16,272 cfg.model.decoder.embeddings.embedding_dim : 256\n",
            "2020-01-26 00:38:16,273 cfg.model.decoder.embeddings.scale : True\n",
            "2020-01-26 00:38:16,273 cfg.model.decoder.embeddings.dropout : 0.2\n",
            "2020-01-26 00:38:16,273 cfg.model.decoder.hidden_size      : 256\n",
            "2020-01-26 00:38:16,273 cfg.model.decoder.ff_size          : 1024\n",
            "2020-01-26 00:38:16,273 cfg.model.decoder.dropout          : 0.3\n",
            "2020-01-26 00:38:16,273 Data set sizes: \n",
            "\ttrain 10185,\n",
            "\tvalid 1000,\n",
            "\ttest 847\n",
            "2020-01-26 00:38:16,274 First training example:\n",
            "\t[SRC] T@@ hi@@ r@@ d , elders are com@@ mission@@ ed and tra@@ ined to reas@@ sure repent@@ ant sin@@ n@@ ers , helping to ea@@ se their pain and guil@@ t .\n",
            "\t[TRG] 12 : 11 ) U@@ sun iwinna nọ bi ye ediọn izabọ ọre ne iran gha rhie igiọdu ne emwa ni ro iro fi uyinmwẹ werriẹ , ne iran ghẹ gha gbe ọsu@@ mẹ s@@ egbe .\n",
            "2020-01-26 00:38:16,274 First 10 words (src): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) . (5) , (6) ne (7) the (8) vbe (9) gha\n",
            "2020-01-26 00:38:16,275 First 10 words (trg): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) . (5) , (6) ne (7) the (8) vbe (9) gha\n",
            "2020-01-26 00:38:16,275 Number of Src words (types): 4036\n",
            "2020-01-26 00:38:16,275 Number of Trg words (types): 4036\n",
            "2020-01-26 00:38:16,276 Model(\n",
            "\tencoder=TransformerEncoder(num_layers=6, num_heads=4),\n",
            "\tdecoder=TransformerDecoder(num_layers=6, num_heads=4),\n",
            "\tsrc_embed=Embeddings(embedding_dim=256, vocab_size=4036),\n",
            "\ttrg_embed=Embeddings(embedding_dim=256, vocab_size=4036))\n",
            "2020-01-26 00:38:16,283 EPOCH 1\n",
            "2020-01-26 00:38:30,646 Epoch   1 Step:      100 Batch Loss:     4.821545 Tokens per Sec:    13378, Lr: 0.000300\n",
            "2020-01-26 00:39:13,532 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:39:13,532 Saving new checkpoint.\n",
            "2020-01-26 00:39:14,612 Example #0\n",
            "2020-01-26 00:39:14,613 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:39:14,613 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:39:14,613 \tHypothesis: ( : ne ne .\n",
            "2020-01-26 00:39:14,614 Example #1\n",
            "2020-01-26 00:39:14,614 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:39:14,614 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:39:14,614 \tHypothesis: ( : ne ne .\n",
            "2020-01-26 00:39:14,614 Example #2\n",
            "2020-01-26 00:39:14,615 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:39:14,615 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:39:14,615 \tHypothesis: ( , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , .\n",
            "2020-01-26 00:39:14,615 Example #3\n",
            "2020-01-26 00:39:14,616 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:39:14,616 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:39:14,616 \tHypothesis: ( , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , .\n",
            "2020-01-26 00:39:14,616 Validation result (greedy) at epoch   1, step      100: bleu:   0.00, loss: 125476.3672, ppl: 141.4720, duration: 43.9696s\n",
            "2020-01-26 00:39:19,533 Epoch   1: total training loss 699.00\n",
            "2020-01-26 00:39:19,534 EPOCH 2\n",
            "2020-01-26 00:39:29,632 Epoch   2 Step:      200 Batch Loss:     4.880541 Tokens per Sec:    12819, Lr: 0.000300\n",
            "2020-01-26 00:40:15,416 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:40:15,417 Saving new checkpoint.\n",
            "2020-01-26 00:40:16,499 Example #0\n",
            "2020-01-26 00:40:16,500 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:40:16,500 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:40:16,500 \tHypothesis: ( De : : : : : : De : De : De ne : De ne : De ne : De ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne\n",
            "2020-01-26 00:40:16,500 Example #1\n",
            "2020-01-26 00:40:16,501 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:40:16,501 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:40:16,501 \tHypothesis: ( De : : : : De ) De : De : De ne : De ne : De ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima ne ima\n",
            "2020-01-26 00:40:16,501 Example #2\n",
            "2020-01-26 00:40:16,501 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:40:16,502 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:40:16,502 \tHypothesis: ( : : : : : : : : : : : ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( : ( : ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (\n",
            "2020-01-26 00:40:16,502 Example #3\n",
            "2020-01-26 00:40:16,502 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:40:16,502 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:40:16,502 \tHypothesis: ( : : : : : : : : : : : ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( : ( : ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (\n",
            "2020-01-26 00:40:16,502 Validation result (greedy) at epoch   2, step      200: bleu:   0.00, loss: 118648.9688, ppl: 108.0559, duration: 46.8699s\n",
            "2020-01-26 00:40:26,603 Epoch   2: total training loss 615.30\n",
            "2020-01-26 00:40:26,604 EPOCH 3\n",
            "2020-01-26 00:40:31,817 Epoch   3 Step:      300 Batch Loss:     4.325080 Tokens per Sec:    12578, Lr: 0.000300\n",
            "2020-01-26 00:41:17,258 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:41:17,258 Saving new checkpoint.\n",
            "2020-01-26 00:41:18,361 Example #0\n",
            "2020-01-26 00:41:18,362 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:41:18,362 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:41:18,363 \tHypothesis: De ne ima ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya\n",
            "2020-01-26 00:41:18,363 Example #1\n",
            "2020-01-26 00:41:18,363 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:41:18,363 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:41:18,363 \tHypothesis: De ne ima ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ya ?\n",
            "2020-01-26 00:41:18,364 Example #2\n",
            "2020-01-26 00:41:18,364 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:41:18,364 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:41:18,364 \tHypothesis: ( : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : ( : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : )\n",
            "2020-01-26 00:41:18,364 Example #3\n",
            "2020-01-26 00:41:18,365 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:41:18,365 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:41:18,365 \tHypothesis: ( : : : : ) ( ) ( ) ( : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : ( ( : : : : : : : : : ) ( ) ( ) ( ) ( ) ( ) ( ) ( : : : : : ) ( ) ( : : : ) ( ) ( : : : ) (\n",
            "2020-01-26 00:41:18,365 Validation result (greedy) at epoch   3, step      300: bleu:   0.00, loss: 109354.5391, ppl:  74.8758, duration: 46.5472s\n",
            "2020-01-26 00:41:33,453 Epoch   3 Step:      400 Batch Loss:     3.808825 Tokens per Sec:    12771, Lr: 0.000300\n",
            "2020-01-26 00:42:19,274 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:42:19,274 Saving new checkpoint.\n",
            "2020-01-26 00:42:20,526 Example #0\n",
            "2020-01-26 00:42:20,527 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:42:20,527 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:42:20,527 \tHypothesis: De vbene ima ya ya ya ya ya ya gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha\n",
            "2020-01-26 00:42:20,527 Example #1\n",
            "2020-01-26 00:42:20,527 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:42:20,528 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:42:20,528 \tHypothesis: De emwi ne ima ya ya ya ya ru iyobọ ne ima ya ya ru ?\n",
            "2020-01-26 00:42:20,528 Example #2\n",
            "2020-01-26 00:42:20,528 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:42:20,528 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:42:20,528 \tHypothesis: ( 1 : 1 : 1 : 1 : 1 ) Ọ keghi gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha\n",
            "2020-01-26 00:42:20,528 Example #3\n",
            "2020-01-26 00:42:20,529 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:42:20,529 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:42:20,529 \tHypothesis: ( 1 : 1 : 1 ) De emwi ne ima gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha gha\n",
            "2020-01-26 00:42:20,529 Validation result (greedy) at epoch   3, step      400: bleu:   0.20, loss: 101641.4453, ppl:  55.2254, duration: 47.0758s\n",
            "2020-01-26 00:42:20,535 Epoch   3: total training loss 573.61\n",
            "2020-01-26 00:42:20,535 EPOCH 4\n",
            "2020-01-26 00:42:35,633 Epoch   4 Step:      500 Batch Loss:     3.677055 Tokens per Sec:    12396, Lr: 0.000300\n",
            "2020-01-26 00:43:21,298 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:43:21,298 Saving new checkpoint.\n",
            "2020-01-26 00:43:22,843 Example #0\n",
            "2020-01-26 00:43:22,844 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:43:22,844 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:43:22,844 \tHypothesis: De vbene ima ya ya ya ya ya ya ima gha ru iyobọ ne ima ya ru iyobọ ne ima ya ru iyobọ ne ima ?\n",
            "2020-01-26 00:43:22,844 Example #1\n",
            "2020-01-26 00:43:22,845 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:43:22,845 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:43:22,845 \tHypothesis: De vbene ima ya ya ya ya ru iyobọ ne ima ya ru iyobọ ne ima ?\n",
            "2020-01-26 00:43:22,845 Example #2\n",
            "2020-01-26 00:43:22,846 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:43:22,846 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:43:22,846 \tHypothesis: Ọ keghi kha wẹẹ : “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ “ I na gha\n",
            "2020-01-26 00:43:22,847 Example #3\n",
            "2020-01-26 00:43:22,847 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:43:22,847 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:43:22,847 \tHypothesis: Vbe igiemwi , ọ khẹke ne ima ya ya gha ru iyobọ ne ima gha ru iyobọ ne ima gha ru iyobọ ne ima .\n",
            "2020-01-26 00:43:22,848 Validation result (greedy) at epoch   4, step      500: bleu:   0.79, loss: 95679.2969, ppl:  43.6463, duration: 47.2143s\n",
            "2020-01-26 00:43:28,146 Epoch   4: total training loss 525.21\n",
            "2020-01-26 00:43:28,147 EPOCH 5\n",
            "2020-01-26 00:43:38,387 Epoch   5 Step:      600 Batch Loss:     3.835121 Tokens per Sec:    12318, Lr: 0.000300\n",
            "2020-01-26 00:44:24,172 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:44:24,173 Saving new checkpoint.\n",
            "2020-01-26 00:44:25,248 Example #0\n",
            "2020-01-26 00:44:25,248 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:44:25,248 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:44:25,249 \tHypothesis: De vbene ima ya ya ya ima ya ima gha ru iyobọ ne ima ?\n",
            "2020-01-26 00:44:25,249 Example #1\n",
            "2020-01-26 00:44:25,250 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:44:25,250 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:44:25,250 \tHypothesis: De vbene ima ya ya ima ya ima gha ru iyobọ ne ima ?\n",
            "2020-01-26 00:44:25,250 Example #2\n",
            "2020-01-26 00:44:25,251 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:44:25,251 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:44:25,251 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ keghi kha wẹẹ : “ Ọ keghi kha wẹẹ : “ Ọ ghi mwẹ emwi ne a ya gha mwẹ .\n",
            "2020-01-26 00:44:25,251 Example #3\n",
            "2020-01-26 00:44:25,251 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:44:25,252 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:44:25,252 \tHypothesis: ( Mat 1 : 1 ) Vbe ima gha ya ima gha ru iyobọ ne ima , ma gha ru iyobọ ne ima .\n",
            "2020-01-26 00:44:25,252 Validation result (greedy) at epoch   5, step      600: bleu:   2.81, loss: 91839.1328, ppl:  37.5083, duration: 46.8646s\n",
            "2020-01-26 00:44:35,419 Epoch   5: total training loss 482.20\n",
            "2020-01-26 00:44:35,420 EPOCH 6\n",
            "2020-01-26 00:44:40,521 Epoch   6 Step:      700 Batch Loss:     3.797078 Tokens per Sec:    11938, Lr: 0.000300\n",
            "2020-01-26 00:45:26,235 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:45:26,235 Saving new checkpoint.\n",
            "2020-01-26 00:45:27,283 Example #0\n",
            "2020-01-26 00:45:27,283 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:45:27,284 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:45:27,284 \tHypothesis: De vbene ima ya ya ru iyobọ ne ima ya ru iyobọ ne ima ya ru iyobọ ne ima ya ru iyobọ ne ima ya ru iyobọ ne ima ya ru iyobọ ne ima ?\n",
            "2020-01-26 00:45:27,284 Example #1\n",
            "2020-01-26 00:45:27,284 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:45:27,284 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:45:27,284 \tHypothesis: De vbene ima ya ya ru iyobọ ne ima ya ru iyobọ ne ima ?\n",
            "2020-01-26 00:45:27,284 Example #2\n",
            "2020-01-26 00:45:27,284 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:45:27,285 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:45:27,285 \tHypothesis: Ọ keghi kha wẹẹ : “ Wa gha mwẹ emwi ne a ya ru iyobọ ne a ya ru iyobọ ne iran .\n",
            "2020-01-26 00:45:27,285 Example #3\n",
            "2020-01-26 00:45:27,285 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:45:27,285 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:45:27,285 \tHypothesis: ( 1 : 4 ) Ma gha ya ima gha ru iyobọ ne ima ya ima gha ru iyobọ ne ima .\n",
            "2020-01-26 00:45:27,285 Validation result (greedy) at epoch   6, step      700: bleu:   2.76, loss: 89375.1172, ppl:  34.0325, duration: 46.7640s\n",
            "2020-01-26 00:45:42,628 Epoch   6: total training loss 462.33\n",
            "2020-01-26 00:45:42,629 EPOCH 7\n",
            "2020-01-26 00:45:42,815 Epoch   7 Step:      800 Batch Loss:     3.511859 Tokens per Sec:    13027, Lr: 0.000300\n",
            "2020-01-26 00:46:28,556 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:46:28,556 Saving new checkpoint.\n",
            "2020-01-26 00:46:30,493 Example #0\n",
            "2020-01-26 00:46:30,494 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:46:30,495 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:46:30,495 \tHypothesis: De emwi ne ima ya gha ru iyobọ ne ima ?\n",
            "2020-01-26 00:46:30,495 Example #1\n",
            "2020-01-26 00:46:30,495 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:46:30,495 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:46:30,495 \tHypothesis: De emwi ne ima ya gha mwẹ alaghodaro ?\n",
            "2020-01-26 00:46:30,495 Example #2\n",
            "2020-01-26 00:46:30,496 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:46:30,496 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:46:30,496 \tHypothesis: Ọ keghi kha wẹẹ : “ Wa gha mwẹ emwi nọ maan , ọ na gha mwẹ emwi nọ maan , ọ na gha mwẹ emwi nọ maan .\n",
            "2020-01-26 00:46:30,496 Example #3\n",
            "2020-01-26 00:46:30,496 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:46:30,496 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:46:30,496 \tHypothesis: ( 1 Kọr 3 : 1 ) E Jehova keghi kha wẹẹ : “ Wa gha mwẹ emwi ne ima gha mwẹ ekhọe hia .\n",
            "2020-01-26 00:46:30,497 Validation result (greedy) at epoch   7, step      800: bleu:   2.64, loss: 86692.9219, ppl:  30.6140, duration: 47.6814s\n",
            "2020-01-26 00:46:45,912 Epoch   7 Step:      900 Batch Loss:     3.443221 Tokens per Sec:    12759, Lr: 0.000300\n",
            "2020-01-26 00:47:31,731 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:47:31,732 Saving new checkpoint.\n",
            "2020-01-26 00:47:32,888 Example #0\n",
            "2020-01-26 00:47:32,889 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:47:32,889 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:47:32,890 \tHypothesis: De vbene ima ya ya gha ya ima gha ya ima gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:47:32,890 Example #1\n",
            "2020-01-26 00:47:32,890 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:47:32,890 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:47:32,890 \tHypothesis: De vbene Jehova ya ya gha ya egbe tae Jehova hẹ ?\n",
            "2020-01-26 00:47:32,890 Example #2\n",
            "2020-01-26 00:47:32,891 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:47:32,891 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:47:32,891 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ghi sẹ ẹghẹ ne a ya gha mwẹ mwẹ ekhọe hia , ọ na vbe gha mwẹ alaghodaro vbe ẹghẹ ne a ya gha mwẹ mwẹ alaghodaro vbe ẹghẹ hia .\n",
            "2020-01-26 00:47:32,891 Example #3\n",
            "2020-01-26 00:47:32,891 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:47:32,891 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:47:32,891 \tHypothesis: ( Mat . 1 : 1 ) E Jehova keghi rhie igiọdu ne ima , ne ima gha ya ima gha ru iyobọ ne ima , ne ima gha ya ima gha ya ima gha ru iyobọ ne ima .\n",
            "2020-01-26 00:47:32,892 Validation result (greedy) at epoch   7, step      900: bleu:   2.72, loss: 84652.3750, ppl:  28.2453, duration: 46.9789s\n",
            "2020-01-26 00:47:37,640 Epoch   7: total training loss 449.12\n",
            "2020-01-26 00:47:37,641 EPOCH 8\n",
            "2020-01-26 00:47:48,270 Epoch   8 Step:     1000 Batch Loss:     3.156454 Tokens per Sec:    12340, Lr: 0.000300\n",
            "2020-01-26 00:48:33,834 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:48:33,835 Saving new checkpoint.\n",
            "2020-01-26 00:48:34,940 Example #0\n",
            "2020-01-26 00:48:34,940 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:48:34,940 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:48:34,941 \tHypothesis: De emwi ne ima gha ya sẹtin ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:48:34,941 Example #1\n",
            "2020-01-26 00:48:34,941 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:48:34,941 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:48:34,941 \tHypothesis: De emwi ne Jehova ya ru iyobọ ne emwa ni rre Baibol ?\n",
            "2020-01-26 00:48:34,941 Example #2\n",
            "2020-01-26 00:48:34,941 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:48:34,941 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:48:34,942 \tHypothesis: Ọ keghi kha wẹẹ : “ Wa gha mwẹ ekhọe hia , te ọ khẹke ne ima gha mwẹ ekhọe hia .\n",
            "2020-01-26 00:48:34,942 Example #3\n",
            "2020-01-26 00:48:34,942 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:48:34,942 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:48:34,942 \tHypothesis: ( 1 Kọr . 2 : 1 ) Ma gha ya ima gha mwẹ izinegbe vbe odẹ ọghe orhiọn , ma vbe gha mwẹ izinegbe vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 00:48:34,942 Validation result (greedy) at epoch   8, step     1000: bleu:   2.71, loss: 83337.4375, ppl:  26.8168, duration: 46.6714s\n",
            "2020-01-26 00:48:44,969 Epoch   8: total training loss 435.71\n",
            "2020-01-26 00:48:44,969 EPOCH 9\n",
            "2020-01-26 00:48:50,453 Epoch   9 Step:     1100 Batch Loss:     3.434236 Tokens per Sec:    12432, Lr: 0.000300\n",
            "2020-01-26 00:49:36,525 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:49:36,525 Saving new checkpoint.\n",
            "2020-01-26 00:49:37,556 Example #0\n",
            "2020-01-26 00:49:37,557 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:49:37,557 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:49:37,557 \tHypothesis: De vbene ima ya sẹtin ya sẹtin ya ima gha ya ima gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:49:37,557 Example #1\n",
            "2020-01-26 00:49:37,557 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:49:37,557 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:49:37,557 \tHypothesis: De vbene ọ ya sẹtin ya egbe fiohan ne emwa ni rre Baibol ?\n",
            "2020-01-26 00:49:37,557 Example #2\n",
            "2020-01-26 00:49:37,558 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:49:37,558 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:49:37,558 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma khẹke ne ima gha mwẹ ekhọe hia , ọ na gha mwẹ ekhọe hia , ọ na gha mwẹ ekhọe hia , ọ na gha mwẹ ekhọe hia .\n",
            "2020-01-26 00:49:37,558 Example #3\n",
            "2020-01-26 00:49:37,558 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:49:37,558 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:49:37,558 \tHypothesis: ( 1 Kọr . 3 : 1 ) Ma gha ya ima gha mwẹ alaghodaro vbe odẹ ọghe orhiọn , ma gha ya ima gha mwẹ alaghodaro vbe odẹ ọghe orhiọn , ma vbe gha ya ima gha mwẹ izinegbe .\n",
            "2020-01-26 00:49:37,558 Validation result (greedy) at epoch   9, step     1100: bleu:   2.64, loss: 82218.5234, ppl:  25.6584, duration: 47.1048s\n",
            "2020-01-26 00:49:52,995 Epoch   9: total training loss 424.14\n",
            "2020-01-26 00:49:52,995 EPOCH 10\n",
            "2020-01-26 00:49:53,327 Epoch  10 Step:     1200 Batch Loss:     3.239022 Tokens per Sec:    13356, Lr: 0.000300\n",
            "2020-01-26 00:50:39,453 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:50:39,453 Saving new checkpoint.\n",
            "2020-01-26 00:50:40,739 Example #0\n",
            "2020-01-26 00:50:40,740 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:50:40,740 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:50:40,740 \tHypothesis: De vbene ima ya sẹtin ya sẹtin ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:50:40,740 Example #1\n",
            "2020-01-26 00:50:40,741 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:50:40,741 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:50:40,741 \tHypothesis: De vbene Jehova ya ru iyobọ ne emwa ni rre Baibol ?\n",
            "2020-01-26 00:50:40,741 Example #2\n",
            "2020-01-26 00:50:40,742 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:50:40,742 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:50:40,742 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma khẹke ne a gha ru iyobọ ne a ya egbe fiohan ne a ya ru iyobọ ne iran .\n",
            "2020-01-26 00:50:40,742 Example #3\n",
            "2020-01-26 00:50:40,743 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:50:40,743 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:50:40,743 \tHypothesis: ( 1 Kọr . 4 : 1 ) Ma gha ya ima rẹn wẹẹ , e Jehova gha ya ima rẹn emwi nọ khẹke ne ima gha ru iyobọ ne ima ya ru iyobọ ne ima .\n",
            "2020-01-26 00:50:40,743 Validation result (greedy) at epoch  10, step     1200: bleu:   3.31, loss: 80442.4062, ppl:  23.9214, duration: 47.4161s\n",
            "2020-01-26 00:50:56,237 Epoch  10 Step:     1300 Batch Loss:     3.127620 Tokens per Sec:    12663, Lr: 0.000300\n",
            "2020-01-26 00:51:42,271 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:51:42,272 Saving new checkpoint.\n",
            "2020-01-26 00:51:43,419 Example #0\n",
            "2020-01-26 00:51:43,420 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:51:43,420 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:51:43,420 \tHypothesis: De vbene ima khian ya sẹtin gha ya sẹtin gha ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:51:43,420 Example #1\n",
            "2020-01-26 00:51:43,420 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:51:43,421 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:51:43,421 \tHypothesis: De vbene Jehova ya sẹtin ru iyobọ ne emwa ni rre Baibol hẹ ?\n",
            "2020-01-26 00:51:43,421 Example #2\n",
            "2020-01-26 00:51:43,421 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:51:43,422 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:51:43,422 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ keghi kha wẹẹ : “ Ọ ma ghi sẹ ẹghẹ ne I ya gha ya gha mwẹ alaghodaro vbe odẹ ọghe ẹmwata .\n",
            "2020-01-26 00:51:43,422 Example #3\n",
            "2020-01-26 00:51:43,422 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:51:43,422 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:51:43,425 \tHypothesis: ( 1 Kọr . 1 : 1 ) Ma gha ya ima gha ru iyobọ ne ima ya sẹtin gha ru iyobọ ne ima .\n",
            "2020-01-26 00:51:43,425 Validation result (greedy) at epoch  10, step     1300: bleu:   3.33, loss: 79197.8203, ppl:  22.7748, duration: 47.1882s\n",
            "2020-01-26 00:51:47,978 Epoch  10: total training loss 410.70\n",
            "2020-01-26 00:51:47,979 EPOCH 11\n",
            "2020-01-26 00:51:59,102 Epoch  11 Step:     1400 Batch Loss:     2.971020 Tokens per Sec:    12624, Lr: 0.000300\n",
            "2020-01-26 00:52:44,928 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:52:44,928 Saving new checkpoint.\n",
            "2020-01-26 00:52:46,047 Example #0\n",
            "2020-01-26 00:52:46,048 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:52:46,048 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:52:46,048 \tHypothesis: De emwi ne ima gha ya sẹtin ru iyobọ ne ima ya sẹtin ru iyobọ ne ima ya sẹtin ru iyobọ ne ima ?\n",
            "2020-01-26 00:52:46,049 Example #1\n",
            "2020-01-26 00:52:46,049 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:52:46,049 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:52:46,050 \tHypothesis: De emwi ne u gha ru ne u mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:52:46,050 Example #2\n",
            "2020-01-26 00:52:46,050 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:52:46,050 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:52:46,051 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma khẹke ne a gha mwẹ ekhọe , ọ na gha mwẹ ilẹkẹtin wẹẹ , ọ ma khẹke ne a gha mwẹ alaghodaro vbe odẹ ọghe ẹmwata .\n",
            "2020-01-26 00:52:46,051 Example #3\n",
            "2020-01-26 00:52:46,051 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:52:46,052 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:52:46,052 \tHypothesis: ( 1 Kọr . 1 : 1 ) Ma ghaa ya ima gha mwẹ izinegbe vbe odẹ ọghe orhiọn , ma vbe gha ya ima gha mwẹ izinegbe vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 00:52:46,052 Validation result (greedy) at epoch  11, step     1400: bleu:   3.12, loss: 78018.1719, ppl:  21.7388, duration: 46.9496s\n",
            "2020-01-26 00:52:55,869 Epoch  11: total training loss 403.10\n",
            "2020-01-26 00:52:55,870 EPOCH 12\n",
            "2020-01-26 00:53:01,702 Epoch  12 Step:     1500 Batch Loss:     2.987669 Tokens per Sec:    12281, Lr: 0.000300\n",
            "2020-01-26 00:53:47,712 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:53:47,713 Saving new checkpoint.\n",
            "2020-01-26 00:53:48,973 Example #0\n",
            "2020-01-26 00:53:48,974 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:53:48,974 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:53:48,974 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:53:48,974 Example #1\n",
            "2020-01-26 00:53:48,975 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:53:48,975 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:53:48,975 \tHypothesis: De emwi ne u gha ru ne u mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:53:48,975 Example #2\n",
            "2020-01-26 00:53:48,976 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:53:48,976 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:53:48,976 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma zẹdẹ gha mwẹ ẹghẹ ne I ya gha mwẹ alaghodaro vbe odẹ ọghe ẹmwata .\n",
            "2020-01-26 00:53:48,976 Example #3\n",
            "2020-01-26 00:53:48,977 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:53:48,977 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:53:48,977 \tHypothesis: ( 1 Kọr . 1 : 1 ) Ma gha sẹtin gha ya ima gha mwẹ izinegbe vbe odẹ ọghe orhiọn , ma gha ya ima gha mwẹ izinegbe .\n",
            "2020-01-26 00:53:48,977 Validation result (greedy) at epoch  12, step     1500: bleu:   3.73, loss: 76878.2500, ppl:  20.7824, duration: 47.2753s\n",
            "2020-01-26 00:54:04,021 Epoch  12: total training loss 401.99\n",
            "2020-01-26 00:54:04,022 EPOCH 13\n",
            "2020-01-26 00:54:04,334 Epoch  13 Step:     1600 Batch Loss:     2.887049 Tokens per Sec:    11716, Lr: 0.000300\n",
            "2020-01-26 00:54:50,217 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:54:50,217 Saving new checkpoint.\n",
            "2020-01-26 00:54:51,289 Example #0\n",
            "2020-01-26 00:54:51,290 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:54:51,290 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:54:51,290 \tHypothesis: De emwi ne ima gha ya sẹtin ru iyobọ ne ima ya gha ru iyobọ ne ima ya gha ru iyobọ ne ima ?\n",
            "2020-01-26 00:54:51,290 Example #1\n",
            "2020-01-26 00:54:51,291 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:54:51,291 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:54:51,291 \tHypothesis: De emwi ne Jehova ya ru iyobọ ne emwa ni rre Baibol ?\n",
            "2020-01-26 00:54:51,291 Example #2\n",
            "2020-01-26 00:54:51,291 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:54:51,291 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:54:51,291 \tHypothesis: Ọ keghi kha wẹẹ : “ I ma he kpẹ sẹ , ọ na vbe gha re ọmwa nọ gha ru emwi nọ maan .\n",
            "2020-01-26 00:54:51,292 Example #3\n",
            "2020-01-26 00:54:51,292 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:54:51,292 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:54:51,292 \tHypothesis: ( 1 Kọr . 4 : 1 ) Ma ghaa ya ima rẹn emwi nọ khẹke ne ima gha ru , ma gha ya ima gha mwẹ izinegbe vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 00:54:51,292 Validation result (greedy) at epoch  13, step     1600: bleu:   3.68, loss: 76290.8984, ppl:  20.3062, duration: 46.9575s\n",
            "2020-01-26 00:55:07,007 Epoch  13 Step:     1700 Batch Loss:     3.221068 Tokens per Sec:    12329, Lr: 0.000300\n",
            "2020-01-26 00:55:53,086 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:55:53,086 Saving new checkpoint.\n",
            "2020-01-26 00:55:54,200 Example #0\n",
            "2020-01-26 00:55:54,201 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:55:54,201 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:55:54,201 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:55:54,202 Example #1\n",
            "2020-01-26 00:55:54,202 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:55:54,202 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:55:54,202 \tHypothesis: De emwi ne ọ khẹke ne ima gha mwẹ ?\n",
            "2020-01-26 00:55:54,202 Example #2\n",
            "2020-01-26 00:55:54,203 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:55:54,203 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:55:54,203 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma he kpẹ sẹ , ọ na vbe gha mwẹ irẹnmwi nọ wegbe sayọ .\n",
            "2020-01-26 00:55:54,203 Example #3\n",
            "2020-01-26 00:55:54,204 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:55:54,204 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:55:54,204 \tHypothesis: ( 1 Kọr . 1 : 1 ) Ma gha miẹn vbene ima ya sẹtin gha mwẹ ilẹkẹtin wẹẹ , e Jehova ẹre ọ khẹke ne ima gha mwẹ izinegbe vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 00:55:54,204 Validation result (greedy) at epoch  13, step     1700: bleu:   3.62, loss: 75560.9688, ppl:  19.7296, duration: 47.1967s\n",
            "2020-01-26 00:55:59,109 Epoch  13: total training loss 388.79\n",
            "2020-01-26 00:55:59,109 EPOCH 14\n",
            "2020-01-26 00:56:10,117 Epoch  14 Step:     1800 Batch Loss:     3.113921 Tokens per Sec:    12007, Lr: 0.000300\n",
            "2020-01-26 00:56:56,375 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:56:56,375 Saving new checkpoint.\n",
            "2020-01-26 00:56:57,542 Example #0\n",
            "2020-01-26 00:56:57,543 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:56:57,543 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:56:57,543 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:56:57,544 Example #1\n",
            "2020-01-26 00:56:57,544 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:56:57,544 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:56:57,544 \tHypothesis: De emwi ne u miẹn ruẹ vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:56:57,545 Example #2\n",
            "2020-01-26 00:56:57,545 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:56:57,545 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:56:57,546 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ keghi re ọmwa ne ọ ru emwi ne ọ ru , ọ na vbe gha mwẹ asẹ ne a ya gha mwẹ alaghodaro vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 00:56:57,546 Example #3\n",
            "2020-01-26 00:56:57,546 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:56:57,546 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:56:57,547 \tHypothesis: ( 1 Kọr . 4 : 1 ) Nọnaghiyerriọ , ma gha ya ima rẹn wẹẹ , e Jehova ẹre ọ khẹke ne ima gha ru iyobọ ne ima .\n",
            "2020-01-26 00:56:57,547 Validation result (greedy) at epoch  14, step     1800: bleu:   4.01, loss: 74449.5156, ppl:  18.8829, duration: 47.4297s\n",
            "2020-01-26 00:57:07,758 Epoch  14: total training loss 380.92\n",
            "2020-01-26 00:57:07,759 EPOCH 15\n",
            "2020-01-26 00:57:13,006 Epoch  15 Step:     1900 Batch Loss:     2.277574 Tokens per Sec:    12780, Lr: 0.000300\n",
            "2020-01-26 00:57:58,881 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:57:58,881 Saving new checkpoint.\n",
            "2020-01-26 00:58:00,082 Example #0\n",
            "2020-01-26 00:58:00,083 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:58:00,083 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:58:00,083 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:58:00,083 Example #1\n",
            "2020-01-26 00:58:00,083 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:58:00,084 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:58:00,084 \tHypothesis: De emwi ne u gha ru ne u mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:58:00,084 Example #2\n",
            "2020-01-26 00:58:00,084 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:58:00,084 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:58:00,084 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma khẹke ne emwa ni rre uwu ẹbu iran gha rrie ehe ne iran ya gha mwẹ alaghodaro vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 00:58:00,084 Example #3\n",
            "2020-01-26 00:58:00,085 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:58:00,085 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:58:00,085 \tHypothesis: ( 1 Kọr . 4 : 1 ) Ma gha ya egbe tae Jehova , ma gha mwẹ ilẹkẹtin wẹẹ , e Jehova gha ya ima gha mwẹ izinegbe .\n",
            "2020-01-26 00:58:00,085 Validation result (greedy) at epoch  15, step     1900: bleu:   4.46, loss: 73871.9922, ppl:  18.4573, duration: 47.0781s\n",
            "2020-01-26 00:58:15,602 Epoch  15 Step:     2000 Batch Loss:     3.125184 Tokens per Sec:    12320, Lr: 0.000300\n",
            "2020-01-26 00:59:01,633 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 00:59:01,633 Saving new checkpoint.\n",
            "2020-01-26 00:59:02,730 Example #0\n",
            "2020-01-26 00:59:02,730 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 00:59:02,731 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 00:59:02,731 \tHypothesis: De emwi nọ khẹke ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 00:59:02,731 Example #1\n",
            "2020-01-26 00:59:02,731 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 00:59:02,731 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 00:59:02,731 \tHypothesis: De emwi nọ khẹke ne ima gha ru ne ima mieke na sẹtin gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe ?\n",
            "2020-01-26 00:59:02,732 Example #2\n",
            "2020-01-26 00:59:02,732 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 00:59:02,732 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 00:59:02,732 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma khẹke ne iran gha mwẹ ekhọe , ọ keghi ya egbe fiohan ne Jehova .\n",
            "2020-01-26 00:59:02,732 Example #3\n",
            "2020-01-26 00:59:02,732 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 00:59:02,732 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 00:59:02,733 \tHypothesis: ( 1 Kọr . 5 : 1 ) E Jehova keghi re emwi nọ khẹke ne ima gha ru , ne ima mieke na sẹtin gha mwẹ izinegbe .\n",
            "2020-01-26 00:59:02,733 Validation result (greedy) at epoch  15, step     2000: bleu:   4.05, loss: 73189.6641, ppl:  17.9669, duration: 47.1298s\n",
            "2020-01-26 00:59:02,737 Epoch  15: total training loss 374.35\n",
            "2020-01-26 00:59:02,737 EPOCH 16\n",
            "2020-01-26 00:59:18,266 Epoch  16 Step:     2100 Batch Loss:     2.922251 Tokens per Sec:    12547, Lr: 0.000300\n",
            "2020-01-26 01:00:04,180 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:00:04,181 Saving new checkpoint.\n",
            "2020-01-26 01:00:05,211 Example #0\n",
            "2020-01-26 01:00:05,212 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:00:05,212 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:00:05,212 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:00:05,212 Example #1\n",
            "2020-01-26 01:00:05,212 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:00:05,212 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:00:05,212 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na sẹtin gha mwẹ udinmwẹ ?\n",
            "2020-01-26 01:00:05,213 Example #2\n",
            "2020-01-26 01:00:05,213 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:00:05,213 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:00:05,213 \tHypothesis: Vbe Jesu rre ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo ukpo nibun ẹre ọ ya gha ya gha khian .\n",
            "2020-01-26 01:00:05,213 Example #3\n",
            "2020-01-26 01:00:05,213 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:00:05,214 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:00:05,214 \tHypothesis: ( 1 Kọr . 2 : 1 ) Nọnaghiyerriọ , te ọ khẹke ne ima gha mwẹ ilẹkẹtin wẹẹ , e Jehova ẹre ọ khẹke ne ima gha ru iyobọ ne ima .\n",
            "2020-01-26 01:00:05,214 Validation result (greedy) at epoch  16, step     2100: bleu:   4.22, loss: 72345.9453, ppl:  17.3785, duration: 46.9469s\n",
            "2020-01-26 01:00:10,204 Epoch  16: total training loss 359.96\n",
            "2020-01-26 01:00:10,204 EPOCH 17\n",
            "2020-01-26 01:00:20,892 Epoch  17 Step:     2200 Batch Loss:     2.097275 Tokens per Sec:    12395, Lr: 0.000300\n",
            "2020-01-26 01:01:06,828 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:01:06,828 Saving new checkpoint.\n",
            "2020-01-26 01:01:07,930 Example #0\n",
            "2020-01-26 01:01:07,931 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:01:07,931 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:01:07,931 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:01:07,931 Example #1\n",
            "2020-01-26 01:01:07,932 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:01:07,932 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:01:07,932 \tHypothesis: De emwi ne ima miẹn ruẹ vbe emwi ne Jehova ru ?\n",
            "2020-01-26 01:01:07,932 Example #2\n",
            "2020-01-26 01:01:07,933 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:01:07,933 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:01:07,933 \tHypothesis: Vbe iyeke ọni , ọ na ghi sẹ ẹghẹ , ọ na vbe gha re emwa ni wulo kpaegbe vbe odẹ ọghe ẹmwata .\n",
            "2020-01-26 01:01:07,933 Example #3\n",
            "2020-01-26 01:01:07,934 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:01:07,934 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:01:07,934 \tHypothesis: ( Mat . 4 : ​ 1 ) Ọna rhiema wẹẹ , te ọ khẹke ne ima gha mwẹ ilẹkẹtin wẹẹ , e Jehova gha ya ima gha mwẹ izinegbe .\n",
            "2020-01-26 01:01:07,935 Validation result (greedy) at epoch  17, step     2200: bleu:   4.74, loss: 72096.8125, ppl:  17.2085, duration: 47.0425s\n",
            "2020-01-26 01:01:18,010 Epoch  17: total training loss 350.75\n",
            "2020-01-26 01:01:18,011 EPOCH 18\n",
            "2020-01-26 01:01:23,689 Epoch  18 Step:     2300 Batch Loss:     2.545426 Tokens per Sec:    13131, Lr: 0.000300\n",
            "2020-01-26 01:02:09,497 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:02:09,497 Saving new checkpoint.\n",
            "2020-01-26 01:02:10,602 Example #0\n",
            "2020-01-26 01:02:10,603 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:02:10,603 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:02:10,603 \tHypothesis: De emwi nọ khẹke ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:02:10,603 Example #1\n",
            "2020-01-26 01:02:10,603 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:02:10,603 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:02:10,603 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na sẹtin gha mwẹ udinmwẹ ?\n",
            "2020-01-26 01:02:10,604 Example #2\n",
            "2020-01-26 01:02:10,604 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:02:10,604 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:02:10,604 \tHypothesis: Vbe Jesu rre ukpo 1914 , ọ keghi kha wẹẹ , ọ ma khẹke ne iran gha mu egbe rriotọ .\n",
            "2020-01-26 01:02:10,604 Example #3\n",
            "2020-01-26 01:02:10,604 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:02:10,605 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:02:10,606 \tHypothesis: ( Mat 10 : 10 ) Nọnaghiyerriọ , ma gha mwẹ ilẹkẹtin wẹẹ , te ọ khẹke ne ima gha mwẹ ahoẹmwọmwa nọ wegbe sayọ .\n",
            "2020-01-26 01:02:10,606 Validation result (greedy) at epoch  18, step     2300: bleu:   4.52, loss: 71423.8203, ppl:  16.7574, duration: 46.9160s\n",
            "2020-01-26 01:02:25,247 Epoch  18: total training loss 350.05\n",
            "2020-01-26 01:02:25,247 EPOCH 19\n",
            "2020-01-26 01:02:25,848 Epoch  19 Step:     2400 Batch Loss:     2.966531 Tokens per Sec:    11317, Lr: 0.000300\n",
            "2020-01-26 01:03:11,691 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:03:11,692 Saving new checkpoint.\n",
            "2020-01-26 01:03:13,193 Example #0\n",
            "2020-01-26 01:03:13,194 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:03:13,194 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:03:13,194 \tHypothesis: De vbene ima khian ya sẹtin ya sẹtin ru iyobọ ne ima hẹ vbe ima ghaa rre uwu ẹbu ima ?\n",
            "2020-01-26 01:03:13,194 Example #1\n",
            "2020-01-26 01:03:13,194 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:03:13,195 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:03:13,195 \tHypothesis: De emwi nọ sunu daa emwa ọvbehe ?\n",
            "2020-01-26 01:03:13,195 Example #2\n",
            "2020-01-26 01:03:13,195 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:03:13,195 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:03:13,195 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma he kpẹ vba , ọ na vbe gha mu okhian gha rrie ẹvbo nii , ọ na vbe gha re emwa ni rre ẹvbo nii .\n",
            "2020-01-26 01:03:13,195 Example #3\n",
            "2020-01-26 01:03:13,196 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:03:13,196 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:03:13,196 \tHypothesis: ( 1 Kọr . 5 : 1 ) Vbe igiemwi , ma gha miẹn afiangbe vbe igiemwi esi ọghe Jesu , ma ghi gha mwẹ izinegbe vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 01:03:13,196 Validation result (greedy) at epoch  19, step     2400: bleu:   4.74, loss: 71028.2500, ppl:  16.4979, duration: 47.3475s\n",
            "2020-01-26 01:03:28,562 Epoch  19 Step:     2500 Batch Loss:     2.137297 Tokens per Sec:    12665, Lr: 0.000300\n",
            "2020-01-26 01:04:14,670 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:04:14,670 Saving new checkpoint.\n",
            "2020-01-26 01:04:15,793 Example #0\n",
            "2020-01-26 01:04:15,793 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:04:15,793 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:04:15,793 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:04:15,793 Example #1\n",
            "2020-01-26 01:04:15,794 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:04:15,794 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:04:15,794 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ udinmwẹ ?\n",
            "2020-01-26 01:04:15,794 Example #2\n",
            "2020-01-26 01:04:15,794 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:04:15,795 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:04:15,795 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma he kpẹ vba , ọ na vbe gha re ne ọ na gha re emwa ni wulo vbe odẹ ọghe ẹmwata .\n",
            "2020-01-26 01:04:15,795 Example #3\n",
            "2020-01-26 01:04:15,795 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:04:15,795 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:04:15,796 \tHypothesis: ( Mat . 2 : ​ 1 - 14 ) E Jehova keghi re akpa nẹdẹ , ọ khẹke ne ima gha mwẹ itohan daa emwa ọvbehe .\n",
            "2020-01-26 01:04:15,796 Validation result (greedy) at epoch  19, step     2500: bleu:   4.46, loss: 70475.7578, ppl:  16.1420, duration: 47.2334s\n",
            "2020-01-26 01:04:20,387 Epoch  19: total training loss 347.15\n",
            "2020-01-26 01:04:20,387 EPOCH 20\n",
            "2020-01-26 01:04:31,199 Epoch  20 Step:     2600 Batch Loss:     2.315899 Tokens per Sec:    12698, Lr: 0.000300\n",
            "2020-01-26 01:05:17,219 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:05:17,220 Saving new checkpoint.\n",
            "2020-01-26 01:05:18,333 Example #0\n",
            "2020-01-26 01:05:18,333 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:05:18,333 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:05:18,333 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ irẹnmwi sayọ hẹ vbe ima ghaa rre uwu ọlọghọmwa ?\n",
            "2020-01-26 01:05:18,334 Example #1\n",
            "2020-01-26 01:05:18,334 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:05:18,334 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:05:18,334 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe mieke na sẹtin gha mwẹ ekhọe agiẹngiẹn daa re ?\n",
            "2020-01-26 01:05:18,334 Example #2\n",
            "2020-01-26 01:05:18,334 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:05:18,334 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:05:18,334 \tHypothesis: Ọ keghi kha wẹẹ : “ E Jesu wa mu ẹtin yan Ivbi Izrẹl .\n",
            "2020-01-26 01:05:18,335 Example #3\n",
            "2020-01-26 01:05:18,335 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:05:18,335 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:05:18,335 \tHypothesis: ( 1 Kọr . 4 : 1 ) E Jehova keghi rhie akpa na ma , ọ khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe .\n",
            "2020-01-26 01:05:18,335 Validation result (greedy) at epoch  20, step     2600: bleu:   4.99, loss: 70019.7109, ppl:  15.8541, duration: 47.1358s\n",
            "2020-01-26 01:05:28,103 Epoch  20: total training loss 342.85\n",
            "2020-01-26 01:05:28,104 EPOCH 21\n",
            "2020-01-26 01:05:33,605 Epoch  21 Step:     2700 Batch Loss:     2.625189 Tokens per Sec:    12920, Lr: 0.000300\n",
            "2020-01-26 01:06:19,633 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:06:19,633 Saving new checkpoint.\n",
            "2020-01-26 01:06:21,152 Example #0\n",
            "2020-01-26 01:06:21,153 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:06:21,153 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:06:21,153 \tHypothesis: De vbene ima khian ya sẹtin ya ima rẹn hẹ vbe odẹ ne ima gha ya sẹtin ya egbe tae Jehova ?\n",
            "2020-01-26 01:06:21,153 Example #1\n",
            "2020-01-26 01:06:21,153 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:06:21,153 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:06:21,153 \tHypothesis: De emwi nọ khẹke ne ima ru ne emwa ọvbehe mieke na sẹtin gha mwẹ izinegbe ?\n",
            "2020-01-26 01:06:21,153 Example #2\n",
            "2020-01-26 01:06:21,153 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:06:21,153 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:06:21,154 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma he kpẹ vba , ọ na vbe gha mu otuẹ gie Jehova .\n",
            "2020-01-26 01:06:21,154 Example #3\n",
            "2020-01-26 01:06:21,154 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:06:21,154 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:06:21,154 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima gha ru ekpataki sẹ vbe ima gha ya egbe tae Jehova .\n",
            "2020-01-26 01:06:21,154 Validation result (greedy) at epoch  21, step     2700: bleu:   5.02, loss: 69720.8672, ppl:  15.6682, duration: 47.5485s\n",
            "2020-01-26 01:06:36,143 Epoch  21: total training loss 330.91\n",
            "2020-01-26 01:06:36,144 EPOCH 22\n",
            "2020-01-26 01:06:36,723 Epoch  22 Step:     2800 Batch Loss:     2.523574 Tokens per Sec:    10254, Lr: 0.000300\n",
            "2020-01-26 01:07:22,708 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:07:22,709 Saving new checkpoint.\n",
            "2020-01-26 01:07:23,817 Example #0\n",
            "2020-01-26 01:07:23,818 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:07:23,818 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:07:23,818 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:07:23,818 Example #1\n",
            "2020-01-26 01:07:23,818 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:07:23,819 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:07:23,819 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe mieke na sẹtin gha mwẹ afanvbimu ?\n",
            "2020-01-26 01:07:23,819 Example #2\n",
            "2020-01-26 01:07:23,819 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:07:23,819 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:07:23,819 \tHypothesis: Vbe ọ ghi rre ukpo 500 , ọ na do gha mwẹ udinmwẹ vbe iwinna ugamwẹ e Jehova .\n",
            "2020-01-26 01:07:23,819 Example #3\n",
            "2020-01-26 01:07:23,820 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:07:23,820 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:07:23,820 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru , ne ima mieke na sẹtin gha mwẹ ahoẹmwọmwa daa emwa ọvbehe .\n",
            "2020-01-26 01:07:23,820 Validation result (greedy) at epoch  22, step     2800: bleu:   5.05, loss: 69225.8516, ppl:  15.3651, duration: 47.0958s\n",
            "2020-01-26 01:07:39,512 Epoch  22 Step:     2900 Batch Loss:     2.870596 Tokens per Sec:    12544, Lr: 0.000300\n",
            "2020-01-26 01:08:25,742 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:08:25,743 Saving new checkpoint.\n",
            "2020-01-26 01:08:26,820 Example #0\n",
            "2020-01-26 01:08:26,820 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:08:26,820 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:08:26,821 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ irẹnmwi ọghe Baibol hẹ vbe ima ghaa rre Baibol ?\n",
            "2020-01-26 01:08:26,821 Example #1\n",
            "2020-01-26 01:08:26,821 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:08:26,821 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:08:26,821 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe mieke na sẹtin gha mwẹ izinegbe ?\n",
            "2020-01-26 01:08:26,821 Example #2\n",
            "2020-01-26 01:08:26,822 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:08:26,822 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:08:26,822 \tHypothesis: Vbe Jesu rre agbọn , ọ na ghi do gha mwẹ egbe iziro nọ ma gba vbe ẹghẹ ne Jesu ya gha mwẹ udinmwẹ .\n",
            "2020-01-26 01:08:26,822 Example #3\n",
            "2020-01-26 01:08:26,822 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:08:26,822 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:08:26,822 \tHypothesis: ( 1 Kọr . 2 : 13 ) E Baibol khare wẹẹ , ọ khẹke ne ima gha mwẹ ahoẹmwọmwa nọ sẹ otọ ẹko , ne ima mieke na sẹtin gha mwẹ izinegbe .\n",
            "2020-01-26 01:08:26,823 Validation result (greedy) at epoch  22, step     2900: bleu:   5.00, loss: 68906.1797, ppl:  15.1724, duration: 47.3103s\n",
            "2020-01-26 01:08:31,222 Epoch  22: total training loss 327.53\n",
            "2020-01-26 01:08:31,222 EPOCH 23\n",
            "2020-01-26 01:08:42,386 Epoch  23 Step:     3000 Batch Loss:     2.467261 Tokens per Sec:    12291, Lr: 0.000300\n",
            "2020-01-26 01:09:28,551 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:09:28,551 Saving new checkpoint.\n",
            "2020-01-26 01:09:29,633 Example #0\n",
            "2020-01-26 01:09:29,634 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:09:29,635 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:09:29,635 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ irẹnmwi sayọ hẹ vbe ima ghaa rre Ẹmwẹ Osanobua ?\n",
            "2020-01-26 01:09:29,635 Example #1\n",
            "2020-01-26 01:09:29,635 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:09:29,636 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:09:29,636 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe ?\n",
            "2020-01-26 01:09:29,636 Example #2\n",
            "2020-01-26 01:09:29,636 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:09:29,636 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:09:29,637 \tHypothesis: Ọ keghi kha wẹẹ : “ Ọ ma ghi sẹ ẹghẹ , ọ na gha re ọmwa nọ dizigha ọyẹvbu , sokpan ọ ma ghi gha mu egbe ye okaro vbe ẹghẹ hia .\n",
            "2020-01-26 01:09:29,637 Example #3\n",
            "2020-01-26 01:09:29,637 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:09:29,637 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:09:29,637 \tHypothesis: ( Mat . 5 : ​ 13 ) E Jehova keghi re ahoẹmwọmwa nọ sẹ otọ ẹko , ọ keghi re ahoẹmwọmwa nọ sẹ otọ ẹko .\n",
            "2020-01-26 01:09:29,638 Validation result (greedy) at epoch  23, step     3000: bleu:   5.32, loss: 68791.1953, ppl:  15.1037, duration: 47.2515s\n",
            "2020-01-26 01:09:39,964 Epoch  23: total training loss 325.54\n",
            "2020-01-26 01:09:39,965 EPOCH 24\n",
            "2020-01-26 01:09:45,479 Epoch  24 Step:     3100 Batch Loss:     2.488948 Tokens per Sec:    12348, Lr: 0.000300\n",
            "2020-01-26 01:10:31,325 Example #0\n",
            "2020-01-26 01:10:31,325 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:10:31,326 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:10:31,326 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:10:31,326 Example #1\n",
            "2020-01-26 01:10:31,326 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:10:31,326 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:10:31,327 \tHypothesis: De emwi ne iran ru ne iran mieke na sẹtin gha mwẹ udinmwẹ ?\n",
            "2020-01-26 01:10:31,327 Example #2\n",
            "2020-01-26 01:10:31,327 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:10:31,327 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:10:31,327 \tHypothesis: Vbe Jesu rre agbọn , ọ keghi mu okhian gha rrie ehe ne iran ye vbe ẹghẹ ne Jesu ya gha kporhu ma emwa ni ma he miẹn odẹ ọghe ẹmwata yi .\n",
            "2020-01-26 01:10:31,327 Example #3\n",
            "2020-01-26 01:10:31,328 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:10:31,328 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:10:31,328 \tHypothesis: Vbe ako iruẹmwi na , ma gha ziro yan emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ ẹbo izinegbe .\n",
            "2020-01-26 01:10:31,328 Validation result (greedy) at epoch  24, step     3100: bleu:   5.00, loss: 69157.2266, ppl:  15.3235, duration: 45.8484s\n",
            "2020-01-26 01:10:46,140 Epoch  24: total training loss 321.68\n",
            "2020-01-26 01:10:46,141 EPOCH 25\n",
            "2020-01-26 01:10:46,626 Epoch  25 Step:     3200 Batch Loss:     2.501143 Tokens per Sec:    12346, Lr: 0.000300\n",
            "2020-01-26 01:11:32,589 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:11:32,589 Saving new checkpoint.\n",
            "2020-01-26 01:11:33,696 Example #0\n",
            "2020-01-26 01:11:33,696 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:11:33,696 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:11:33,696 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ irẹnmwi sayọ hẹ ?\n",
            "2020-01-26 01:11:33,699 Example #1\n",
            "2020-01-26 01:11:33,700 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:11:33,700 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:11:33,700 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe iran mieke na sẹtin gha mwẹ afanvbimu ?\n",
            "2020-01-26 01:11:33,701 Example #2\n",
            "2020-01-26 01:11:33,701 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:11:33,701 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:11:33,701 \tHypothesis: Vbe ọ ghi rre ukpo 20 , ọ na suẹn gha kporhu ma emwa ni ma he kpẹ vba .\n",
            "2020-01-26 01:11:33,702 Example #3\n",
            "2020-01-26 01:11:33,702 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:11:33,702 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:11:33,702 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru ne ima mieke na sẹtin gha mwẹ ahoẹmwọmwa ne ima mwẹ daa emwa ọvbehe .\n",
            "2020-01-26 01:11:33,703 Validation result (greedy) at epoch  25, step     3200: bleu:   5.68, loss: 68055.7578, ppl:  14.6716, duration: 47.0764s\n",
            "2020-01-26 01:11:49,421 Epoch  25 Step:     3300 Batch Loss:     1.871578 Tokens per Sec:    12290, Lr: 0.000300\n",
            "2020-01-26 01:12:35,435 Example #0\n",
            "2020-01-26 01:12:35,436 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:12:35,437 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:12:35,437 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:12:35,437 Example #1\n",
            "2020-01-26 01:12:35,438 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:12:35,438 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:12:35,438 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe mieke na gha mwẹ ekhọe agiẹngiẹn daa re ?\n",
            "2020-01-26 01:12:35,438 Example #2\n",
            "2020-01-26 01:12:35,439 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:12:35,439 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:12:35,439 \tHypothesis: Ọ ghi sẹ ẹghẹ , ọ na vbe gha re ọmwa nọ dizigha ọyẹvbu .\n",
            "2020-01-26 01:12:35,439 Example #3\n",
            "2020-01-26 01:12:35,439 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:12:35,440 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:12:35,440 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya sẹtin ya egbe tae Jehova kevbe odẹ ne ọ ya ya ima mwẹ akugbe .\n",
            "2020-01-26 01:12:35,440 Validation result (greedy) at epoch  25, step     3300: bleu:   5.67, loss: 68056.8594, ppl:  14.6723, duration: 46.0184s\n",
            "2020-01-26 01:12:40,202 Epoch  25: total training loss 315.50\n",
            "2020-01-26 01:12:40,203 EPOCH 26\n",
            "2020-01-26 01:12:50,895 Epoch  26 Step:     3400 Batch Loss:     2.437987 Tokens per Sec:    12400, Lr: 0.000300\n",
            "2020-01-26 01:13:36,820 Example #0\n",
            "2020-01-26 01:13:36,820 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:13:36,821 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:13:36,821 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ irẹnmwi nọ gbae vbekpae Jehova ?\n",
            "2020-01-26 01:13:36,821 Example #1\n",
            "2020-01-26 01:13:36,821 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:13:36,822 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:13:36,822 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe ?\n",
            "2020-01-26 01:13:36,822 Example #2\n",
            "2020-01-26 01:13:36,822 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:13:36,822 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:13:36,823 \tHypothesis: Vbe Jesu rre ukpo 20 , ọ keghi kha wẹẹ : “ Vbe ọ ghi rre ukpo ọmọ ọdiọn ne a ya mu egbe ye otọ .\n",
            "2020-01-26 01:13:36,823 Example #3\n",
            "2020-01-26 01:13:36,823 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:13:36,823 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:13:36,823 \tHypothesis: ( 1 Kọr . 4 : 1 ) Ọ khẹke ne ima gha mwẹ ahoẹmwọmwa nọ sẹ otọ ẹko , ma ghi gha mwẹ ilẹkẹtin wẹẹ , ọ gha sẹtin ya ima mwẹ ilẹkẹtin wẹẹ , ọ gha mwẹ ahoẹmwọmwa nọ sẹ otọ ẹko .\n",
            "2020-01-26 01:13:36,824 Validation result (greedy) at epoch  26, step     3400: bleu:   5.16, loss: 68085.7188, ppl:  14.6890, duration: 45.9282s\n",
            "2020-01-26 01:13:46,569 Epoch  26: total training loss 310.78\n",
            "2020-01-26 01:13:46,569 EPOCH 27\n",
            "2020-01-26 01:13:52,062 Epoch  27 Step:     3500 Batch Loss:     2.327887 Tokens per Sec:    12740, Lr: 0.000300\n",
            "2020-01-26 01:14:37,954 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:14:37,954 Saving new checkpoint.\n",
            "2020-01-26 01:14:39,531 Example #0\n",
            "2020-01-26 01:14:39,531 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:14:39,531 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:14:39,532 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:14:39,532 Example #1\n",
            "2020-01-26 01:14:39,532 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:14:39,532 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:14:39,533 \tHypothesis: De emwi ne ima gha ru ne Jehova mieke na sẹtin gha mwẹ udinmwẹ ?\n",
            "2020-01-26 01:14:39,533 Example #2\n",
            "2020-01-26 01:14:39,533 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:14:39,533 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:14:39,534 \tHypothesis: Vbe ọ ghi rre ukpo 20 , ọ na suẹn gha kporhu ma emwa ni khian eniwanrẹn nẹ , ọ na vbe gha mwẹ obọ vbe iwinna ugamwẹ ọghe iran .\n",
            "2020-01-26 01:14:39,534 Example #3\n",
            "2020-01-26 01:14:39,534 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:14:39,534 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:14:39,535 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rhiẹre ma wẹẹ ima hoẹmwẹ Osanobua , ọ gha ya ima rẹn odẹ ne ima ya rhiẹre ma wẹẹ , irẹn ẹre ọ ya ima mwẹ ilẹkẹtin wẹẹ , irẹn ẹre ọ khẹke ne ima gha mwẹ ẹtin yan rẹn .\n",
            "2020-01-26 01:14:39,535 Validation result (greedy) at epoch  27, step     3500: bleu:   5.71, loss: 67995.0938, ppl:  14.6366, duration: 47.4727s\n",
            "2020-01-26 01:14:55,000 Epoch  27: total training loss 301.92\n",
            "2020-01-26 01:14:55,001 EPOCH 28\n",
            "2020-01-26 01:14:55,488 Epoch  28 Step:     3600 Batch Loss:     1.934720 Tokens per Sec:    11828, Lr: 0.000300\n",
            "2020-01-26 01:15:41,420 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:15:41,420 Saving new checkpoint.\n",
            "2020-01-26 01:15:42,613 Example #0\n",
            "2020-01-26 01:15:42,613 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:15:42,613 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:15:42,613 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:15:42,614 Example #1\n",
            "2020-01-26 01:15:42,614 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:15:42,614 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:15:42,614 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe ?\n",
            "2020-01-26 01:15:42,614 Example #2\n",
            "2020-01-26 01:15:42,615 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:15:42,615 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:15:42,615 \tHypothesis: Vbe ọ rre ukpo 20 , ọ keghi do gha mwẹ iyayi nọ wegbe .\n",
            "2020-01-26 01:15:42,615 Example #3\n",
            "2020-01-26 01:15:42,615 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:15:42,615 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:15:42,615 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru , ne ima mieke na sẹtin gha mwẹ ilẹkẹtin wẹẹ , irẹn ẹre ọ mwẹ asẹ ne ima ya khaevbisẹ .\n",
            "2020-01-26 01:15:42,615 Validation result (greedy) at epoch  28, step     3600: bleu:   5.61, loss: 67231.4219, ppl:  14.2020, duration: 47.1267s\n",
            "2020-01-26 01:15:58,475 Epoch  28 Step:     3700 Batch Loss:     2.301742 Tokens per Sec:    12467, Lr: 0.000300\n",
            "2020-01-26 01:16:44,674 Example #0\n",
            "2020-01-26 01:16:44,674 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:16:44,675 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:16:44,675 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:16:44,675 Example #1\n",
            "2020-01-26 01:16:44,676 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:16:44,676 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:16:44,676 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 01:16:44,676 Example #2\n",
            "2020-01-26 01:16:44,676 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:16:44,677 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:16:44,677 \tHypothesis: Vbe iyeke ọni , ọ na suẹn gha kporhu vbe ẹvbo nii , ọ na vbe gha re okpẹvbo ne a tie ẹre Luca .\n",
            "2020-01-26 01:16:44,677 Example #3\n",
            "2020-01-26 01:16:44,677 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:16:44,678 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:16:44,678 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rhiẹre ma wẹẹ , ahoẹmwọmwa ẹre ọ mwẹ daa ima .\n",
            "2020-01-26 01:16:44,678 Validation result (greedy) at epoch  28, step     3700: bleu:   5.60, loss: 67388.3281, ppl:  14.2902, duration: 46.2023s\n",
            "2020-01-26 01:16:49,350 Epoch  28: total training loss 299.91\n",
            "2020-01-26 01:16:49,351 EPOCH 29\n",
            "2020-01-26 01:17:00,102 Epoch  29 Step:     3800 Batch Loss:     2.425150 Tokens per Sec:    12553, Lr: 0.000300\n",
            "2020-01-26 01:17:46,093 Example #0\n",
            "2020-01-26 01:17:46,094 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:17:46,094 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:17:46,094 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:17:46,094 Example #1\n",
            "2020-01-26 01:17:46,095 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:17:46,095 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:17:46,096 \tHypothesis: De emwi ne Jehova ru ne ọ ya egbe tae yotọ ?\n",
            "2020-01-26 01:17:46,096 Example #2\n",
            "2020-01-26 01:17:46,096 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:17:46,097 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:17:46,097 \tHypothesis: Vbe ọ rre ukpo 20 , ọ keghi kpa hin owa rre , ọ na vbe gha rrie ẹvbo nii , ọ na gha re Osẹe Jehova .\n",
            "2020-01-26 01:17:46,097 Example #3\n",
            "2020-01-26 01:17:46,097 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:17:46,098 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:17:46,098 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rhiẹre ma wẹẹ ima hoẹmwẹ egbe ima , ọ khẹke ne ima gha mwẹ ilẹkẹtin wẹẹ , ọ khẹke ne ima gha mwẹ ilẹkẹtin wẹẹ , ọ gha ru iyobọ ne ima ya gha mwẹ ilẹkẹtin yan rẹn .\n",
            "2020-01-26 01:17:46,098 Validation result (greedy) at epoch  29, step     3800: bleu:   6.32, loss: 67441.6953, ppl:  14.3204, duration: 45.9956s\n",
            "2020-01-26 01:17:56,033 Epoch  29: total training loss 296.48\n",
            "2020-01-26 01:17:56,034 EPOCH 30\n",
            "2020-01-26 01:18:01,789 Epoch  30 Step:     3900 Batch Loss:     1.639272 Tokens per Sec:    12436, Lr: 0.000300\n",
            "2020-01-26 01:18:47,935 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:18:47,935 Saving new checkpoint.\n",
            "2020-01-26 01:18:49,637 Example #0\n",
            "2020-01-26 01:18:49,638 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:18:49,638 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:18:49,638 \tHypothesis: De vbene ima khian ya sẹtin ya egbe tae Jehova hẹ vbe odẹ ne ọ ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:18:49,639 Example #1\n",
            "2020-01-26 01:18:49,639 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:18:49,639 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:18:49,639 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe ?\n",
            "2020-01-26 01:18:49,639 Example #2\n",
            "2020-01-26 01:18:49,640 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:18:49,640 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:18:49,640 \tHypothesis: Vbe ọ rre ukpo 20 , ọ keghi kpa hin owa rre , ọ na vbe gha rrie ẹvbo nii , ọ na vbe gha re emwa ni rre ẹvbo nii .\n",
            "2020-01-26 01:18:49,640 Example #3\n",
            "2020-01-26 01:18:49,640 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:18:49,641 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:18:49,641 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rẹn wẹẹ , ahoẹmwọmwa ẹre ọ khẹke ne ima mwẹ daa emwa ọvbehe .\n",
            "2020-01-26 01:18:49,641 Validation result (greedy) at epoch  30, step     3900: bleu:   5.79, loss: 67131.8438, ppl:  14.1463, duration: 47.8509s\n",
            "2020-01-26 01:19:04,712 Epoch  30: total training loss 289.23\n",
            "2020-01-26 01:19:04,712 EPOCH 31\n",
            "2020-01-26 01:19:05,527 Epoch  31 Step:     4000 Batch Loss:     2.456038 Tokens per Sec:    13110, Lr: 0.000300\n",
            "2020-01-26 01:19:51,717 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:19:51,717 Saving new checkpoint.\n",
            "2020-01-26 01:19:52,924 Example #0\n",
            "2020-01-26 01:19:52,925 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:19:52,925 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:19:52,925 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:19:52,925 Example #1\n",
            "2020-01-26 01:19:52,926 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:19:52,926 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:19:52,926 \tHypothesis: De emwi nọ khẹke ne ima ru ne Jehova mieke na gha mwẹ ekhọe agiẹngiẹn ?\n",
            "2020-01-26 01:19:52,926 Example #2\n",
            "2020-01-26 01:19:52,926 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:19:52,927 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:19:52,928 \tHypothesis: Vbe Jesu rre ukpo 20 , ọ keghi do gha mwẹ iyayi nọ wegbe rhunmwuda , ọ ma he kpẹ vba .\n",
            "2020-01-26 01:19:52,928 Example #3\n",
            "2020-01-26 01:19:52,929 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:19:52,929 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:19:52,929 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rhiẹre ma wẹẹ ima hoẹmwẹ emwa ọvbehe , ọ ma khẹke ne ima gha mwẹ ahoẹmwọmwa daa emwa ọvbehe .\n",
            "2020-01-26 01:19:52,929 Validation result (greedy) at epoch  31, step     4000: bleu:   6.28, loss: 66808.6328, ppl:  13.9670, duration: 47.4021s\n",
            "2020-01-26 01:20:08,608 Epoch  31 Step:     4100 Batch Loss:     2.091172 Tokens per Sec:    12260, Lr: 0.000300\n",
            "2020-01-26 01:20:54,766 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:20:54,766 Saving new checkpoint.\n",
            "2020-01-26 01:20:55,917 Example #0\n",
            "2020-01-26 01:20:55,919 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:20:55,920 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:20:55,922 \tHypothesis: De vbene ima khian ya sẹtin gha mwẹ irẹnmwi ọghe Baibol hẹ ?\n",
            "2020-01-26 01:20:55,924 Example #1\n",
            "2020-01-26 01:20:55,925 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:20:55,925 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:20:55,925 \tHypothesis: De emwi ne ima gha ru ne emwa ọvbehe ?\n",
            "2020-01-26 01:20:55,925 Example #2\n",
            "2020-01-26 01:20:55,926 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:20:55,926 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:20:55,926 \tHypothesis: Vbe Jesu rre ukpo 20 , ọ keghi do gha re emwa ni rre uwu ẹbu emwa ni rre ẹvbo nii .\n",
            "2020-01-26 01:20:55,926 Example #3\n",
            "2020-01-26 01:20:55,927 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:20:55,927 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:20:55,927 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ ma khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe .\n",
            "2020-01-26 01:20:55,928 Validation result (greedy) at epoch  31, step     4100: bleu:   6.08, loss: 66764.7422, ppl:  13.9428, duration: 47.3190s\n",
            "2020-01-26 01:21:00,410 Epoch  31: total training loss 289.78\n",
            "2020-01-26 01:21:00,411 EPOCH 32\n",
            "2020-01-26 01:21:11,799 Epoch  32 Step:     4200 Batch Loss:     1.969729 Tokens per Sec:    12092, Lr: 0.000300\n",
            "2020-01-26 01:21:58,015 Example #0\n",
            "2020-01-26 01:21:58,016 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:21:58,016 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:21:58,017 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:21:58,017 Example #1\n",
            "2020-01-26 01:21:58,017 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:21:58,017 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:21:58,017 \tHypothesis: De emwi nọ khẹke ne ima ru ?\n",
            "2020-01-26 01:21:58,018 Example #2\n",
            "2020-01-26 01:21:58,018 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:21:58,018 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:21:58,018 \tHypothesis: Vbe ọ rre ukpo 20 , ọ keghi do gha mwẹ iyayi nọ wegbe wẹẹ , ọ mwẹ emwi eso ne iran ma he miẹn vbe obọ vbe iwinna ugamwẹ ọghe ẹghẹ hia .\n",
            "2020-01-26 01:21:58,018 Example #3\n",
            "2020-01-26 01:21:58,019 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:21:58,019 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:21:58,019 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru , ne ima mieke na sẹtin gha mwẹ irẹnmwi nọ gbae vbekpae Jehova .\n",
            "2020-01-26 01:21:58,019 Validation result (greedy) at epoch  32, step     4200: bleu:   6.32, loss: 66829.3906, ppl:  13.9784, duration: 46.2203s\n",
            "2020-01-26 01:22:07,681 Epoch  32: total training loss 283.52\n",
            "2020-01-26 01:22:07,682 EPOCH 33\n",
            "2020-01-26 01:22:13,595 Epoch  33 Step:     4300 Batch Loss:     2.226443 Tokens per Sec:    12549, Lr: 0.000300\n",
            "2020-01-26 01:22:59,865 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:22:59,865 Saving new checkpoint.\n",
            "2020-01-26 01:23:01,028 Example #0\n",
            "2020-01-26 01:23:01,029 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:23:01,029 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:23:01,029 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin gha mwẹ irẹnmwi nọ dinmwi vbekpae Baibol ?\n",
            "2020-01-26 01:23:01,030 Example #1\n",
            "2020-01-26 01:23:01,030 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:23:01,030 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:23:01,031 \tHypothesis: De emwi ne Jehova ru ne ọmwa ?\n",
            "2020-01-26 01:23:01,031 Example #2\n",
            "2020-01-26 01:23:01,031 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:23:01,032 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:23:01,032 \tHypothesis: Vbe Jesu rre ukpo 20 , ọ na suẹn gha kporhu ma emwa ni ma he rọnmwẹ ọdọ vbe amwẹ .\n",
            "2020-01-26 01:23:01,032 Example #3\n",
            "2020-01-26 01:23:01,033 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:23:01,033 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:23:01,033 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru , ne ima mieke na rẹn vbene ọ ru ekpataki sẹ hẹ .\n",
            "2020-01-26 01:23:01,034 Validation result (greedy) at epoch  33, step     4300: bleu:   6.37, loss: 66457.7578, ppl:  13.7749, duration: 47.4380s\n",
            "2020-01-26 01:23:16,065 Epoch  33: total training loss 278.98\n",
            "2020-01-26 01:23:16,065 EPOCH 34\n",
            "2020-01-26 01:23:16,949 Epoch  34 Step:     4400 Batch Loss:     1.625541 Tokens per Sec:    12263, Lr: 0.000300\n",
            "2020-01-26 01:24:03,044 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:24:03,044 Saving new checkpoint.\n",
            "2020-01-26 01:24:04,238 Example #0\n",
            "2020-01-26 01:24:04,239 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:24:04,239 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:24:04,239 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:24:04,239 Example #1\n",
            "2020-01-26 01:24:04,240 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:24:04,240 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:24:04,240 \tHypothesis: De emwi nọ khẹke ne ima ru ne Jehova mieke na sẹtin gha dia egbe ọmwa ?\n",
            "2020-01-26 01:24:04,240 Example #2\n",
            "2020-01-26 01:24:04,241 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:24:04,241 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:24:04,241 \tHypothesis: Vbe ọ rre ukpo 40 , ọ keghi do gha mwẹ udinmwẹ vbe ọ da ayọn vbe otọ ẹvbo ne a lẹgbinna .\n",
            "2020-01-26 01:24:04,241 Example #3\n",
            "2020-01-26 01:24:04,241 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:24:04,242 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:24:04,242 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya sẹtin ya ahoẹmwọmwa ọghe ẹmwata ru iyobọ ne ima ya rẹn odẹ ne ima ya khaevbisẹ .\n",
            "2020-01-26 01:24:04,242 Validation result (greedy) at epoch  34, step     4400: bleu:   6.72, loss: 66373.8672, ppl:  13.7294, duration: 47.2916s\n",
            "2020-01-26 01:24:19,951 Epoch  34 Step:     4500 Batch Loss:     1.985672 Tokens per Sec:    12123, Lr: 0.000300\n",
            "2020-01-26 01:25:06,139 Example #0\n",
            "2020-01-26 01:25:06,140 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:25:06,140 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:25:06,140 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha mwẹ irẹnmwi nọ gbae vbekpae Jehova ?\n",
            "2020-01-26 01:25:06,141 Example #1\n",
            "2020-01-26 01:25:06,141 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:25:06,141 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:25:06,141 \tHypothesis: De emwi nọ khẹke ne ima ru ne Jehova mieke na gha re ọmwa nọ tọn egbe ?\n",
            "2020-01-26 01:25:06,142 Example #2\n",
            "2020-01-26 01:25:06,142 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:25:06,142 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:25:06,142 \tHypothesis: Vbe ọ rre ukpo 20 , ọ keghi do gha re ọmwa nọ da ayọn vbe otọ ẹvbo ọvbehe nọ rre otọ ẹvbo ọvbehe .\n",
            "2020-01-26 01:25:06,142 Example #3\n",
            "2020-01-26 01:25:06,143 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:25:06,143 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:25:06,143 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ ima hoẹmwẹ emwa ọvbehe , ọ khẹke ne ima gha mwẹ irẹnmwi nọ gbae vbekpae Jehova .\n",
            "2020-01-26 01:25:06,143 Validation result (greedy) at epoch  34, step     4500: bleu:   6.94, loss: 66614.5234, ppl:  13.8604, duration: 46.1920s\n",
            "2020-01-26 01:25:10,783 Epoch  34: total training loss 280.88\n",
            "2020-01-26 01:25:10,783 EPOCH 35\n",
            "2020-01-26 01:25:21,585 Epoch  35 Step:     4600 Batch Loss:     1.723932 Tokens per Sec:    12483, Lr: 0.000300\n",
            "2020-01-26 01:26:07,690 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:26:07,690 Saving new checkpoint.\n",
            "2020-01-26 01:26:08,907 Example #0\n",
            "2020-01-26 01:26:08,907 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:26:08,907 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:26:08,908 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:26:08,908 Example #1\n",
            "2020-01-26 01:26:08,908 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:26:08,908 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:26:08,908 \tHypothesis: De emwi nọ khẹke ne ima ru ?\n",
            "2020-01-26 01:26:08,908 Example #2\n",
            "2020-01-26 01:26:08,908 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:26:08,909 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:26:08,909 \tHypothesis: Vbe ọ rre ukpo 40 , e Jesu keghi do deba irẹn vbe ẹgbẹe ọre vbe ẹgbẹe ọre .\n",
            "2020-01-26 01:26:08,909 Example #3\n",
            "2020-01-26 01:26:08,909 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:26:08,909 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:26:08,909 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru , ne ima mieke na sẹtin gha mwẹ irẹnmwi nọ gbae vbekpae Jehova .\n",
            "2020-01-26 01:26:08,909 Validation result (greedy) at epoch  35, step     4600: bleu:   7.06, loss: 66329.3906, ppl:  13.7053, duration: 47.3236s\n",
            "2020-01-26 01:26:19,030 Epoch  35: total training loss 272.03\n",
            "2020-01-26 01:26:19,031 EPOCH 36\n",
            "2020-01-26 01:26:25,012 Epoch  36 Step:     4700 Batch Loss:     1.800356 Tokens per Sec:    12502, Lr: 0.000300\n",
            "2020-01-26 01:27:11,171 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:27:11,171 Saving new checkpoint.\n",
            "2020-01-26 01:27:12,370 Example #0\n",
            "2020-01-26 01:27:12,371 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:27:12,371 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:27:12,372 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:27:12,372 Example #1\n",
            "2020-01-26 01:27:12,372 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:27:12,372 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:27:12,372 \tHypothesis: De emwi ne ọmwa nọ dizigha ọyẹvbu ru ?\n",
            "2020-01-26 01:27:12,372 Example #2\n",
            "2020-01-26 01:27:12,373 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:27:12,373 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:27:12,373 \tHypothesis: Vbe ọ rre ukpo iyisẹn isẹn nibun vbe otọ ẹvbo nii , ọ keghi kpa hin owa rre .\n",
            "2020-01-26 01:27:12,373 Example #3\n",
            "2020-01-26 01:27:12,373 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:27:12,373 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:27:12,373 \tHypothesis: Ọ keghi re emwi nọ khẹke ne ima ru , ne ima mieke na sẹtin gha mwẹ irẹnmwi nọ gbae vbekpae Jehova .\n",
            "2020-01-26 01:27:12,374 Validation result (greedy) at epoch  36, step     4700: bleu:   6.93, loss: 66241.2344, ppl:  13.6577, duration: 47.3607s\n",
            "2020-01-26 01:27:27,505 Epoch  36: total training loss 267.35\n",
            "2020-01-26 01:27:27,506 EPOCH 37\n",
            "2020-01-26 01:27:28,153 Epoch  37 Step:     4800 Batch Loss:     1.962434 Tokens per Sec:    12901, Lr: 0.000300\n",
            "2020-01-26 01:28:14,439 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:28:14,440 Saving new checkpoint.\n",
            "2020-01-26 01:28:15,750 Example #0\n",
            "2020-01-26 01:28:15,751 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:28:15,751 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:28:15,751 \tHypothesis: De emwi ne ima gha ru ne ima mieke na sẹtin ya egbe ta e Baibol kevbe odẹ ne ima ya ru erria yan rẹn ?\n",
            "2020-01-26 01:28:15,751 Example #1\n",
            "2020-01-26 01:28:15,752 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:28:15,752 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:28:15,752 \tHypothesis: De emwi nọ khẹke ne ima ru ?\n",
            "2020-01-26 01:28:15,752 Example #2\n",
            "2020-01-26 01:28:15,753 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:28:15,753 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:28:15,753 \tHypothesis: Vbe Jesu rre ukpo iyisẹn ne Jesu gbe vbe eghan , ọ na vbe gha mwẹ obọ vbe iwinna ne a ru .\n",
            "2020-01-26 01:28:15,753 Example #3\n",
            "2020-01-26 01:28:15,754 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:28:15,754 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:28:15,755 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe khẹke ne ima gha mwẹ irẹnmwi nọ gbae vbekpae Jehova .\n",
            "2020-01-26 01:28:15,755 Validation result (greedy) at epoch  37, step     4800: bleu:   7.05, loss: 66121.7578, ppl:  13.5935, duration: 47.6010s\n",
            "2020-01-26 01:28:31,379 Epoch  37 Step:     4900 Batch Loss:     1.346593 Tokens per Sec:    12251, Lr: 0.000300\n",
            "2020-01-26 01:29:17,726 Example #0\n",
            "2020-01-26 01:29:17,727 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:29:17,727 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:29:17,728 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:29:17,728 Example #1\n",
            "2020-01-26 01:29:17,728 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:29:17,729 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:29:17,729 \tHypothesis: De emwi ne Jehova ru ne ọmwa nọ dizigha ọyẹvbu ?\n",
            "2020-01-26 01:29:17,729 Example #2\n",
            "2020-01-26 01:29:17,729 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:29:17,730 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:29:17,730 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ keghi do gha mwẹ akugbe vbe ẹgbẹe .\n",
            "2020-01-26 01:29:17,730 Example #3\n",
            "2020-01-26 01:29:17,731 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:29:17,731 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:29:17,731 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rhiẹre ma wẹẹ ima hoẹmwẹ emwa ọvbehe .\n",
            "2020-01-26 01:29:17,731 Validation result (greedy) at epoch  37, step     4900: bleu:   7.30, loss: 66205.3750, ppl:  13.6384, duration: 46.3517s\n",
            "2020-01-26 01:29:22,375 Epoch  37: total training loss 266.49\n",
            "2020-01-26 01:29:22,375 EPOCH 38\n",
            "2020-01-26 01:29:33,246 Epoch  38 Step:     5000 Batch Loss:     2.067524 Tokens per Sec:    12617, Lr: 0.000300\n",
            "2020-01-26 01:30:19,318 Example #0\n",
            "2020-01-26 01:30:19,319 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:30:19,319 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:30:19,320 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:30:19,320 Example #1\n",
            "2020-01-26 01:30:19,320 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:30:19,321 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:30:19,321 \tHypothesis: De emwi nọ khẹke ne ima ru ne Jehova mieke na mu egbe rriotọ ?\n",
            "2020-01-26 01:30:19,321 Example #2\n",
            "2020-01-26 01:30:19,321 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:30:19,322 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:30:19,322 \tHypothesis: Vbe Jesu rre ukpo iyisẹn nibun nẹ , ọ na gha mwẹ udinmwẹ vbe iwinna ugamwẹ ọghe iran .\n",
            "2020-01-26 01:30:19,322 Example #3\n",
            "2020-01-26 01:30:19,322 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:30:19,323 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:30:19,323 \tHypothesis: Ọ keghi re emwi nọ ru ekpataki ne ima ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe .\n",
            "2020-01-26 01:30:19,323 Validation result (greedy) at epoch  38, step     5000: bleu:   7.54, loss: 66186.0391, ppl:  13.6280, duration: 46.0761s\n",
            "2020-01-26 01:30:29,073 Epoch  38: total training loss 263.32\n",
            "2020-01-26 01:30:29,073 EPOCH 39\n",
            "2020-01-26 01:30:34,830 Epoch  39 Step:     5100 Batch Loss:     2.290383 Tokens per Sec:    12476, Lr: 0.000300\n",
            "2020-01-26 01:31:21,103 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:31:21,103 Saving new checkpoint.\n",
            "2020-01-26 01:31:22,173 Example #0\n",
            "2020-01-26 01:31:22,174 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:31:22,174 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:31:22,174 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:31:22,175 Example #1\n",
            "2020-01-26 01:31:22,175 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:31:22,175 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:31:22,176 \tHypothesis: De emwi nọ khẹke ne ima ru ?\n",
            "2020-01-26 01:31:22,176 Example #2\n",
            "2020-01-26 01:31:22,176 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:31:22,176 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:31:22,177 \tHypothesis: Vbe Jesu rre ukpo iyisẹn isẹn vbe Jesu wu , ọ keghi kpa hin owa rre .\n",
            "2020-01-26 01:31:22,177 Example #3\n",
            "2020-01-26 01:31:22,178 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:31:22,178 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:31:22,178 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rhiẹre ma wẹẹ ima hoẹmwẹ ọnrẹn kevbe wẹẹ , irẹn ẹre ọ gele mwẹ asẹ ne a ya khaevbisẹ .\n",
            "2020-01-26 01:31:22,178 Validation result (greedy) at epoch  39, step     5100: bleu:   7.57, loss: 66085.0938, ppl:  13.5738, duration: 47.3481s\n",
            "2020-01-26 01:31:37,260 Epoch  39: total training loss 259.94\n",
            "2020-01-26 01:31:37,260 EPOCH 40\n",
            "2020-01-26 01:31:37,793 Epoch  40 Step:     5200 Batch Loss:     1.875083 Tokens per Sec:    12253, Lr: 0.000300\n",
            "2020-01-26 01:32:23,977 Hooray! New best validation result [ppl]!\n",
            "2020-01-26 01:32:23,977 Saving new checkpoint.\n",
            "2020-01-26 01:32:25,085 Example #0\n",
            "2020-01-26 01:32:25,086 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:32:25,086 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:32:25,087 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ irẹnmwi nọ gbae vbekpae Jehova ?\n",
            "2020-01-26 01:32:25,087 Example #1\n",
            "2020-01-26 01:32:25,087 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:32:25,087 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:32:25,087 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ dizigha ọyẹvbu ru ?\n",
            "2020-01-26 01:32:25,087 Example #2\n",
            "2020-01-26 01:32:25,087 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:32:25,088 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:32:25,088 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ keghi do gha mwẹ egbe iziro ọghe alama oghẹ .\n",
            "2020-01-26 01:32:25,088 Example #3\n",
            "2020-01-26 01:32:25,088 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:32:25,088 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:32:25,088 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rẹn wẹẹ , irẹn ẹre ọ mwẹ asẹ ne ima ya khaevbisẹ .\n",
            "2020-01-26 01:32:25,088 Validation result (greedy) at epoch  40, step     5200: bleu:   7.68, loss: 65724.9453, ppl:  13.3822, duration: 47.2945s\n",
            "2020-01-26 01:32:40,912 Epoch  40 Step:     5300 Batch Loss:     1.226460 Tokens per Sec:    12401, Lr: 0.000300\n",
            "2020-01-26 01:33:26,849 Example #0\n",
            "2020-01-26 01:33:26,850 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:33:26,850 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:33:26,851 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ irẹnmwi nọ gbae vbekpae Jehova ?\n",
            "2020-01-26 01:33:26,851 Example #1\n",
            "2020-01-26 01:33:26,851 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:33:26,851 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:33:26,852 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ dizigha ọyẹvbu ?\n",
            "2020-01-26 01:33:26,852 Example #2\n",
            "2020-01-26 01:33:26,852 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:33:26,852 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:33:26,852 \tHypothesis: Vbe Jesu rre agbọn , ọ keghi do gha mwẹ udinmwẹ vbe ẹgbẹe ọre vbe ẹgbẹe ọre .\n",
            "2020-01-26 01:33:26,853 Example #3\n",
            "2020-01-26 01:33:26,853 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:33:26,853 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:33:26,853 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya ru iyobọ ne ima ya rẹn wẹẹ , ọ mwẹ asẹ ne a ya ru emwi nọ maan , ne ima mieke na sẹtin ya ekhọe hia hoẹmwẹ ọnrẹn .\n",
            "2020-01-26 01:33:26,854 Validation result (greedy) at epoch  40, step     5300: bleu:   7.24, loss: 66018.1172, ppl:  13.5380, duration: 45.9409s\n",
            "2020-01-26 01:33:31,392 Epoch  40: total training loss 254.06\n",
            "2020-01-26 01:33:31,392 EPOCH 41\n",
            "2020-01-26 01:33:42,109 Epoch  41 Step:     5400 Batch Loss:     1.474907 Tokens per Sec:    12506, Lr: 0.000300\n",
            "2020-01-26 01:34:28,208 Example #0\n",
            "2020-01-26 01:34:28,210 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:34:28,210 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:34:28,210 \tHypothesis: De emwi ne ima khian ru ne ima mieke na gha mwẹ irẹnmwi nọ gbae vbekpae Jehova ?\n",
            "2020-01-26 01:34:28,210 Example #1\n",
            "2020-01-26 01:34:28,211 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:34:28,211 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:34:28,211 \tHypothesis: De emwi nọ khẹke ne ima ru ?\n",
            "2020-01-26 01:34:28,211 Example #2\n",
            "2020-01-26 01:34:28,212 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:34:28,212 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:34:28,212 \tHypothesis: Vbe Jesu rre agbọn , ọ ma ghi gha mwẹ egbe iziro vbenian .\n",
            "2020-01-26 01:34:28,212 Example #3\n",
            "2020-01-26 01:34:28,213 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:34:28,213 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:34:28,213 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ irẹnmwi sayọ .\n",
            "2020-01-26 01:34:28,213 Validation result (greedy) at epoch  41, step     5400: bleu:   7.81, loss: 66345.3359, ppl:  13.7139, duration: 46.1033s\n",
            "2020-01-26 01:34:38,026 Epoch  41: total training loss 253.76\n",
            "2020-01-26 01:34:38,026 EPOCH 42\n",
            "2020-01-26 01:34:43,624 Epoch  42 Step:     5500 Batch Loss:     1.144276 Tokens per Sec:    12672, Lr: 0.000300\n",
            "2020-01-26 01:35:29,886 Example #0\n",
            "2020-01-26 01:35:29,888 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:35:29,888 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:35:29,888 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:35:29,888 Example #1\n",
            "2020-01-26 01:35:29,889 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:35:29,889 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:35:29,889 \tHypothesis: De aro nọ khẹke ne ọmwa nọ tọn egbe mu ?\n",
            "2020-01-26 01:35:29,889 Example #2\n",
            "2020-01-26 01:35:29,890 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:35:29,890 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:35:29,890 \tHypothesis: Vbe ọ rre ukpo iyisẹn , ọ ma he kpẹ vba , ẹgbẹe ọre vbe ọ rre uwu ẹgbẹe .\n",
            "2020-01-26 01:35:29,890 Example #3\n",
            "2020-01-26 01:35:29,890 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:35:29,891 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:35:29,891 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya sẹtin ya ima rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe ọguọmwaziro nọ wegbe .\n",
            "2020-01-26 01:35:29,891 Validation result (greedy) at epoch  42, step     5500: bleu:   7.78, loss: 66119.8984, ppl:  13.5925, duration: 46.2667s\n",
            "2020-01-26 01:35:44,962 Epoch  42: total training loss 249.61\n",
            "2020-01-26 01:35:44,963 EPOCH 43\n",
            "2020-01-26 01:35:45,310 Epoch  43 Step:     5600 Batch Loss:     2.021386 Tokens per Sec:    12742, Lr: 0.000300\n",
            "2020-01-26 01:36:31,470 Example #0\n",
            "2020-01-26 01:36:31,471 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:36:31,472 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:36:31,472 \tHypothesis: De emwi ne ima khian ru ne ima mieke na gha mwẹ irẹnmwi sayọ vbe Baibol ?\n",
            "2020-01-26 01:36:31,472 Example #1\n",
            "2020-01-26 01:36:31,473 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:36:31,473 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:36:31,473 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe iran ?\n",
            "2020-01-26 01:36:31,473 Example #2\n",
            "2020-01-26 01:36:31,474 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:36:31,474 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:36:31,474 \tHypothesis: Vbe ọ rre ukpo ihinrọn , ọ ma he kpẹ gbe , vbe ọ ghi kpa hin otu e Jehova rre .\n",
            "2020-01-26 01:36:31,474 Example #3\n",
            "2020-01-26 01:36:31,475 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:36:31,475 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:36:31,475 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya sẹtin ya egbe tae Jehova , ọ vbe ya ima rẹn odẹ ne ima ya sẹtin dia egbe ima .\n",
            "2020-01-26 01:36:31,476 Validation result (greedy) at epoch  43, step     5600: bleu:   7.77, loss: 66039.3672, ppl:  13.5493, duration: 46.1652s\n",
            "2020-01-26 01:36:46,757 Epoch  43 Step:     5700 Batch Loss:     2.332794 Tokens per Sec:    12631, Lr: 0.000300\n",
            "2020-01-26 01:37:32,957 Example #0\n",
            "2020-01-26 01:37:32,958 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:37:32,959 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:37:32,959 \tHypothesis: De odẹ eso ne ima gha ya sẹtin ru erria yan ilele ni rre Baibol kevbe odẹ ne ima ya loo arrọọ ọghomwa ?\n",
            "2020-01-26 01:37:32,959 Example #1\n",
            "2020-01-26 01:37:32,960 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:37:32,960 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:37:32,960 \tHypothesis: De aro nọ khẹke ne ọmwa nọ dizigha ọyẹvbu ?\n",
            "2020-01-26 01:37:32,960 Example #2\n",
            "2020-01-26 01:37:32,961 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:37:32,961 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:37:32,961 \tHypothesis: Vbe ọ rre ukpo iyisẹn , ọ na gha mwẹ udinmwẹ vbe ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 01:37:32,961 Example #3\n",
            "2020-01-26 01:37:32,962 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:37:32,962 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:37:32,962 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ ma khẹke ne ima gha mwẹ ẹwaẹn nọ ke obọ Osanobua rre .\n",
            "2020-01-26 01:37:32,963 Validation result (greedy) at epoch  43, step     5700: bleu:   7.76, loss: 65805.8125, ppl:  13.4250, duration: 46.2049s\n",
            "2020-01-26 01:37:37,701 Epoch  43: total training loss 246.60\n",
            "2020-01-26 01:37:37,701 EPOCH 44\n",
            "2020-01-26 01:37:48,254 Epoch  44 Step:     5800 Batch Loss:     1.956841 Tokens per Sec:    12615, Lr: 0.000300\n",
            "2020-01-26 01:38:34,035 Example #0\n",
            "2020-01-26 01:38:34,036 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:38:34,036 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:38:34,036 \tHypothesis: De emwi nọ khẹke ne ima ru yan rẹn vbe ima ghaa ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:38:34,036 Example #1\n",
            "2020-01-26 01:38:34,036 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:38:34,037 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:38:34,037 \tHypothesis: De uyi ne Jehova mu ye ọmwa egbe ?\n",
            "2020-01-26 01:38:34,037 Example #2\n",
            "2020-01-26 01:38:34,037 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:38:34,037 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:38:34,038 \tHypothesis: Vbe ọ rre ukpo eso , ọ na do gha re ẹgbẹe nọ ka gha re Osẹe Jehova .\n",
            "2020-01-26 01:38:34,038 Example #3\n",
            "2020-01-26 01:38:34,038 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:38:34,038 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:38:34,038 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn emwi nọ gbae ne ima ru , ọ na vbe ya ima rẹn odẹ nọ gbae .\n",
            "2020-01-26 01:38:34,038 Validation result (greedy) at epoch  44, step     5800: bleu:   8.20, loss: 66231.1016, ppl:  13.6522, duration: 45.7838s\n",
            "2020-01-26 01:38:44,250 Epoch  44: total training loss 241.95\n",
            "2020-01-26 01:38:44,250 EPOCH 45\n",
            "2020-01-26 01:38:49,525 Epoch  45 Step:     5900 Batch Loss:     1.903749 Tokens per Sec:    12697, Lr: 0.000210\n",
            "2020-01-26 01:39:35,701 Example #0\n",
            "2020-01-26 01:39:35,702 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:39:35,703 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:39:35,703 \tHypothesis: De odẹ eso ne ima gha ya sẹtin ru erria yan emwi ne ima tie vbe Baibol kevbe odẹ ne ima ya ru erria yan rẹn ?\n",
            "2020-01-26 01:39:35,703 Example #1\n",
            "2020-01-26 01:39:35,703 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:39:35,703 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:39:35,704 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ tọn egbe mu ?\n",
            "2020-01-26 01:39:35,704 Example #2\n",
            "2020-01-26 01:39:35,704 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:39:35,704 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:39:35,704 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ keghi kpa hin owa rre .\n",
            "2020-01-26 01:39:35,705 Example #3\n",
            "2020-01-26 01:39:35,705 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:39:35,705 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:39:35,705 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya sẹtin ya ima rẹn wẹẹ , ọ mwẹ irẹnmwi nọ gbae vbekpae Jehova , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne ọ ya khaevbisẹ .\n",
            "2020-01-26 01:39:35,706 Validation result (greedy) at epoch  45, step     5900: bleu:   8.62, loss: 65995.3203, ppl:  13.5258, duration: 46.1805s\n",
            "2020-01-26 01:39:51,042 Epoch  45: total training loss 234.62\n",
            "2020-01-26 01:39:51,042 EPOCH 46\n",
            "2020-01-26 01:39:51,225 Epoch  46 Step:     6000 Batch Loss:     1.639863 Tokens per Sec:    11948, Lr: 0.000210\n",
            "2020-01-26 01:40:37,218 Example #0\n",
            "2020-01-26 01:40:37,219 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:40:37,220 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:40:37,220 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:40:37,220 Example #1\n",
            "2020-01-26 01:40:37,220 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:40:37,220 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:40:37,221 \tHypothesis: De aro nọ khẹke ne ọmwa nọ tọn egbe mu ?\n",
            "2020-01-26 01:40:37,221 Example #2\n",
            "2020-01-26 01:40:37,221 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:40:37,221 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:40:37,221 \tHypothesis: Vbe ọ rre ukpo ihinrọn , ọ keghi kpa hin otu e Jesu rre vbe ẹgbẹe ọre kevbe ẹgbẹe .\n",
            "2020-01-26 01:40:37,221 Example #3\n",
            "2020-01-26 01:40:37,222 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:40:37,222 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:40:37,222 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya sẹtin ya ima rẹn emwi nọ ru ekpataki sẹ hẹ .\n",
            "2020-01-26 01:40:37,222 Validation result (greedy) at epoch  46, step     6000: bleu:   8.40, loss: 66038.6641, ppl:  13.5490, duration: 45.9965s\n",
            "2020-01-26 01:40:52,827 Epoch  46 Step:     6100 Batch Loss:     1.714107 Tokens per Sec:    12450, Lr: 0.000210\n",
            "2020-01-26 01:41:38,784 Example #0\n",
            "2020-01-26 01:41:38,785 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:41:38,785 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:41:38,785 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:41:38,785 Example #1\n",
            "2020-01-26 01:41:38,786 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:41:38,786 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:41:38,786 \tHypothesis: De emwi nọ khẹke ne ima kọ ye orhiọn ?\n",
            "2020-01-26 01:41:38,786 Example #2\n",
            "2020-01-26 01:41:38,787 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:41:38,787 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:41:38,787 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun nẹ , ọ ma he kpẹ gbe , ẹgbẹe ọre kevbe ẹgbẹe ọre na gha mwẹ obọ vbe iwinna nọ khian vbe ẹdẹ okiekie .\n",
            "2020-01-26 01:41:38,788 Example #3\n",
            "2020-01-26 01:41:38,788 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:41:38,788 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:41:38,788 \tHypothesis: Ọ keghi re odẹ ọkpa ne ima ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe .\n",
            "2020-01-26 01:41:38,789 Validation result (greedy) at epoch  46, step     6100: bleu:   8.25, loss: 66013.3047, ppl:  13.5354, duration: 45.9612s\n",
            "2020-01-26 01:41:43,754 Epoch  46: total training loss 231.51\n",
            "2020-01-26 01:41:43,755 EPOCH 47\n",
            "2020-01-26 01:41:54,375 Epoch  47 Step:     6200 Batch Loss:     1.680230 Tokens per Sec:    12381, Lr: 0.000210\n",
            "2020-01-26 01:42:40,341 Example #0\n",
            "2020-01-26 01:42:40,342 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:42:40,343 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:42:40,343 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ irẹnmwi nọ dinmwi vbekpae Baibol ?\n",
            "2020-01-26 01:42:40,343 Example #1\n",
            "2020-01-26 01:42:40,343 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:42:40,343 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:42:40,344 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:42:40,344 Example #2\n",
            "2020-01-26 01:42:40,344 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:42:40,344 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:42:40,344 \tHypothesis: Vbe Jesu rre agbọn , ọ keghi do bẹghe ẹre wẹẹ , ọ mwẹ ẹghẹ ne irẹn ya gha re ẹgbẹe kevbe ẹgbẹe ọkpa .\n",
            "2020-01-26 01:42:40,345 Example #3\n",
            "2020-01-26 01:42:40,345 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:42:40,345 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:42:40,345 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe .\n",
            "2020-01-26 01:42:40,346 Validation result (greedy) at epoch  47, step     6200: bleu:   8.23, loss: 66171.4922, ppl:  13.6202, duration: 45.9701s\n",
            "2020-01-26 01:42:50,374 Epoch  47: total training loss 229.24\n",
            "2020-01-26 01:42:50,374 EPOCH 48\n",
            "2020-01-26 01:42:55,745 Epoch  48 Step:     6300 Batch Loss:     1.619237 Tokens per Sec:    12494, Lr: 0.000210\n",
            "2020-01-26 01:43:41,885 Example #0\n",
            "2020-01-26 01:43:41,886 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:43:41,886 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:43:41,886 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:43:41,887 Example #1\n",
            "2020-01-26 01:43:41,887 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:43:41,887 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:43:41,887 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ dizigha ọyẹvbu ru ?\n",
            "2020-01-26 01:43:41,888 Example #2\n",
            "2020-01-26 01:43:41,888 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:43:41,888 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:43:41,888 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre vbe ẹgbẹe ọre vbe ẹgbẹe ọre vbe ẹgbẹe ọre vbe Sameria .\n",
            "2020-01-26 01:43:41,888 Example #3\n",
            "2020-01-26 01:43:41,889 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:43:41,889 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:43:41,889 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne Jehova ya rhie ahoẹmwọmwa ne ima , ne ima mieke na sẹtin do gha mwẹ akueyi ọghe ima .\n",
            "2020-01-26 01:43:41,889 Validation result (greedy) at epoch  48, step     6300: bleu:   8.27, loss: 66477.5781, ppl:  13.7857, duration: 46.1436s\n",
            "2020-01-26 01:43:57,226 Epoch  48: total training loss 227.44\n",
            "2020-01-26 01:43:57,226 EPOCH 49\n",
            "2020-01-26 01:43:57,553 Epoch  49 Step:     6400 Batch Loss:     1.872538 Tokens per Sec:    12048, Lr: 0.000210\n",
            "2020-01-26 01:44:43,579 Example #0\n",
            "2020-01-26 01:44:43,580 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:44:43,581 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:44:43,581 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:44:43,581 Example #1\n",
            "2020-01-26 01:44:43,582 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:44:43,582 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:44:43,582 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:44:43,582 Example #2\n",
            "2020-01-26 01:44:43,583 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:44:43,583 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:44:43,583 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ keghi kpa emwa odin odin rhunmwuda iran ma rẹn odẹ ọghe ẹmwata .\n",
            "2020-01-26 01:44:43,583 Example #3\n",
            "2020-01-26 01:44:43,583 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:44:43,584 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:44:43,584 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa ima .\n",
            "2020-01-26 01:44:43,584 Validation result (greedy) at epoch  49, step     6400: bleu:   8.16, loss: 66052.2812, ppl:  13.5562, duration: 46.0308s\n",
            "2020-01-26 01:44:59,095 Epoch  49 Step:     6500 Batch Loss:     1.821737 Tokens per Sec:    12626, Lr: 0.000147\n",
            "2020-01-26 01:45:45,147 Example #0\n",
            "2020-01-26 01:45:45,148 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:45:45,149 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:45:45,149 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:45:45,149 Example #1\n",
            "2020-01-26 01:45:45,150 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:45:45,150 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:45:45,150 \tHypothesis: De aro nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:45:45,150 Example #2\n",
            "2020-01-26 01:45:45,151 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:45:45,151 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:45:45,151 \tHypothesis: Vbe ọ rre ukpo eso , ọ keghi kpa e Jesu odin vbe ẹgbẹe ọre kevbe ẹgbẹe ọkpa .\n",
            "2020-01-26 01:45:45,151 Example #3\n",
            "2020-01-26 01:45:45,152 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:45:45,152 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:45:45,152 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe khẹke ne ima gha mwẹ irẹnmwi nọ dinmwi vbekpae Jehova .\n",
            "2020-01-26 01:45:45,152 Validation result (greedy) at epoch  49, step     6500: bleu:   8.64, loss: 66314.0781, ppl:  13.6970, duration: 46.0564s\n",
            "2020-01-26 01:45:49,952 Epoch  49: total training loss 222.69\n",
            "2020-01-26 01:45:49,953 EPOCH 50\n",
            "2020-01-26 01:46:00,621 Epoch  50 Step:     6600 Batch Loss:     1.401509 Tokens per Sec:    12598, Lr: 0.000147\n",
            "2020-01-26 01:46:46,806 Example #0\n",
            "2020-01-26 01:46:46,807 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:46:46,807 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:46:46,807 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:46:46,808 Example #1\n",
            "2020-01-26 01:46:46,808 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:46:46,808 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:46:46,808 \tHypothesis: De aro nọ khẹke ne ọmwa ya egbe ta e Jehova ?\n",
            "2020-01-26 01:46:46,809 Example #2\n",
            "2020-01-26 01:46:46,809 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:46:46,809 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:46:46,809 \tHypothesis: Vbe Jesu rre agbọn , ọ keghi kpa hin owa rre , ọ na gha re okpọmwa kevbe ẹgbẹe kevbe ẹgbẹe .\n",
            "2020-01-26 01:46:46,810 Example #3\n",
            "2020-01-26 01:46:46,810 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:46:46,810 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:46:46,810 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe khẹke ne ima gha mwẹ irẹnmwi nọ dinmwi vbekpae Osanobua .\n",
            "2020-01-26 01:46:46,811 Validation result (greedy) at epoch  50, step     6600: bleu:   8.60, loss: 66391.7500, ppl:  13.7391, duration: 46.1894s\n",
            "2020-01-26 01:46:56,742 Epoch  50: total training loss 219.80\n",
            "2020-01-26 01:46:56,742 EPOCH 51\n",
            "2020-01-26 01:47:02,368 Epoch  51 Step:     6700 Batch Loss:     1.341244 Tokens per Sec:    13073, Lr: 0.000147\n",
            "2020-01-26 01:47:48,442 Example #0\n",
            "2020-01-26 01:47:48,443 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:47:48,443 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:47:48,443 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:47:48,444 Example #1\n",
            "2020-01-26 01:47:48,444 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:47:48,444 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:47:48,444 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:47:48,444 Example #2\n",
            "2020-01-26 01:47:48,444 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:47:48,445 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:47:48,445 \tHypothesis: Vbe Jesu rre agbọn , ọ keghi kpa e Jesu odin vbe ẹgbẹe ọre kevbe ẹgbẹe ọkpa vbuwe ẹgbẹe .\n",
            "2020-01-26 01:47:48,445 Example #3\n",
            "2020-01-26 01:47:48,445 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:47:48,446 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:47:48,446 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 01:47:48,446 Validation result (greedy) at epoch  51, step     6700: bleu:   8.53, loss: 66471.2812, ppl:  13.7823, duration: 46.0767s\n",
            "2020-01-26 01:48:03,474 Epoch  51: total training loss 217.13\n",
            "2020-01-26 01:48:03,474 EPOCH 52\n",
            "2020-01-26 01:48:04,074 Epoch  52 Step:     6800 Batch Loss:     1.794706 Tokens per Sec:    12109, Lr: 0.000147\n",
            "2020-01-26 01:48:50,314 Example #0\n",
            "2020-01-26 01:48:50,315 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:48:50,316 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:48:50,316 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha mwẹ irẹnmwi nọ dinmwi vbekpae Baibol ?\n",
            "2020-01-26 01:48:50,316 Example #1\n",
            "2020-01-26 01:48:50,316 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:48:50,317 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:48:50,317 \tHypothesis: De aro nọ khẹke ne ọmwa nọ tọn egbe mu ?\n",
            "2020-01-26 01:48:50,317 Example #2\n",
            "2020-01-26 01:48:50,317 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:48:50,317 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:48:50,318 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre rre ukpo ihinrọn , ẹgbẹe ọre keghi mu egbe rriotọ .\n",
            "2020-01-26 01:48:50,318 Example #3\n",
            "2020-01-26 01:48:50,318 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:48:50,318 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:48:50,318 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn emwi nọ gbae nọ ya ima mwẹ ilẹkẹtin wẹẹ , ọ gele rẹn emwi nọ ru ekpataki .\n",
            "2020-01-26 01:48:50,319 Validation result (greedy) at epoch  52, step     6800: bleu:   8.54, loss: 66387.0781, ppl:  13.7365, duration: 46.2446s\n",
            "2020-01-26 01:49:05,746 Epoch  52 Step:     6900 Batch Loss:     1.950410 Tokens per Sec:    12456, Lr: 0.000147\n",
            "2020-01-26 01:49:51,722 Example #0\n",
            "2020-01-26 01:49:51,723 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:49:51,723 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:49:51,724 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:49:51,724 Example #1\n",
            "2020-01-26 01:49:51,724 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:49:51,724 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:49:51,725 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:49:51,725 Example #2\n",
            "2020-01-26 01:49:51,725 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:49:51,725 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:49:51,725 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe Jesu wu , ọ keghi kpa hin otu e Jehova rre .\n",
            "2020-01-26 01:49:51,726 Example #3\n",
            "2020-01-26 01:49:51,726 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:49:51,726 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:49:51,726 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya sẹtin rhie ahoẹmwọmwa ne ima , ne ima mieke na sẹtin mudia gbain vbe odẹ ọghe orhiọn .\n",
            "2020-01-26 01:49:51,727 Validation result (greedy) at epoch  52, step     6900: bleu:   8.65, loss: 66540.1953, ppl:  13.8198, duration: 45.9801s\n",
            "2020-01-26 01:49:56,439 Epoch  52: total training loss 217.88\n",
            "2020-01-26 01:49:56,439 EPOCH 53\n",
            "2020-01-26 01:50:07,175 Epoch  53 Step:     7000 Batch Loss:     1.952455 Tokens per Sec:    12388, Lr: 0.000147\n",
            "2020-01-26 01:50:53,447 Example #0\n",
            "2020-01-26 01:50:53,447 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:50:53,447 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:50:53,448 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:50:53,448 Example #1\n",
            "2020-01-26 01:50:53,448 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:50:53,448 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:50:53,448 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe rriotọ ?\n",
            "2020-01-26 01:50:53,448 Example #2\n",
            "2020-01-26 01:50:53,449 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:50:53,449 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:50:53,449 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre ẹdẹ ọkpa , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 01:50:53,449 Example #3\n",
            "2020-01-26 01:50:53,449 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:50:53,449 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:50:53,449 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 01:50:53,449 Validation result (greedy) at epoch  53, step     7000: bleu:   8.80, loss: 66639.5391, ppl:  13.8741, duration: 46.2737s\n",
            "2020-01-26 01:51:03,413 Epoch  53: total training loss 217.11\n",
            "2020-01-26 01:51:03,413 EPOCH 54\n",
            "2020-01-26 01:51:08,748 Epoch  54 Step:     7100 Batch Loss:     0.833282 Tokens per Sec:    12782, Lr: 0.000103\n",
            "2020-01-26 01:51:54,843 Example #0\n",
            "2020-01-26 01:51:54,845 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:51:54,845 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:51:54,845 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:51:54,845 Example #1\n",
            "2020-01-26 01:51:54,846 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:51:54,846 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:51:54,846 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:51:54,847 Example #2\n",
            "2020-01-26 01:51:54,847 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:51:54,847 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:51:54,847 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ na gha re ẹgbẹe ọre vbe ẹgbẹe ọre kevbe ẹgbẹe .\n",
            "2020-01-26 01:51:54,847 Example #3\n",
            "2020-01-26 01:51:54,848 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:51:54,848 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:51:54,848 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn emwi nọ ru ekpataki sẹ hẹ ne ima mieke na sẹtin sinmwi oseghe ne ọdakha ọghe Jehova .\n",
            "2020-01-26 01:51:54,849 Validation result (greedy) at epoch  54, step     7100: bleu:   8.48, loss: 66615.6953, ppl:  13.8610, duration: 46.1004s\n",
            "2020-01-26 01:52:09,838 Epoch  54: total training loss 209.91\n",
            "2020-01-26 01:52:09,839 EPOCH 55\n",
            "2020-01-26 01:52:10,309 Epoch  55 Step:     7200 Batch Loss:     1.393627 Tokens per Sec:    11421, Lr: 0.000103\n",
            "2020-01-26 01:52:56,463 Example #0\n",
            "2020-01-26 01:52:56,463 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:52:56,464 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:52:56,464 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:52:56,464 Example #1\n",
            "2020-01-26 01:52:56,465 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:52:56,465 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:52:56,465 \tHypothesis: De aro nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:52:56,465 Example #2\n",
            "2020-01-26 01:52:56,465 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:52:56,466 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:52:56,466 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre vbe ẹgbẹe ọre .\n",
            "2020-01-26 01:52:56,466 Example #3\n",
            "2020-01-26 01:52:56,467 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:52:56,467 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:52:56,467 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe khẹke ne ima gha mwẹ ẹwaẹn nọ ke obọ Osanobua rre .\n",
            "2020-01-26 01:52:56,467 Validation result (greedy) at epoch  55, step     7200: bleu:   8.67, loss: 66593.6094, ppl:  13.8490, duration: 46.1574s\n",
            "2020-01-26 01:53:11,882 Epoch  55 Step:     7300 Batch Loss:     1.750369 Tokens per Sec:    12578, Lr: 0.000103\n",
            "2020-01-26 01:53:58,249 Example #0\n",
            "2020-01-26 01:53:58,250 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:53:58,250 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:53:58,250 \tHypothesis: De emwi nọ khẹke ne ima ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:53:58,250 Example #1\n",
            "2020-01-26 01:53:58,251 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:53:58,251 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:53:58,251 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 01:53:58,251 Example #2\n",
            "2020-01-26 01:53:58,252 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:53:58,252 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:53:58,252 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre vbe ẹgbẹe ọre kpa hin owa rre .\n",
            "2020-01-26 01:53:58,253 Example #3\n",
            "2020-01-26 01:53:58,253 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:53:58,253 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:53:58,253 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ ne ima .\n",
            "2020-01-26 01:53:58,253 Validation result (greedy) at epoch  55, step     7300: bleu:   8.49, loss: 66560.3906, ppl:  13.8308, duration: 46.3711s\n",
            "2020-01-26 01:54:02,999 Epoch  55: total training loss 209.11\n",
            "2020-01-26 01:54:02,999 EPOCH 56\n",
            "2020-01-26 01:54:13,861 Epoch  56 Step:     7400 Batch Loss:     0.840035 Tokens per Sec:    12511, Lr: 0.000103\n",
            "2020-01-26 01:54:59,889 Example #0\n",
            "2020-01-26 01:54:59,890 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:54:59,890 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:54:59,890 \tHypothesis: De emwi ne ima gha ru ne ima mieke na gha mwẹ alaghodaro vbe odẹ ọghe orhiọn ?\n",
            "2020-01-26 01:54:59,891 Example #1\n",
            "2020-01-26 01:54:59,891 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:54:59,891 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:54:59,891 \tHypothesis: De emwi ne emwa ọvbehe miẹn ?\n",
            "2020-01-26 01:54:59,892 Example #2\n",
            "2020-01-26 01:54:59,892 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:54:59,892 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:54:59,893 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre vbe ẹgbẹe ọre .\n",
            "2020-01-26 01:54:59,893 Example #3\n",
            "2020-01-26 01:54:59,893 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:54:59,893 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:54:59,894 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 01:54:59,894 Validation result (greedy) at epoch  56, step     7400: bleu:   8.89, loss: 66691.0156, ppl:  13.9023, duration: 46.0321s\n",
            "2020-01-26 01:55:09,625 Epoch  56: total training loss 208.81\n",
            "2020-01-26 01:55:09,625 EPOCH 57\n",
            "2020-01-26 01:55:15,328 Epoch  57 Step:     7500 Batch Loss:     1.831252 Tokens per Sec:    12311, Lr: 0.000103\n",
            "2020-01-26 01:56:01,345 Example #0\n",
            "2020-01-26 01:56:01,346 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:56:01,346 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:56:01,346 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:56:01,347 Example #1\n",
            "2020-01-26 01:56:01,347 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:56:01,347 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:56:01,347 \tHypothesis: De egbe iziro nọ ma gba ?\n",
            "2020-01-26 01:56:01,348 Example #2\n",
            "2020-01-26 01:56:01,348 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:56:01,348 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:56:01,349 \tHypothesis: Vbe ọ rre ukpo ihinrọn , ọ ma he kpẹ gbe , ẹgbẹe ọre kevbe ẹgbẹe ọre kevbe ẹgbẹe ọre kevbe nọfua vbe iko .\n",
            "2020-01-26 01:56:01,349 Example #3\n",
            "2020-01-26 01:56:01,349 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:56:01,350 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:56:01,350 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 01:56:01,350 Validation result (greedy) at epoch  57, step     7500: bleu:   8.74, loss: 66849.8125, ppl:  13.9897, duration: 46.0213s\n",
            "2020-01-26 01:56:16,335 Epoch  57: total training loss 209.07\n",
            "2020-01-26 01:56:16,336 EPOCH 58\n",
            "2020-01-26 01:56:16,873 Epoch  58 Step:     7600 Batch Loss:     1.867052 Tokens per Sec:    12559, Lr: 0.000103\n",
            "2020-01-26 01:57:03,080 Example #0\n",
            "2020-01-26 01:57:03,081 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:57:03,082 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:57:03,082 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:57:03,082 Example #1\n",
            "2020-01-26 01:57:03,083 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:57:03,083 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:57:03,083 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 01:57:03,083 Example #2\n",
            "2020-01-26 01:57:03,084 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:57:03,084 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:57:03,084 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre vbe ẹgbẹe ọre ye rre uwu ẹgbẹe .\n",
            "2020-01-26 01:57:03,084 Example #3\n",
            "2020-01-26 01:57:03,085 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:57:03,085 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:57:03,085 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 01:57:03,085 Validation result (greedy) at epoch  58, step     7600: bleu:   8.73, loss: 66799.5234, ppl:  13.9620, duration: 46.2121s\n",
            "2020-01-26 01:57:18,411 Epoch  58 Step:     7700 Batch Loss:     1.534190 Tokens per Sec:    12922, Lr: 0.000072\n",
            "2020-01-26 01:58:04,493 Example #0\n",
            "2020-01-26 01:58:04,495 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:58:04,495 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:58:04,495 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:58:04,495 Example #1\n",
            "2020-01-26 01:58:04,496 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:58:04,497 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:58:04,497 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 01:58:04,497 Example #2\n",
            "2020-01-26 01:58:04,498 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:58:04,498 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:58:04,498 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 01:58:04,498 Example #3\n",
            "2020-01-26 01:58:04,499 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:58:04,499 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:58:04,499 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ ne ima .\n",
            "2020-01-26 01:58:04,499 Validation result (greedy) at epoch  58, step     7700: bleu:   8.52, loss: 66923.1094, ppl:  14.0302, duration: 46.0873s\n",
            "2020-01-26 01:58:09,090 Epoch  58: total training loss 204.89\n",
            "2020-01-26 01:58:09,090 EPOCH 59\n",
            "2020-01-26 01:58:19,792 Epoch  59 Step:     7800 Batch Loss:     1.386724 Tokens per Sec:    12556, Lr: 0.000072\n",
            "2020-01-26 01:59:05,977 Example #0\n",
            "2020-01-26 01:59:05,978 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 01:59:05,978 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 01:59:05,978 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 01:59:05,979 Example #1\n",
            "2020-01-26 01:59:05,979 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 01:59:05,979 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 01:59:05,980 \tHypothesis: De emwi nọ khẹke ne ima kọ ye orhiọn ?\n",
            "2020-01-26 01:59:05,980 Example #2\n",
            "2020-01-26 01:59:05,980 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 01:59:05,980 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 01:59:05,981 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre vbe ẹgbẹe ọre ye rre uwu ẹgbẹe .\n",
            "2020-01-26 01:59:05,981 Example #3\n",
            "2020-01-26 01:59:05,981 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 01:59:05,981 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 01:59:05,982 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ ne ima .\n",
            "2020-01-26 01:59:05,982 Validation result (greedy) at epoch  59, step     7800: bleu:   8.48, loss: 66943.3047, ppl:  14.0414, duration: 46.1891s\n",
            "2020-01-26 01:59:15,628 Epoch  59: total training loss 204.74\n",
            "2020-01-26 01:59:15,628 EPOCH 60\n",
            "2020-01-26 01:59:21,182 Epoch  60 Step:     7900 Batch Loss:     1.230370 Tokens per Sec:    12658, Lr: 0.000072\n",
            "2020-01-26 02:00:07,298 Example #0\n",
            "2020-01-26 02:00:07,299 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:00:07,299 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:00:07,299 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:00:07,299 Example #1\n",
            "2020-01-26 02:00:07,300 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:00:07,300 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:00:07,300 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 02:00:07,300 Example #2\n",
            "2020-01-26 02:00:07,300 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:00:07,300 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:00:07,301 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre . Ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe Milan .\n",
            "2020-01-26 02:00:07,301 Example #3\n",
            "2020-01-26 02:00:07,301 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:00:07,301 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:00:07,301 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:00:07,301 Validation result (greedy) at epoch  60, step     7900: bleu:   9.12, loss: 66957.1406, ppl:  14.0491, duration: 46.1190s\n",
            "2020-01-26 02:00:22,325 Epoch  60: total training loss 204.67\n",
            "2020-01-26 02:00:22,326 EPOCH 61\n",
            "2020-01-26 02:00:22,825 Epoch  61 Step:     8000 Batch Loss:     1.206385 Tokens per Sec:    10940, Lr: 0.000072\n",
            "2020-01-26 02:01:09,006 Example #0\n",
            "2020-01-26 02:01:09,007 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:01:09,007 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:01:09,007 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:01:09,007 Example #1\n",
            "2020-01-26 02:01:09,008 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:01:09,008 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:01:09,008 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:01:09,008 Example #2\n",
            "2020-01-26 02:01:09,009 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:01:09,009 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:01:09,009 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe , ẹgbẹe ọre kevbe ẹgbẹe ọre kevbe ẹgbẹe ọre kevbe ẹgbẹe ọre na gha kpokpo egbe .\n",
            "2020-01-26 02:01:09,009 Example #3\n",
            "2020-01-26 02:01:09,010 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:01:09,010 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:01:09,010 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:01:09,011 Validation result (greedy) at epoch  61, step     8000: bleu:   8.58, loss: 67023.5078, ppl:  14.0859, duration: 46.1847s\n",
            "2020-01-26 02:01:24,403 Epoch  61 Step:     8100 Batch Loss:     1.386473 Tokens per Sec:    12736, Lr: 0.000072\n",
            "2020-01-26 02:02:10,138 Example #0\n",
            "2020-01-26 02:02:10,139 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:02:10,139 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:02:10,139 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:02:10,139 Example #1\n",
            "2020-01-26 02:02:10,140 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:02:10,140 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:02:10,140 \tHypothesis: De emwi ọvbehe ne emwa ọvbehe miẹn ?\n",
            "2020-01-26 02:02:10,140 Example #2\n",
            "2020-01-26 02:02:10,141 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:02:10,141 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:02:10,141 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre vbe ẹgbẹe ọre ye rre uwu ẹgbẹe .\n",
            "2020-01-26 02:02:10,141 Example #3\n",
            "2020-01-26 02:02:10,141 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:02:10,141 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:02:10,142 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:02:10,142 Validation result (greedy) at epoch  61, step     8100: bleu:   8.95, loss: 67116.2812, ppl:  14.1376, duration: 45.7378s\n",
            "2020-01-26 02:02:14,565 Epoch  61: total training loss 200.74\n",
            "2020-01-26 02:02:14,566 EPOCH 62\n",
            "2020-01-26 02:02:25,282 Epoch  62 Step:     8200 Batch Loss:     1.637065 Tokens per Sec:    12539, Lr: 0.000072\n",
            "2020-01-26 02:03:11,229 Example #0\n",
            "2020-01-26 02:03:11,230 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:03:11,230 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:03:11,230 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:03:11,230 Example #1\n",
            "2020-01-26 02:03:11,231 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:03:11,231 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:03:11,231 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:03:11,231 Example #2\n",
            "2020-01-26 02:03:11,232 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:03:11,232 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:03:11,232 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre ye rre ọvbokhan rẹn .\n",
            "2020-01-26 02:03:11,232 Example #3\n",
            "2020-01-26 02:03:11,233 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:03:11,233 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:03:11,233 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ ne ima .\n",
            "2020-01-26 02:03:11,233 Validation result (greedy) at epoch  62, step     8200: bleu:   8.85, loss: 67135.5156, ppl:  14.1483, duration: 45.9509s\n",
            "2020-01-26 02:03:20,927 Epoch  62: total training loss 201.70\n",
            "2020-01-26 02:03:20,927 EPOCH 63\n",
            "2020-01-26 02:03:26,389 Epoch  63 Step:     8300 Batch Loss:     1.426634 Tokens per Sec:    12818, Lr: 0.000050\n",
            "2020-01-26 02:04:12,490 Example #0\n",
            "2020-01-26 02:04:12,491 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:04:12,491 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:04:12,491 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:04:12,491 Example #1\n",
            "2020-01-26 02:04:12,492 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:04:12,492 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:04:12,492 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 02:04:12,492 Example #2\n",
            "2020-01-26 02:04:12,493 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:04:12,493 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:04:12,493 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe , ẹgbẹe ọre kevbe ẹgbẹe ọre kevbe ẹgbẹe ọre na gha kpokpo egbe .\n",
            "2020-01-26 02:04:12,493 Example #3\n",
            "2020-01-26 02:04:12,494 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:04:12,494 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:04:12,494 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:04:12,494 Validation result (greedy) at epoch  63, step     8300: bleu:   8.85, loss: 67236.5703, ppl:  14.2049, duration: 46.1050s\n",
            "2020-01-26 02:04:27,353 Epoch  63: total training loss 200.75\n",
            "2020-01-26 02:04:27,353 EPOCH 64\n",
            "2020-01-26 02:04:27,828 Epoch  64 Step:     8400 Batch Loss:     1.691077 Tokens per Sec:    12362, Lr: 0.000050\n",
            "2020-01-26 02:05:13,817 Example #0\n",
            "2020-01-26 02:05:13,818 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:05:13,818 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:05:13,818 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:05:13,819 Example #1\n",
            "2020-01-26 02:05:13,819 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:05:13,819 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:05:13,819 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:05:13,820 Example #2\n",
            "2020-01-26 02:05:13,820 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:05:13,820 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:05:13,821 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:05:13,821 Example #3\n",
            "2020-01-26 02:05:13,821 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:05:13,821 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:05:13,822 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn emwi nọ gbae nọ ru ekpataki sẹ hẹ vbe ima ghaa sinmwi oseghe ne ọdakha ọghe Jehova .\n",
            "2020-01-26 02:05:13,822 Validation result (greedy) at epoch  64, step     8400: bleu:   9.00, loss: 67196.6562, ppl:  14.1825, duration: 45.9929s\n",
            "2020-01-26 02:05:28,900 Epoch  64 Step:     8500 Batch Loss:     1.517921 Tokens per Sec:    12778, Lr: 0.000050\n",
            "2020-01-26 02:06:14,913 Example #0\n",
            "2020-01-26 02:06:14,914 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:06:14,914 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:06:14,914 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:06:14,915 Example #1\n",
            "2020-01-26 02:06:14,915 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:06:14,915 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:06:14,915 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 02:06:14,916 Example #2\n",
            "2020-01-26 02:06:14,916 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:06:14,916 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:06:14,916 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ keghi kpa emwa odin rhunmwuda iran ma rẹn ọkpa vbe ẹgbẹe ọre kevbe ẹgbẹe ọre kevbe ẹgbẹe .\n",
            "2020-01-26 02:06:14,917 Example #3\n",
            "2020-01-26 02:06:14,917 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:06:14,917 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:06:14,917 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:06:14,918 Validation result (greedy) at epoch  64, step     8500: bleu:   8.71, loss: 67229.1172, ppl:  14.2007, duration: 46.0173s\n",
            "2020-01-26 02:06:19,661 Epoch  64: total training loss 201.87\n",
            "2020-01-26 02:06:19,661 EPOCH 65\n",
            "2020-01-26 02:06:29,913 Epoch  65 Step:     8600 Batch Loss:     1.577120 Tokens per Sec:    12718, Lr: 0.000050\n",
            "2020-01-26 02:07:15,915 Example #0\n",
            "2020-01-26 02:07:15,917 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:07:15,917 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:07:15,917 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:07:15,917 Example #1\n",
            "2020-01-26 02:07:15,918 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:07:15,918 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:07:15,918 \tHypothesis: De emwi ọvbehe ne a lae miẹn deghẹ a mu egbe rriotọ ?\n",
            "2020-01-26 02:07:15,918 Example #2\n",
            "2020-01-26 02:07:15,919 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:07:15,919 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:07:15,919 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:07:15,919 Example #3\n",
            "2020-01-26 02:07:15,920 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:07:15,920 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:07:15,920 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:07:15,921 Validation result (greedy) at epoch  65, step     8600: bleu:   8.92, loss: 67359.8906, ppl:  14.2742, duration: 46.0068s\n",
            "2020-01-26 02:07:25,867 Epoch  65: total training loss 199.68\n",
            "2020-01-26 02:07:25,868 EPOCH 66\n",
            "2020-01-26 02:07:31,022 Epoch  66 Step:     8700 Batch Loss:     1.441566 Tokens per Sec:    13005, Lr: 0.000050\n",
            "2020-01-26 02:08:16,917 Example #0\n",
            "2020-01-26 02:08:16,918 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:08:16,919 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:08:16,919 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:08:16,919 Example #1\n",
            "2020-01-26 02:08:16,920 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:08:16,920 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:08:16,920 \tHypothesis: De emwi nọ khẹke ne ọmwa nọ mu egbe rriotọ ?\n",
            "2020-01-26 02:08:16,920 Example #2\n",
            "2020-01-26 02:08:16,921 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:08:16,921 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:08:16,921 \tHypothesis: ( Mat 24 : 19 ) Ukpo eva ghi gberra nẹ , ọ ma he kpẹ vba , ẹgbẹe kevbe etiọnrẹn kevbe etiọnrẹn kevbe nọfua vbe ẹgbẹe .\n",
            "2020-01-26 02:08:16,921 Example #3\n",
            "2020-01-26 02:08:16,922 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:08:16,922 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:08:16,922 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:08:16,922 Validation result (greedy) at epoch  66, step     8700: bleu:   8.69, loss: 67260.8750, ppl:  14.2185, duration: 45.8999s\n",
            "2020-01-26 02:08:31,940 Epoch  66 Step:     8800 Batch Loss:     1.598998 Tokens per Sec:    12732, Lr: 0.000050\n",
            "2020-01-26 02:09:17,695 Example #0\n",
            "2020-01-26 02:09:17,697 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:09:17,697 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:09:17,697 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:09:17,697 Example #1\n",
            "2020-01-26 02:09:17,698 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:09:17,698 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:09:17,698 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 02:09:17,699 Example #2\n",
            "2020-01-26 02:09:17,699 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:09:17,700 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:09:17,700 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:09:17,700 Example #3\n",
            "2020-01-26 02:09:17,701 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:09:17,701 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:09:17,701 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn odẹ ne irẹn ya khaevbisẹ .\n",
            "2020-01-26 02:09:17,701 Validation result (greedy) at epoch  66, step     8800: bleu:   8.91, loss: 67368.3828, ppl:  14.2790, duration: 45.7605s\n",
            "2020-01-26 02:09:17,706 Epoch  66: total training loss 198.87\n",
            "2020-01-26 02:09:17,707 EPOCH 67\n",
            "2020-01-26 02:09:32,596 Epoch  67 Step:     8900 Batch Loss:     1.594130 Tokens per Sec:    12862, Lr: 0.000035\n",
            "2020-01-26 02:10:18,802 Example #0\n",
            "2020-01-26 02:10:18,803 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:10:18,803 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:10:18,803 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:10:18,803 Example #1\n",
            "2020-01-26 02:10:18,804 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:10:18,804 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:10:18,804 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:10:18,804 Example #2\n",
            "2020-01-26 02:10:18,804 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:10:18,805 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:10:18,805 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre .\n",
            "2020-01-26 02:10:18,805 Example #3\n",
            "2020-01-26 02:10:18,806 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:10:18,806 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:10:18,806 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne ọ ya khaevbisẹ .\n",
            "2020-01-26 02:10:18,806 Validation result (greedy) at epoch  67, step     8900: bleu:   8.84, loss: 67394.5547, ppl:  14.2937, duration: 46.2091s\n",
            "2020-01-26 02:10:23,991 Epoch  67: total training loss 197.45\n",
            "2020-01-26 02:10:23,992 EPOCH 68\n",
            "2020-01-26 02:10:34,017 Epoch  68 Step:     9000 Batch Loss:     1.293725 Tokens per Sec:    12571, Lr: 0.000035\n",
            "2020-01-26 02:11:19,918 Example #0\n",
            "2020-01-26 02:11:19,919 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:11:19,919 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:11:19,920 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:11:19,920 Example #1\n",
            "2020-01-26 02:11:19,920 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:11:19,920 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:11:19,921 \tHypothesis: De emwi nọ khẹke ne ima ru ne egbe ọmwa ?\n",
            "2020-01-26 02:11:19,921 Example #2\n",
            "2020-01-26 02:11:19,921 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:11:19,921 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:11:19,922 \tHypothesis: ( Mat 24 : 19 ) Ukpo eva ghi gberra nẹ , e Jesu na gha re ẹgbẹe ọkpa vbe ẹgbẹe .\n",
            "2020-01-26 02:11:19,922 Example #3\n",
            "2020-01-26 02:11:19,922 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:11:19,922 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:11:19,923 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:11:19,923 Validation result (greedy) at epoch  68, step     9000: bleu:   8.72, loss: 67415.0312, ppl:  14.3053, duration: 45.9049s\n",
            "2020-01-26 02:11:29,873 Epoch  68: total training loss 194.41\n",
            "2020-01-26 02:11:29,874 EPOCH 69\n",
            "2020-01-26 02:11:35,129 Epoch  69 Step:     9100 Batch Loss:     1.573143 Tokens per Sec:    12918, Lr: 0.000035\n",
            "2020-01-26 02:12:21,309 Example #0\n",
            "2020-01-26 02:12:21,310 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:12:21,310 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:12:21,310 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:12:21,311 Example #1\n",
            "2020-01-26 02:12:21,311 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:12:21,311 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:12:21,312 \tHypothesis: De emwi ọvbehe ne a lae miẹn ?\n",
            "2020-01-26 02:12:21,312 Example #2\n",
            "2020-01-26 02:12:21,312 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:12:21,312 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:12:21,313 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:12:21,313 Example #3\n",
            "2020-01-26 02:12:21,313 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:12:21,313 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:12:21,314 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:12:21,314 Validation result (greedy) at epoch  69, step     9100: bleu:   8.64, loss: 67438.4531, ppl:  14.3185, duration: 46.1841s\n",
            "2020-01-26 02:12:36,415 Epoch  69 Step:     9200 Batch Loss:     1.006205 Tokens per Sec:    12606, Lr: 0.000035\n",
            "2020-01-26 02:13:22,623 Example #0\n",
            "2020-01-26 02:13:22,623 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:13:22,623 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:13:22,623 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:13:22,623 Example #1\n",
            "2020-01-26 02:13:22,623 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:13:22,624 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:13:22,624 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:13:22,624 Example #2\n",
            "2020-01-26 02:13:22,624 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:13:22,624 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:13:22,624 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:13:22,624 Example #3\n",
            "2020-01-26 02:13:22,624 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:13:22,624 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:13:22,625 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa emwa ọvbehe .\n",
            "2020-01-26 02:13:22,625 Validation result (greedy) at epoch  69, step     9200: bleu:   8.85, loss: 67391.4375, ppl:  14.2920, duration: 46.2096s\n",
            "2020-01-26 02:13:22,629 Epoch  69: total training loss 197.04\n",
            "2020-01-26 02:13:22,629 EPOCH 70\n",
            "2020-01-26 02:13:37,928 Epoch  70 Step:     9300 Batch Loss:     1.494810 Tokens per Sec:    12603, Lr: 0.000035\n",
            "2020-01-26 02:14:24,200 Example #0\n",
            "2020-01-26 02:14:24,202 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:14:24,202 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:14:24,202 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:14:24,202 Example #1\n",
            "2020-01-26 02:14:24,203 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:14:24,203 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:14:24,204 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:14:24,204 Example #2\n",
            "2020-01-26 02:14:24,205 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:14:24,205 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:14:24,205 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:14:24,205 Example #3\n",
            "2020-01-26 02:14:24,206 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:14:24,206 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:14:24,206 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn emwi nọ gbae nọ ru ekpataki sẹ hẹ ne ima gha sinmwi oseghe ne ọdakha ọghe Jehova .\n",
            "2020-01-26 02:14:24,207 Validation result (greedy) at epoch  70, step     9300: bleu:   8.78, loss: 67535.1953, ppl:  14.3733, duration: 46.2775s\n",
            "2020-01-26 02:14:29,412 Epoch  70: total training loss 195.97\n",
            "2020-01-26 02:14:29,412 EPOCH 71\n",
            "2020-01-26 02:14:39,716 Epoch  71 Step:     9400 Batch Loss:     1.475954 Tokens per Sec:    12534, Lr: 0.000035\n",
            "2020-01-26 02:15:25,912 Example #0\n",
            "2020-01-26 02:15:25,913 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:15:25,913 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:15:25,913 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:15:25,913 Example #1\n",
            "2020-01-26 02:15:25,913 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:15:25,913 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:15:25,914 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:15:25,914 Example #2\n",
            "2020-01-26 02:15:25,914 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:15:25,914 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:15:25,914 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:15:25,915 Example #3\n",
            "2020-01-26 02:15:25,916 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:15:25,916 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:15:25,916 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:15:25,916 Validation result (greedy) at epoch  71, step     9400: bleu:   8.79, loss: 67509.2578, ppl:  14.3586, duration: 46.1997s\n",
            "2020-01-26 02:15:36,263 Epoch  71: total training loss 194.72\n",
            "2020-01-26 02:15:36,263 EPOCH 72\n",
            "2020-01-26 02:15:41,326 Epoch  72 Step:     9500 Batch Loss:     1.272474 Tokens per Sec:    12320, Lr: 0.000025\n",
            "2020-01-26 02:16:27,808 Example #0\n",
            "2020-01-26 02:16:27,809 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:16:27,809 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:16:27,809 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:16:27,810 Example #1\n",
            "2020-01-26 02:16:27,810 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:16:27,810 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:16:27,810 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:16:27,811 Example #2\n",
            "2020-01-26 02:16:27,811 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:16:27,811 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:16:27,811 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:16:27,811 Example #3\n",
            "2020-01-26 02:16:27,812 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:16:27,812 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:16:27,812 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ ne ima .\n",
            "2020-01-26 02:16:27,813 Validation result (greedy) at epoch  72, step     9500: bleu:   9.04, loss: 67497.9609, ppl:  14.3522, duration: 46.4856s\n",
            "2020-01-26 02:16:43,230 Epoch  72 Step:     9600 Batch Loss:     1.736515 Tokens per Sec:    12705, Lr: 0.000025\n",
            "2020-01-26 02:17:29,284 Example #0\n",
            "2020-01-26 02:17:29,285 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:17:29,285 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:17:29,285 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:17:29,286 Example #1\n",
            "2020-01-26 02:17:29,286 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:17:29,286 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:17:29,286 \tHypothesis: De emwi ọvbehe ne a lae miẹn deghẹ a mu egbe rriotọ ?\n",
            "2020-01-26 02:17:29,286 Example #2\n",
            "2020-01-26 02:17:29,287 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:17:29,287 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:17:29,287 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:17:29,287 Example #3\n",
            "2020-01-26 02:17:29,288 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:17:29,288 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:17:29,288 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:17:29,288 Validation result (greedy) at epoch  72, step     9600: bleu:   9.04, loss: 67494.0625, ppl:  14.3500, duration: 46.0578s\n",
            "2020-01-26 02:17:29,292 Epoch  72: total training loss 193.36\n",
            "2020-01-26 02:17:29,292 EPOCH 73\n",
            "2020-01-26 02:17:44,647 Epoch  73 Step:     9700 Batch Loss:     1.455012 Tokens per Sec:    12824, Lr: 0.000025\n",
            "2020-01-26 02:18:30,448 Example #0\n",
            "2020-01-26 02:18:30,449 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:18:30,449 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:18:30,450 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:18:30,450 Example #1\n",
            "2020-01-26 02:18:30,450 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:18:30,450 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:18:30,451 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:18:30,451 Example #2\n",
            "2020-01-26 02:18:30,451 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:18:30,451 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:18:30,452 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:18:30,452 Example #3\n",
            "2020-01-26 02:18:30,452 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:18:30,452 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:18:30,452 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ ne ima .\n",
            "2020-01-26 02:18:30,453 Validation result (greedy) at epoch  73, step     9700: bleu:   8.84, loss: 67517.0156, ppl:  14.3630, duration: 45.8050s\n",
            "2020-01-26 02:18:35,331 Epoch  73: total training loss 191.50\n",
            "2020-01-26 02:18:35,331 EPOCH 74\n",
            "2020-01-26 02:18:45,748 Epoch  74 Step:     9800 Batch Loss:     1.350528 Tokens per Sec:    12692, Lr: 0.000025\n",
            "2020-01-26 02:19:31,910 Example #0\n",
            "2020-01-26 02:19:31,911 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:19:31,911 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:19:31,911 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:19:31,912 Example #1\n",
            "2020-01-26 02:19:31,912 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:19:31,912 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:19:31,912 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:19:31,913 Example #2\n",
            "2020-01-26 02:19:31,913 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:19:31,913 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:19:31,913 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:19:31,914 Example #3\n",
            "2020-01-26 02:19:31,914 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:19:31,914 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:19:31,914 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:19:31,915 Validation result (greedy) at epoch  74, step     9800: bleu:   8.88, loss: 67603.0234, ppl:  14.4118, duration: 46.1666s\n",
            "2020-01-26 02:19:42,010 Epoch  74: total training loss 195.55\n",
            "2020-01-26 02:19:42,010 EPOCH 75\n",
            "2020-01-26 02:19:46,968 Epoch  75 Step:     9900 Batch Loss:     0.826328 Tokens per Sec:    12436, Lr: 0.000025\n",
            "2020-01-26 02:20:33,004 Example #0\n",
            "2020-01-26 02:20:33,005 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:20:33,005 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:20:33,005 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:20:33,005 Example #1\n",
            "2020-01-26 02:20:33,006 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:20:33,006 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:20:33,006 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:20:33,006 Example #2\n",
            "2020-01-26 02:20:33,006 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:20:33,007 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:20:33,007 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:20:33,007 Example #3\n",
            "2020-01-26 02:20:33,007 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:20:33,007 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:20:33,007 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa Osanobua .\n",
            "2020-01-26 02:20:33,008 Validation result (greedy) at epoch  75, step     9900: bleu:   9.20, loss: 67640.2656, ppl:  14.4330, duration: 46.0389s\n",
            "2020-01-26 02:20:48,319 Epoch  75 Step:    10000 Batch Loss:     1.786913 Tokens per Sec:    12673, Lr: 0.000025\n",
            "2020-01-26 02:21:34,337 Example #0\n",
            "2020-01-26 02:21:34,338 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:21:34,339 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:21:34,339 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:21:34,339 Example #1\n",
            "2020-01-26 02:21:34,339 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:21:34,339 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:21:34,340 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:21:34,340 Example #2\n",
            "2020-01-26 02:21:34,340 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:21:34,340 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:21:34,341 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:21:34,341 Example #3\n",
            "2020-01-26 02:21:34,341 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:21:34,341 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:21:34,341 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya rhie adia ne ima .\n",
            "2020-01-26 02:21:34,342 Validation result (greedy) at epoch  75, step    10000: bleu:   8.77, loss: 67654.8203, ppl:  14.4413, duration: 46.0222s\n",
            "2020-01-26 02:21:34,508 Epoch  75: total training loss 192.79\n",
            "2020-01-26 02:21:34,509 EPOCH 76\n",
            "2020-01-26 02:21:49,827 Epoch  76 Step:    10100 Batch Loss:     1.551966 Tokens per Sec:    12752, Lr: 0.000017\n",
            "2020-01-26 02:22:35,770 Example #0\n",
            "2020-01-26 02:22:35,771 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:22:35,772 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:22:35,772 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:22:35,772 Example #1\n",
            "2020-01-26 02:22:35,773 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:22:35,773 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:22:35,773 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:22:35,773 Example #2\n",
            "2020-01-26 02:22:35,774 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:22:35,774 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:22:35,774 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:22:35,774 Example #3\n",
            "2020-01-26 02:22:35,774 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:22:35,775 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:22:35,775 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:22:35,775 Validation result (greedy) at epoch  76, step    10100: bleu:   8.88, loss: 67677.2969, ppl:  14.4541, duration: 45.9474s\n",
            "2020-01-26 02:22:40,995 Epoch  76: total training loss 191.51\n",
            "2020-01-26 02:22:40,995 EPOCH 77\n",
            "2020-01-26 02:22:51,135 Epoch  77 Step:    10200 Batch Loss:     1.533165 Tokens per Sec:    12662, Lr: 0.000017\n",
            "2020-01-26 02:23:37,074 Example #0\n",
            "2020-01-26 02:23:37,075 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:23:37,075 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:23:37,075 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:23:37,075 Example #1\n",
            "2020-01-26 02:23:37,076 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:23:37,076 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:23:37,076 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:23:37,076 Example #2\n",
            "2020-01-26 02:23:37,077 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:23:37,077 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:23:37,077 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre kpa hin otu e Jehova rre .\n",
            "2020-01-26 02:23:37,077 Example #3\n",
            "2020-01-26 02:23:37,078 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:23:37,078 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:23:37,078 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 02:23:37,078 Validation result (greedy) at epoch  77, step    10200: bleu:   8.78, loss: 67688.9609, ppl:  14.4608, duration: 45.9422s\n",
            "2020-01-26 02:23:47,408 Epoch  77: total training loss 192.53\n",
            "2020-01-26 02:23:47,408 EPOCH 78\n",
            "2020-01-26 02:23:52,507 Epoch  78 Step:    10300 Batch Loss:     1.696002 Tokens per Sec:    12254, Lr: 0.000017\n",
            "2020-01-26 02:24:38,664 Example #0\n",
            "2020-01-26 02:24:38,665 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:24:38,665 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:24:38,665 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:24:38,666 Example #1\n",
            "2020-01-26 02:24:38,666 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:24:38,666 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:24:38,667 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:24:38,667 Example #2\n",
            "2020-01-26 02:24:38,667 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:24:38,667 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:24:38,668 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:24:38,668 Example #3\n",
            "2020-01-26 02:24:38,668 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:24:38,669 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:24:38,669 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:24:38,669 Validation result (greedy) at epoch  78, step    10300: bleu:   8.91, loss: 67644.1641, ppl:  14.4352, duration: 46.1615s\n",
            "2020-01-26 02:24:54,082 Epoch  78 Step:    10400 Batch Loss:     1.160017 Tokens per Sec:    12591, Lr: 0.000017\n",
            "2020-01-26 02:25:40,035 Example #0\n",
            "2020-01-26 02:25:40,036 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:25:40,036 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:25:40,037 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:25:40,037 Example #1\n",
            "2020-01-26 02:25:40,037 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:25:40,038 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:25:40,038 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:25:40,038 Example #2\n",
            "2020-01-26 02:25:40,039 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:25:40,039 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:25:40,039 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:25:40,039 Example #3\n",
            "2020-01-26 02:25:40,039 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:25:40,040 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:25:40,040 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:25:40,040 Validation result (greedy) at epoch  78, step    10400: bleu:   8.93, loss: 67676.1094, ppl:  14.4535, duration: 45.9575s\n",
            "2020-01-26 02:25:40,193 Epoch  78: total training loss 192.77\n",
            "2020-01-26 02:25:40,193 EPOCH 79\n",
            "2020-01-26 02:25:55,411 Epoch  79 Step:    10500 Batch Loss:     1.371390 Tokens per Sec:    12403, Lr: 0.000017\n",
            "2020-01-26 02:26:41,259 Example #0\n",
            "2020-01-26 02:26:41,260 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:26:41,260 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:26:41,260 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:26:41,260 Example #1\n",
            "2020-01-26 02:26:41,261 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:26:41,261 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:26:41,261 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:26:41,261 Example #2\n",
            "2020-01-26 02:26:41,262 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:26:41,262 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:26:41,262 \tHypothesis: Vbe ọ rre ukpo iyisẹn nibun vbe ẹgbẹe ọre kpa hin owa rre , ọ keghi kpa hin otu e Jehova rre .\n",
            "2020-01-26 02:26:41,263 Example #3\n",
            "2020-01-26 02:26:41,263 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:26:41,263 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:26:41,264 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya rhie adia ne ima .\n",
            "2020-01-26 02:26:41,264 Validation result (greedy) at epoch  79, step    10500: bleu:   8.97, loss: 67697.5938, ppl:  14.4657, duration: 45.8529s\n",
            "2020-01-26 02:26:46,625 Epoch  79: total training loss 192.26\n",
            "2020-01-26 02:26:46,626 EPOCH 80\n",
            "2020-01-26 02:26:56,752 Epoch  80 Step:    10600 Batch Loss:     1.353474 Tokens per Sec:    12922, Lr: 0.000017\n",
            "2020-01-26 02:27:42,793 Example #0\n",
            "2020-01-26 02:27:42,794 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:27:42,794 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:27:42,794 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:27:42,794 Example #1\n",
            "2020-01-26 02:27:42,795 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:27:42,795 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:27:42,795 \tHypothesis: De emwi ọvbehe ne a lae miẹn deghẹ a mu egbe rriotọ ?\n",
            "2020-01-26 02:27:42,795 Example #2\n",
            "2020-01-26 02:27:42,796 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:27:42,796 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:27:42,796 \tHypothesis: Vbe ọ rre ukpo iyisẹn nọ gberra , ọ ma he kpẹ gbe vbe ẹgbẹe ọre .\n",
            "2020-01-26 02:27:42,796 Example #3\n",
            "2020-01-26 02:27:42,797 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:27:42,797 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:27:42,797 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:27:42,797 Validation result (greedy) at epoch  80, step    10600: bleu:   8.88, loss: 67717.8125, ppl:  14.4773, duration: 46.0446s\n",
            "2020-01-26 02:27:53,288 Epoch  80: total training loss 190.99\n",
            "2020-01-26 02:27:53,289 EPOCH 81\n",
            "2020-01-26 02:27:58,148 Epoch  81 Step:    10700 Batch Loss:     1.314982 Tokens per Sec:    12646, Lr: 0.000012\n",
            "2020-01-26 02:28:44,288 Example #0\n",
            "2020-01-26 02:28:44,289 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:28:44,289 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:28:44,289 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:28:44,290 Example #1\n",
            "2020-01-26 02:28:44,290 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:28:44,290 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:28:44,290 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:28:44,291 Example #2\n",
            "2020-01-26 02:28:44,291 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:28:44,291 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:28:44,291 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:28:44,291 Example #3\n",
            "2020-01-26 02:28:44,292 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:28:44,292 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:28:44,292 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne ọ ya khaevbisẹ na rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:28:44,292 Validation result (greedy) at epoch  81, step    10700: bleu:   8.83, loss: 67700.4922, ppl:  14.4674, duration: 46.1436s\n",
            "2020-01-26 02:28:59,747 Epoch  81 Step:    10800 Batch Loss:     0.712849 Tokens per Sec:    12597, Lr: 0.000012\n",
            "2020-01-26 02:29:45,813 Example #0\n",
            "2020-01-26 02:29:45,814 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:29:45,814 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:29:45,814 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:29:45,814 Example #1\n",
            "2020-01-26 02:29:45,815 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:29:45,815 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:29:45,815 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:29:45,815 Example #2\n",
            "2020-01-26 02:29:45,816 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:29:45,816 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:29:45,816 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:29:45,816 Example #3\n",
            "2020-01-26 02:29:45,817 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:29:45,817 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:29:45,817 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya rhie adia ne ima .\n",
            "2020-01-26 02:29:45,817 Validation result (greedy) at epoch  81, step    10800: bleu:   9.05, loss: 67689.5312, ppl:  14.4611, duration: 46.0699s\n",
            "2020-01-26 02:29:45,970 Epoch  81: total training loss 191.04\n",
            "2020-01-26 02:29:45,970 EPOCH 82\n",
            "2020-01-26 02:30:01,430 Epoch  82 Step:    10900 Batch Loss:     1.245867 Tokens per Sec:    12259, Lr: 0.000012\n",
            "2020-01-26 02:30:47,904 Example #0\n",
            "2020-01-26 02:30:47,905 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:30:47,906 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:30:47,906 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:30:47,906 Example #1\n",
            "2020-01-26 02:30:47,907 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:30:47,907 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:30:47,907 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:30:47,907 Example #2\n",
            "2020-01-26 02:30:47,908 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:30:47,908 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:30:47,908 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:30:47,908 Example #3\n",
            "2020-01-26 02:30:47,909 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:30:47,909 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:30:47,909 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:30:47,909 Validation result (greedy) at epoch  82, step    10900: bleu:   8.90, loss: 67698.1719, ppl:  14.4660, duration: 46.4789s\n",
            "2020-01-26 02:30:53,334 Epoch  82: total training loss 191.19\n",
            "2020-01-26 02:30:53,334 EPOCH 83\n",
            "2020-01-26 02:31:03,696 Epoch  83 Step:    11000 Batch Loss:     1.668035 Tokens per Sec:    12250, Lr: 0.000012\n",
            "2020-01-26 02:31:49,939 Example #0\n",
            "2020-01-26 02:31:49,940 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:31:49,940 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:31:49,941 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:31:49,941 Example #1\n",
            "2020-01-26 02:31:49,941 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:31:49,942 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:31:49,942 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:31:49,942 Example #2\n",
            "2020-01-26 02:31:49,943 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:31:49,943 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:31:49,943 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:31:49,943 Example #3\n",
            "2020-01-26 02:31:49,944 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:31:49,944 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:31:49,944 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya rhie adia ne ima .\n",
            "2020-01-26 02:31:49,944 Validation result (greedy) at epoch  83, step    11000: bleu:   9.00, loss: 67727.9688, ppl:  14.4831, duration: 46.2482s\n",
            "2020-01-26 02:32:00,582 Epoch  83: total training loss 190.95\n",
            "2020-01-26 02:32:00,583 EPOCH 84\n",
            "2020-01-26 02:32:05,637 Epoch  84 Step:    11100 Batch Loss:     1.110473 Tokens per Sec:    12487, Lr: 0.000012\n",
            "2020-01-26 02:32:51,802 Example #0\n",
            "2020-01-26 02:32:51,803 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:32:51,803 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:32:51,803 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:32:51,803 Example #1\n",
            "2020-01-26 02:32:51,804 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:32:51,804 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:32:51,804 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:32:51,804 Example #2\n",
            "2020-01-26 02:32:51,805 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:32:51,805 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:32:51,805 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:32:51,805 Example #3\n",
            "2020-01-26 02:32:51,806 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:32:51,806 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:32:51,806 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 02:32:51,807 Validation result (greedy) at epoch  84, step    11100: bleu:   8.89, loss: 67785.2188, ppl:  14.5158, duration: 46.1688s\n",
            "2020-01-26 02:33:07,386 Epoch  84 Step:    11200 Batch Loss:     1.508843 Tokens per Sec:    12380, Lr: 0.000012\n",
            "2020-01-26 02:33:53,818 Example #0\n",
            "2020-01-26 02:33:53,819 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:33:53,819 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:33:53,819 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:33:53,820 Example #1\n",
            "2020-01-26 02:33:53,820 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:33:53,820 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:33:53,821 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:33:53,821 Example #2\n",
            "2020-01-26 02:33:53,821 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:33:53,821 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:33:53,822 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:33:53,822 Example #3\n",
            "2020-01-26 02:33:53,822 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:33:53,822 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:33:53,823 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 02:33:53,823 Validation result (greedy) at epoch  84, step    11200: bleu:   8.87, loss: 67723.7266, ppl:  14.4806, duration: 46.4362s\n",
            "2020-01-26 02:33:54,013 Epoch  84: total training loss 190.47\n",
            "2020-01-26 02:33:54,014 EPOCH 85\n",
            "2020-01-26 02:34:09,138 Epoch  85 Step:    11300 Batch Loss:     1.517386 Tokens per Sec:    12605, Lr: 0.000008\n",
            "2020-01-26 02:34:55,242 Example #0\n",
            "2020-01-26 02:34:55,243 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:34:55,243 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:34:55,244 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:34:55,244 Example #1\n",
            "2020-01-26 02:34:55,244 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:34:55,244 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:34:55,245 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:34:55,245 Example #2\n",
            "2020-01-26 02:34:55,245 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:34:55,245 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:34:55,245 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:34:55,246 Example #3\n",
            "2020-01-26 02:34:55,246 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:34:55,246 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:34:55,246 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rhiẹre ma wẹẹ irẹn hoẹmwẹ ima , ọ vbe rẹn emwi nọ ru ekpataki sẹ .\n",
            "2020-01-26 02:34:55,247 Validation result (greedy) at epoch  85, step    11300: bleu:   8.98, loss: 67743.7188, ppl:  14.4921, duration: 46.1078s\n",
            "2020-01-26 02:35:00,556 Epoch  85: total training loss 192.40\n",
            "2020-01-26 02:35:00,556 EPOCH 86\n",
            "2020-01-26 02:35:10,653 Epoch  86 Step:    11400 Batch Loss:     1.421414 Tokens per Sec:    12539, Lr: 0.000008\n",
            "2020-01-26 02:35:56,753 Example #0\n",
            "2020-01-26 02:35:56,754 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:35:56,754 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:35:56,754 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:35:56,754 Example #1\n",
            "2020-01-26 02:35:56,755 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:35:56,755 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:35:56,755 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:35:56,755 Example #2\n",
            "2020-01-26 02:35:56,756 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:35:56,756 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:35:56,756 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:35:56,757 Example #3\n",
            "2020-01-26 02:35:56,757 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:35:56,757 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:35:56,758 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa Osanobua .\n",
            "2020-01-26 02:35:56,758 Validation result (greedy) at epoch  86, step    11400: bleu:   9.00, loss: 67796.8047, ppl:  14.5225, duration: 46.1044s\n",
            "2020-01-26 02:36:07,331 Epoch  86: total training loss 191.46\n",
            "2020-01-26 02:36:07,332 EPOCH 87\n",
            "2020-01-26 02:36:12,109 Epoch  87 Step:    11500 Batch Loss:     1.586748 Tokens per Sec:    12895, Lr: 0.000008\n",
            "2020-01-26 02:36:58,076 Example #0\n",
            "2020-01-26 02:36:58,077 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:36:58,077 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:36:58,077 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:36:58,077 Example #1\n",
            "2020-01-26 02:36:58,078 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:36:58,078 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:36:58,078 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:36:58,078 Example #2\n",
            "2020-01-26 02:36:58,079 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:36:58,079 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:36:58,079 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:36:58,079 Example #3\n",
            "2020-01-26 02:36:58,079 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:36:58,080 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:36:58,080 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 02:36:58,080 Validation result (greedy) at epoch  87, step    11500: bleu:   8.81, loss: 67790.0078, ppl:  14.5186, duration: 45.9704s\n",
            "2020-01-26 02:37:13,429 Epoch  87 Step:    11600 Batch Loss:     1.720174 Tokens per Sec:    12483, Lr: 0.000008\n",
            "2020-01-26 02:37:59,218 Example #0\n",
            "2020-01-26 02:37:59,219 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:37:59,219 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:37:59,219 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:37:59,219 Example #1\n",
            "2020-01-26 02:37:59,219 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:37:59,219 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:37:59,219 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:37:59,219 Example #2\n",
            "2020-01-26 02:37:59,220 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:37:59,220 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:37:59,220 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:37:59,220 Example #3\n",
            "2020-01-26 02:37:59,220 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:37:59,220 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:37:59,220 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:37:59,220 Validation result (greedy) at epoch  87, step    11600: bleu:   8.91, loss: 67802.5156, ppl:  14.5257, duration: 45.7911s\n",
            "2020-01-26 02:37:59,678 Epoch  87: total training loss 190.44\n",
            "2020-01-26 02:37:59,679 EPOCH 88\n",
            "2020-01-26 02:38:14,142 Epoch  88 Step:    11700 Batch Loss:     0.992704 Tokens per Sec:    12896, Lr: 0.000008\n",
            "2020-01-26 02:39:00,294 Example #0\n",
            "2020-01-26 02:39:00,295 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:39:00,295 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:39:00,296 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:39:00,296 Example #1\n",
            "2020-01-26 02:39:00,296 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:39:00,296 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:39:00,297 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:39:00,297 Example #2\n",
            "2020-01-26 02:39:00,297 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:39:00,297 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:39:00,298 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:39:00,298 Example #3\n",
            "2020-01-26 02:39:00,298 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:39:00,298 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:39:00,299 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:39:00,299 Validation result (greedy) at epoch  88, step    11700: bleu:   9.03, loss: 67794.9062, ppl:  14.5214, duration: 46.1563s\n",
            "2020-01-26 02:39:06,117 Epoch  88: total training loss 192.18\n",
            "2020-01-26 02:39:06,118 EPOCH 89\n",
            "2020-01-26 02:39:15,615 Epoch  89 Step:    11800 Batch Loss:     1.080549 Tokens per Sec:    12779, Lr: 0.000008\n",
            "2020-01-26 02:40:01,581 Example #0\n",
            "2020-01-26 02:40:01,583 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:40:01,583 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:40:01,583 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:40:01,584 Example #1\n",
            "2020-01-26 02:40:01,584 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:40:01,584 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:40:01,585 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:40:01,585 Example #2\n",
            "2020-01-26 02:40:01,586 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:40:01,586 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:40:01,586 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:40:01,586 Example #3\n",
            "2020-01-26 02:40:01,587 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:40:01,587 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:40:01,587 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa Osanobua .\n",
            "2020-01-26 02:40:01,588 Validation result (greedy) at epoch  89, step    11800: bleu:   8.96, loss: 67818.7500, ppl:  14.5350, duration: 45.9719s\n",
            "2020-01-26 02:40:12,232 Epoch  89: total training loss 188.45\n",
            "2020-01-26 02:40:12,233 EPOCH 90\n",
            "2020-01-26 02:40:16,834 Epoch  90 Step:    11900 Batch Loss:     1.536855 Tokens per Sec:    12635, Lr: 0.000006\n",
            "2020-01-26 02:41:02,974 Example #0\n",
            "2020-01-26 02:41:02,975 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:41:02,975 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:41:02,976 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:41:02,976 Example #1\n",
            "2020-01-26 02:41:02,976 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:41:02,976 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:41:02,977 \tHypothesis: De emwi ọvbehe ne a lae miẹn deghẹ a mu egbe rriotọ ?\n",
            "2020-01-26 02:41:02,977 Example #2\n",
            "2020-01-26 02:41:02,977 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:41:02,977 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:41:02,977 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:41:02,977 Example #3\n",
            "2020-01-26 02:41:02,978 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:41:02,978 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:41:02,978 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:41:02,978 Validation result (greedy) at epoch  90, step    11900: bleu:   8.83, loss: 67795.4609, ppl:  14.5217, duration: 46.1444s\n",
            "2020-01-26 02:41:18,370 Epoch  90 Step:    12000 Batch Loss:     1.560818 Tokens per Sec:    12469, Lr: 0.000006\n",
            "2020-01-26 02:42:04,096 Example #0\n",
            "2020-01-26 02:42:04,098 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:42:04,098 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:42:04,098 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:42:04,098 Example #1\n",
            "2020-01-26 02:42:04,099 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:42:04,099 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:42:04,100 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:42:04,100 Example #2\n",
            "2020-01-26 02:42:04,101 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:42:04,101 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:42:04,101 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:42:04,102 Example #3\n",
            "2020-01-26 02:42:04,102 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:42:04,103 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:42:04,103 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:42:04,103 Validation result (greedy) at epoch  90, step    12000: bleu:   8.91, loss: 67799.2812, ppl:  14.5239, duration: 45.7325s\n",
            "2020-01-26 02:42:04,703 Epoch  90: total training loss 190.91\n",
            "2020-01-26 02:42:04,703 EPOCH 91\n",
            "2020-01-26 02:42:19,388 Epoch  91 Step:    12100 Batch Loss:     1.654498 Tokens per Sec:    12552, Lr: 0.000006\n",
            "2020-01-26 02:43:05,544 Example #0\n",
            "2020-01-26 02:43:05,545 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:43:05,545 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:43:05,546 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:43:05,546 Example #1\n",
            "2020-01-26 02:43:05,547 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:43:05,547 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:43:05,547 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:43:05,547 Example #2\n",
            "2020-01-26 02:43:05,548 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:43:05,548 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:43:05,548 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:43:05,548 Example #3\n",
            "2020-01-26 02:43:05,549 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:43:05,549 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:43:05,549 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:43:05,549 Validation result (greedy) at epoch  91, step    12100: bleu:   8.98, loss: 67809.5000, ppl:  14.5297, duration: 46.1608s\n",
            "2020-01-26 02:43:11,326 Epoch  91: total training loss 190.48\n",
            "2020-01-26 02:43:11,326 EPOCH 92\n",
            "2020-01-26 02:43:20,866 Epoch  92 Step:    12200 Batch Loss:     1.671062 Tokens per Sec:    12591, Lr: 0.000006\n",
            "2020-01-26 02:44:06,981 Example #0\n",
            "2020-01-26 02:44:06,982 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:44:06,982 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:44:06,983 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:44:06,983 Example #1\n",
            "2020-01-26 02:44:06,983 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:44:06,984 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:44:06,984 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:44:06,984 Example #2\n",
            "2020-01-26 02:44:06,985 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:44:06,985 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:44:06,985 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:44:06,985 Example #3\n",
            "2020-01-26 02:44:06,986 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:44:06,986 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:44:06,987 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 02:44:06,987 Validation result (greedy) at epoch  92, step    12200: bleu:   9.01, loss: 67820.1953, ppl:  14.5359, duration: 46.1178s\n",
            "2020-01-26 02:44:17,859 Epoch  92: total training loss 189.47\n",
            "2020-01-26 02:44:17,860 EPOCH 93\n",
            "2020-01-26 02:44:22,366 Epoch  93 Step:    12300 Batch Loss:     1.546275 Tokens per Sec:    12600, Lr: 0.000006\n",
            "2020-01-26 02:45:08,286 Example #0\n",
            "2020-01-26 02:45:08,287 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:45:08,287 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:45:08,287 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:45:08,287 Example #1\n",
            "2020-01-26 02:45:08,288 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:45:08,288 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:45:08,288 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:45:08,288 Example #2\n",
            "2020-01-26 02:45:08,289 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:45:08,289 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:45:08,289 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:45:08,289 Example #3\n",
            "2020-01-26 02:45:08,290 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:45:08,290 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:45:08,290 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn evbọzẹe ne ọ na khẹke ne ima gha mwẹ ekhọe agiẹngiẹn daa Osanobua .\n",
            "2020-01-26 02:45:08,290 Validation result (greedy) at epoch  93, step    12300: bleu:   9.03, loss: 67823.7344, ppl:  14.5379, duration: 45.9232s\n",
            "2020-01-26 02:45:23,795 Epoch  93 Step:    12400 Batch Loss:     1.566066 Tokens per Sec:    12567, Lr: 0.000006\n",
            "2020-01-26 02:46:09,857 Example #0\n",
            "2020-01-26 02:46:09,858 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:46:09,858 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:46:09,858 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:46:09,859 Example #1\n",
            "2020-01-26 02:46:09,859 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:46:09,859 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:46:09,860 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:46:09,860 Example #2\n",
            "2020-01-26 02:46:09,860 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:46:09,861 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:46:09,861 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:46:09,861 Example #3\n",
            "2020-01-26 02:46:09,861 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:46:09,862 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:46:09,862 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:46:09,862 Validation result (greedy) at epoch  93, step    12400: bleu:   8.98, loss: 67829.2344, ppl:  14.5411, duration: 46.0661s\n",
            "2020-01-26 02:46:10,429 Epoch  93: total training loss 189.53\n",
            "2020-01-26 02:46:10,429 EPOCH 94\n",
            "2020-01-26 02:46:25,024 Epoch  94 Step:    12500 Batch Loss:     1.580140 Tokens per Sec:    12509, Lr: 0.000004\n",
            "2020-01-26 02:47:11,077 Example #0\n",
            "2020-01-26 02:47:11,078 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:47:11,078 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:47:11,078 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:47:11,078 Example #1\n",
            "2020-01-26 02:47:11,079 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:47:11,079 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:47:11,079 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:47:11,079 Example #2\n",
            "2020-01-26 02:47:11,080 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:47:11,080 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:47:11,080 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:47:11,080 Example #3\n",
            "2020-01-26 02:47:11,081 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:47:11,081 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:47:11,081 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:47:11,081 Validation result (greedy) at epoch  94, step    12500: bleu:   8.95, loss: 67826.4375, ppl:  14.5395, duration: 46.0561s\n",
            "2020-01-26 02:47:17,105 Epoch  94: total training loss 191.58\n",
            "2020-01-26 02:47:17,105 EPOCH 95\n",
            "2020-01-26 02:47:26,473 Epoch  95 Step:    12600 Batch Loss:     1.457369 Tokens per Sec:    12501, Lr: 0.000004\n",
            "2020-01-26 02:48:12,581 Example #0\n",
            "2020-01-26 02:48:12,582 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:48:12,582 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:48:12,582 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:48:12,583 Example #1\n",
            "2020-01-26 02:48:12,583 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:48:12,583 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:48:12,584 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:48:12,584 Example #2\n",
            "2020-01-26 02:48:12,584 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:48:12,585 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:48:12,585 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:48:12,585 Example #3\n",
            "2020-01-26 02:48:12,586 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:48:12,586 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:48:12,586 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:48:12,586 Validation result (greedy) at epoch  95, step    12600: bleu:   8.94, loss: 67836.6875, ppl:  14.5453, duration: 46.1127s\n",
            "2020-01-26 02:48:23,699 Epoch  95: total training loss 188.99\n",
            "2020-01-26 02:48:23,699 EPOCH 96\n",
            "2020-01-26 02:48:27,910 Epoch  96 Step:    12700 Batch Loss:     0.713840 Tokens per Sec:    12733, Lr: 0.000004\n",
            "2020-01-26 02:49:14,136 Example #0\n",
            "2020-01-26 02:49:14,137 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:49:14,137 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:49:14,137 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:49:14,137 Example #1\n",
            "2020-01-26 02:49:14,138 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:49:14,138 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:49:14,138 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:49:14,138 Example #2\n",
            "2020-01-26 02:49:14,139 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:49:14,139 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:49:14,139 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:49:14,139 Example #3\n",
            "2020-01-26 02:49:14,140 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:49:14,140 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:49:14,140 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:49:14,140 Validation result (greedy) at epoch  96, step    12700: bleu:   8.95, loss: 67840.2422, ppl:  14.5474, duration: 46.2302s\n",
            "2020-01-26 02:49:29,514 Epoch  96 Step:    12800 Batch Loss:     1.772653 Tokens per Sec:    12773, Lr: 0.000004\n",
            "2020-01-26 02:50:15,821 Example #0\n",
            "2020-01-26 02:50:15,822 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:50:15,822 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:50:15,822 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:50:15,822 Example #1\n",
            "2020-01-26 02:50:15,823 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:50:15,823 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:50:15,823 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:50:15,823 Example #2\n",
            "2020-01-26 02:50:15,824 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:50:15,824 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:50:15,824 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:50:15,824 Example #3\n",
            "2020-01-26 02:50:15,825 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:50:15,825 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:50:15,825 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:50:15,826 Validation result (greedy) at epoch  96, step    12800: bleu:   8.89, loss: 67824.2344, ppl:  14.5382, duration: 46.3108s\n",
            "2020-01-26 02:50:16,603 Epoch  96: total training loss 189.22\n",
            "2020-01-26 02:50:16,603 EPOCH 97\n",
            "2020-01-26 02:50:31,396 Epoch  97 Step:    12900 Batch Loss:     1.604238 Tokens per Sec:    12644, Lr: 0.000004\n",
            "2020-01-26 02:51:17,837 Example #0\n",
            "2020-01-26 02:51:17,838 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:51:17,838 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:51:17,839 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:51:17,839 Example #1\n",
            "2020-01-26 02:51:17,839 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:51:17,840 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:51:17,840 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:51:17,840 Example #2\n",
            "2020-01-26 02:51:17,840 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:51:17,841 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:51:17,841 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:51:17,841 Example #3\n",
            "2020-01-26 02:51:17,841 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:51:17,842 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:51:17,842 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:51:17,842 Validation result (greedy) at epoch  97, step    12900: bleu:   8.89, loss: 67827.5781, ppl:  14.5401, duration: 46.4456s\n",
            "2020-01-26 02:51:23,848 Epoch  97: total training loss 190.57\n",
            "2020-01-26 02:51:23,848 EPOCH 98\n",
            "2020-01-26 02:51:33,386 Epoch  98 Step:    13000 Batch Loss:     1.339684 Tokens per Sec:    12219, Lr: 0.000004\n",
            "2020-01-26 02:52:19,801 Example #0\n",
            "2020-01-26 02:52:19,802 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:52:19,803 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:52:19,803 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:52:19,803 Example #1\n",
            "2020-01-26 02:52:19,804 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:52:19,804 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:52:19,805 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:52:19,805 Example #2\n",
            "2020-01-26 02:52:19,806 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:52:19,806 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:52:19,806 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:52:19,806 Example #3\n",
            "2020-01-26 02:52:19,807 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:52:19,807 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:52:19,807 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:52:19,807 Validation result (greedy) at epoch  98, step    13000: bleu:   8.86, loss: 67821.2422, ppl:  14.5365, duration: 46.4206s\n",
            "2020-01-26 02:52:31,325 Epoch  98: total training loss 190.83\n",
            "2020-01-26 02:52:31,326 EPOCH 99\n",
            "2020-01-26 02:52:35,387 Epoch  99 Step:    13100 Batch Loss:     1.731981 Tokens per Sec:    11871, Lr: 0.000003\n",
            "2020-01-26 02:53:21,707 Example #0\n",
            "2020-01-26 02:53:21,708 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:53:21,708 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:53:21,708 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:53:21,709 Example #1\n",
            "2020-01-26 02:53:21,709 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:53:21,709 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:53:21,709 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:53:21,710 Example #2\n",
            "2020-01-26 02:53:21,710 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:53:21,710 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:53:21,710 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:53:21,710 Example #3\n",
            "2020-01-26 02:53:21,711 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:53:21,711 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:53:21,711 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:53:21,711 Validation result (greedy) at epoch  99, step    13100: bleu:   8.90, loss: 67852.6328, ppl:  14.5545, duration: 46.3238s\n",
            "2020-01-26 02:53:37,124 Epoch  99 Step:    13200 Batch Loss:     1.544361 Tokens per Sec:    12616, Lr: 0.000003\n",
            "2020-01-26 02:54:23,278 Example #0\n",
            "2020-01-26 02:54:23,279 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:54:23,279 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:54:23,279 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:54:23,279 Example #1\n",
            "2020-01-26 02:54:23,280 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:54:23,280 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:54:23,280 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:54:23,280 Example #2\n",
            "2020-01-26 02:54:23,281 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:54:23,281 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:54:23,281 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:54:23,281 Example #3\n",
            "2020-01-26 02:54:23,282 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:54:23,282 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:54:23,282 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:54:23,282 Validation result (greedy) at epoch  99, step    13200: bleu:   8.89, loss: 67843.1875, ppl:  14.5491, duration: 46.1579s\n",
            "2020-01-26 02:54:24,551 Epoch  99: total training loss 189.01\n",
            "2020-01-26 02:54:24,551 EPOCH 100\n",
            "2020-01-26 02:54:39,038 Epoch 100 Step:    13300 Batch Loss:     1.300402 Tokens per Sec:    12543, Lr: 0.000003\n",
            "2020-01-26 02:55:25,193 Example #0\n",
            "2020-01-26 02:55:25,194 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:55:25,194 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:55:25,194 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:55:25,194 Example #1\n",
            "2020-01-26 02:55:25,195 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:55:25,195 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:55:25,195 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:55:25,195 Example #2\n",
            "2020-01-26 02:55:25,196 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:55:25,196 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:55:25,196 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:55:25,196 Example #3\n",
            "2020-01-26 02:55:25,197 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:55:25,197 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:55:25,197 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:55:25,197 Validation result (greedy) at epoch 100, step    13300: bleu:   8.94, loss: 67852.8906, ppl:  14.5546, duration: 46.1586s\n",
            "2020-01-26 02:55:31,117 Epoch 100: total training loss 186.01\n",
            "2020-01-26 02:55:31,117 EPOCH 101\n",
            "2020-01-26 02:55:40,506 Epoch 101 Step:    13400 Batch Loss:     1.272540 Tokens per Sec:    12419, Lr: 0.000003\n",
            "2020-01-26 02:56:26,633 Example #0\n",
            "2020-01-26 02:56:26,634 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:56:26,634 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:56:26,634 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:56:26,635 Example #1\n",
            "2020-01-26 02:56:26,635 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:56:26,635 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:56:26,635 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:56:26,635 Example #2\n",
            "2020-01-26 02:56:26,636 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:56:26,636 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:56:26,636 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:56:26,636 Example #3\n",
            "2020-01-26 02:56:26,637 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:56:26,637 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:56:26,637 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:56:26,637 Validation result (greedy) at epoch 101, step    13400: bleu:   8.94, loss: 67857.5938, ppl:  14.5573, duration: 46.1306s\n",
            "2020-01-26 02:56:37,820 Epoch 101: total training loss 190.40\n",
            "2020-01-26 02:56:37,821 EPOCH 102\n",
            "2020-01-26 02:56:42,171 Epoch 102 Step:    13500 Batch Loss:     1.700558 Tokens per Sec:    12743, Lr: 0.000003\n",
            "2020-01-26 02:57:28,023 Example #0\n",
            "2020-01-26 02:57:28,024 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:57:28,025 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:57:28,025 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:57:28,025 Example #1\n",
            "2020-01-26 02:57:28,026 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:57:28,026 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:57:28,026 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:57:28,027 Example #2\n",
            "2020-01-26 02:57:28,027 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:57:28,027 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:57:28,028 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:57:28,028 Example #3\n",
            "2020-01-26 02:57:28,028 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:57:28,028 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:57:28,029 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:57:28,029 Validation result (greedy) at epoch 102, step    13500: bleu:   8.96, loss: 67852.0859, ppl:  14.5542, duration: 45.8573s\n",
            "2020-01-26 02:57:43,314 Epoch 102 Step:    13600 Batch Loss:     1.671705 Tokens per Sec:    12579, Lr: 0.000003\n",
            "2020-01-26 02:58:29,529 Example #0\n",
            "2020-01-26 02:58:29,530 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:58:29,530 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:58:29,530 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:58:29,531 Example #1\n",
            "2020-01-26 02:58:29,531 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:58:29,531 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:58:29,532 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:58:29,532 Example #2\n",
            "2020-01-26 02:58:29,532 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:58:29,532 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:58:29,532 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:58:29,533 Example #3\n",
            "2020-01-26 02:58:29,533 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:58:29,533 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:58:29,533 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:58:29,533 Validation result (greedy) at epoch 102, step    13600: bleu:   8.97, loss: 67856.6797, ppl:  14.5568, duration: 46.2188s\n",
            "2020-01-26 02:58:30,417 Epoch 102: total training loss 188.78\n",
            "2020-01-26 02:58:30,418 EPOCH 103\n",
            "2020-01-26 02:58:45,003 Epoch 103 Step:    13700 Batch Loss:     1.416684 Tokens per Sec:    12612, Lr: 0.000002\n",
            "2020-01-26 02:59:31,057 Example #0\n",
            "2020-01-26 02:59:31,059 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 02:59:31,066 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 02:59:31,066 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 02:59:31,066 Example #1\n",
            "2020-01-26 02:59:31,067 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 02:59:31,067 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 02:59:31,067 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 02:59:31,068 Example #2\n",
            "2020-01-26 02:59:31,068 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 02:59:31,068 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 02:59:31,068 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 02:59:31,068 Example #3\n",
            "2020-01-26 02:59:31,069 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 02:59:31,069 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 02:59:31,069 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 02:59:31,069 Validation result (greedy) at epoch 103, step    13700: bleu:   8.96, loss: 67857.4062, ppl:  14.5572, duration: 46.0656s\n",
            "2020-01-26 02:59:37,027 Epoch 103: total training loss 188.33\n",
            "2020-01-26 02:59:37,027 EPOCH 104\n",
            "2020-01-26 02:59:46,552 Epoch 104 Step:    13800 Batch Loss:     1.346457 Tokens per Sec:    12534, Lr: 0.000002\n",
            "2020-01-26 03:00:32,460 Example #0\n",
            "2020-01-26 03:00:32,461 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:00:32,461 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:00:32,461 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:00:32,461 Example #1\n",
            "2020-01-26 03:00:32,462 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:00:32,462 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:00:32,462 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:00:32,462 Example #2\n",
            "2020-01-26 03:00:32,463 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:00:32,463 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:00:32,463 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:00:32,463 Example #3\n",
            "2020-01-26 03:00:32,464 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:00:32,464 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:00:32,464 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 03:00:32,464 Validation result (greedy) at epoch 104, step    13800: bleu:   9.03, loss: 67857.5078, ppl:  14.5573, duration: 45.9116s\n",
            "2020-01-26 03:00:43,679 Epoch 104: total training loss 190.23\n",
            "2020-01-26 03:00:43,679 EPOCH 105\n",
            "2020-01-26 03:00:47,825 Epoch 105 Step:    13900 Batch Loss:     1.721960 Tokens per Sec:    13549, Lr: 0.000002\n",
            "2020-01-26 03:01:34,011 Example #0\n",
            "2020-01-26 03:01:34,012 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:01:34,012 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:01:34,012 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:01:34,012 Example #1\n",
            "2020-01-26 03:01:34,013 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:01:34,013 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:01:34,013 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:01:34,013 Example #2\n",
            "2020-01-26 03:01:34,014 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:01:34,014 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:01:34,014 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:01:34,014 Example #3\n",
            "2020-01-26 03:01:34,015 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:01:34,015 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:01:34,015 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:01:34,015 Validation result (greedy) at epoch 105, step    13900: bleu:   8.98, loss: 67860.9219, ppl:  14.5593, duration: 46.1902s\n",
            "2020-01-26 03:01:49,357 Epoch 105 Step:    14000 Batch Loss:     1.700053 Tokens per Sec:    12500, Lr: 0.000002\n",
            "2020-01-26 03:02:35,262 Example #0\n",
            "2020-01-26 03:02:35,263 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:02:35,263 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:02:35,263 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:02:35,263 Example #1\n",
            "2020-01-26 03:02:35,264 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:02:35,264 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:02:35,264 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:02:35,264 Example #2\n",
            "2020-01-26 03:02:35,265 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:02:35,265 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:02:35,265 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:02:35,266 Example #3\n",
            "2020-01-26 03:02:35,266 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:02:35,266 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:02:35,266 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 03:02:35,267 Validation result (greedy) at epoch 105, step    14000: bleu:   8.97, loss: 67856.6250, ppl:  14.5568, duration: 45.9093s\n",
            "2020-01-26 03:02:36,022 Epoch 105: total training loss 187.23\n",
            "2020-01-26 03:02:36,023 EPOCH 106\n",
            "2020-01-26 03:02:50,825 Epoch 106 Step:    14100 Batch Loss:     1.571217 Tokens per Sec:    12372, Lr: 0.000002\n",
            "2020-01-26 03:03:37,031 Example #0\n",
            "2020-01-26 03:03:37,031 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:03:37,032 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:03:37,032 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:03:37,032 Example #1\n",
            "2020-01-26 03:03:37,033 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:03:37,033 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:03:37,033 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:03:37,033 Example #2\n",
            "2020-01-26 03:03:37,034 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:03:37,034 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:03:37,034 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:03:37,034 Example #3\n",
            "2020-01-26 03:03:37,035 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:03:37,035 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:03:37,035 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 03:03:37,036 Validation result (greedy) at epoch 106, step    14100: bleu:   8.91, loss: 67863.0938, ppl:  14.5605, duration: 46.2097s\n",
            "2020-01-26 03:03:43,044 Epoch 106: total training loss 190.13\n",
            "2020-01-26 03:03:43,044 EPOCH 107\n",
            "2020-01-26 03:03:52,357 Epoch 107 Step:    14200 Batch Loss:     0.799073 Tokens per Sec:    12744, Lr: 0.000002\n",
            "2020-01-26 03:04:38,348 Example #0\n",
            "2020-01-26 03:04:38,349 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:04:38,349 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:04:38,350 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:04:38,350 Example #1\n",
            "2020-01-26 03:04:38,351 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:04:38,351 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:04:38,351 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:04:38,351 Example #2\n",
            "2020-01-26 03:04:38,352 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:04:38,352 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:04:38,352 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:04:38,352 Example #3\n",
            "2020-01-26 03:04:38,353 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:04:38,353 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:04:38,353 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi re odẹ nọ ya khaevbisẹ .\n",
            "2020-01-26 03:04:38,353 Validation result (greedy) at epoch 107, step    14200: bleu:   8.97, loss: 67850.3047, ppl:  14.5532, duration: 45.9945s\n",
            "2020-01-26 03:04:49,680 Epoch 107: total training loss 189.50\n",
            "2020-01-26 03:04:49,680 EPOCH 108\n",
            "2020-01-26 03:04:53,900 Epoch 108 Step:    14300 Batch Loss:     1.528365 Tokens per Sec:    12610, Lr: 0.000001\n",
            "2020-01-26 03:05:40,030 Example #0\n",
            "2020-01-26 03:05:40,031 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:05:40,031 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:05:40,032 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:05:40,032 Example #1\n",
            "2020-01-26 03:05:40,032 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:05:40,033 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:05:40,033 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:05:40,033 Example #2\n",
            "2020-01-26 03:05:40,034 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:05:40,034 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:05:40,034 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:05:40,034 Example #3\n",
            "2020-01-26 03:05:40,035 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:05:40,035 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:05:40,035 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:05:40,035 Validation result (greedy) at epoch 108, step    14300: bleu:   8.97, loss: 67855.5859, ppl:  14.5562, duration: 46.1341s\n",
            "2020-01-26 03:05:55,438 Epoch 108 Step:    14400 Batch Loss:     1.553447 Tokens per Sec:    12766, Lr: 0.000001\n",
            "2020-01-26 03:06:41,413 Example #0\n",
            "2020-01-26 03:06:41,414 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:06:41,415 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:06:41,415 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:06:41,415 Example #1\n",
            "2020-01-26 03:06:41,415 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:06:41,416 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:06:41,416 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:06:41,416 Example #2\n",
            "2020-01-26 03:06:41,417 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:06:41,417 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:06:41,417 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:06:41,417 Example #3\n",
            "2020-01-26 03:06:41,418 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:06:41,418 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:06:41,418 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:06:41,419 Validation result (greedy) at epoch 108, step    14400: bleu:   8.99, loss: 67851.4844, ppl:  14.5538, duration: 45.9804s\n",
            "2020-01-26 03:06:42,181 Epoch 108: total training loss 187.91\n",
            "2020-01-26 03:06:42,181 EPOCH 109\n",
            "2020-01-26 03:06:56,928 Epoch 109 Step:    14500 Batch Loss:     1.536593 Tokens per Sec:    12754, Lr: 0.000001\n",
            "2020-01-26 03:07:43,024 Example #0\n",
            "2020-01-26 03:07:43,025 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:07:43,025 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:07:43,025 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:07:43,025 Example #1\n",
            "2020-01-26 03:07:43,026 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:07:43,026 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:07:43,026 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:07:43,026 Example #2\n",
            "2020-01-26 03:07:43,026 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:07:43,027 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:07:43,027 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:07:43,027 Example #3\n",
            "2020-01-26 03:07:43,027 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:07:43,027 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:07:43,027 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:07:43,028 Validation result (greedy) at epoch 109, step    14500: bleu:   8.96, loss: 67862.1641, ppl:  14.5600, duration: 46.0992s\n",
            "2020-01-26 03:07:48,691 Epoch 109: total training loss 186.90\n",
            "2020-01-26 03:07:48,692 EPOCH 110\n",
            "2020-01-26 03:07:58,268 Epoch 110 Step:    14600 Batch Loss:     1.570027 Tokens per Sec:    12533, Lr: 0.000001\n",
            "2020-01-26 03:08:44,331 Example #0\n",
            "2020-01-26 03:08:44,332 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:08:44,332 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:08:44,332 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:08:44,333 Example #1\n",
            "2020-01-26 03:08:44,333 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:08:44,333 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:08:44,333 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:08:44,333 Example #2\n",
            "2020-01-26 03:08:44,334 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:08:44,334 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:08:44,334 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:08:44,334 Example #3\n",
            "2020-01-26 03:08:44,335 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:08:44,335 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:08:44,335 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:08:44,335 Validation result (greedy) at epoch 110, step    14600: bleu:   8.92, loss: 67869.9688, ppl:  14.5645, duration: 46.0670s\n",
            "2020-01-26 03:08:55,222 Epoch 110: total training loss 186.78\n",
            "2020-01-26 03:08:55,222 EPOCH 111\n",
            "2020-01-26 03:08:59,845 Epoch 111 Step:    14700 Batch Loss:     1.082400 Tokens per Sec:    12746, Lr: 0.000001\n",
            "2020-01-26 03:09:45,859 Example #0\n",
            "2020-01-26 03:09:45,860 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:09:45,860 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:09:45,860 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:09:45,861 Example #1\n",
            "2020-01-26 03:09:45,861 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:09:45,861 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:09:45,861 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:09:45,862 Example #2\n",
            "2020-01-26 03:09:45,862 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:09:45,862 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:09:45,862 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:09:45,863 Example #3\n",
            "2020-01-26 03:09:45,863 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:09:45,863 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:09:45,863 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:09:45,863 Validation result (greedy) at epoch 111, step    14700: bleu:   8.94, loss: 67875.7266, ppl:  14.5678, duration: 46.0177s\n",
            "2020-01-26 03:10:01,472 Epoch 111 Step:    14800 Batch Loss:     1.157158 Tokens per Sec:    12435, Lr: 0.000001\n",
            "2020-01-26 03:10:47,708 Example #0\n",
            "2020-01-26 03:10:47,710 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:10:47,710 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:10:47,710 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:10:47,710 Example #1\n",
            "2020-01-26 03:10:47,711 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:10:47,711 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:10:47,711 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:10:47,711 Example #2\n",
            "2020-01-26 03:10:47,712 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:10:47,712 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:10:47,712 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:10:47,713 Example #3\n",
            "2020-01-26 03:10:47,713 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:10:47,713 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:10:47,713 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:10:47,714 Validation result (greedy) at epoch 111, step    14800: bleu:   8.94, loss: 67876.4141, ppl:  14.5682, duration: 46.2409s\n",
            "2020-01-26 03:10:48,201 Epoch 111: total training loss 190.65\n",
            "2020-01-26 03:10:48,202 EPOCH 112\n",
            "2020-01-26 03:11:03,037 Epoch 112 Step:    14900 Batch Loss:     0.959999 Tokens per Sec:    12606, Lr: 0.000001\n",
            "2020-01-26 03:11:48,882 Example #0\n",
            "2020-01-26 03:11:48,883 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:11:48,883 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:11:48,883 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:11:48,884 Example #1\n",
            "2020-01-26 03:11:48,884 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:11:48,884 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:11:48,884 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:11:48,884 Example #2\n",
            "2020-01-26 03:11:48,885 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:11:48,885 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:11:48,885 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:11:48,885 Example #3\n",
            "2020-01-26 03:11:48,886 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:11:48,886 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:11:48,886 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:11:48,886 Validation result (greedy) at epoch 112, step    14900: bleu:   9.00, loss: 67873.5312, ppl:  14.5665, duration: 45.8484s\n",
            "2020-01-26 03:11:54,588 Epoch 112: total training loss 190.38\n",
            "2020-01-26 03:11:54,588 EPOCH 113\n",
            "2020-01-26 03:12:04,452 Epoch 113 Step:    15000 Batch Loss:     1.733730 Tokens per Sec:    12671, Lr: 0.000001\n",
            "2020-01-26 03:12:50,382 Example #0\n",
            "2020-01-26 03:12:50,383 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:12:50,383 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:12:50,383 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:12:50,383 Example #1\n",
            "2020-01-26 03:12:50,384 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:12:50,384 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:12:50,384 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:12:50,384 Example #2\n",
            "2020-01-26 03:12:50,385 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:12:50,385 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:12:50,385 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:12:50,385 Example #3\n",
            "2020-01-26 03:12:50,386 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:12:50,386 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:12:50,386 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:12:50,386 Validation result (greedy) at epoch 113, step    15000: bleu:   8.96, loss: 67872.6016, ppl:  14.5660, duration: 45.9343s\n",
            "2020-01-26 03:13:01,361 Epoch 113: total training loss 190.45\n",
            "2020-01-26 03:13:01,361 EPOCH 114\n",
            "2020-01-26 03:13:05,787 Epoch 114 Step:    15100 Batch Loss:     1.301667 Tokens per Sec:    13072, Lr: 0.000001\n",
            "2020-01-26 03:13:51,853 Example #0\n",
            "2020-01-26 03:13:51,854 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:13:51,854 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:13:51,855 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:13:51,855 Example #1\n",
            "2020-01-26 03:13:51,855 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:13:51,855 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:13:51,856 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:13:51,856 Example #2\n",
            "2020-01-26 03:13:51,856 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:13:51,856 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:13:51,856 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:13:51,856 Example #3\n",
            "2020-01-26 03:13:51,857 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:13:51,857 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:13:51,857 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:13:51,857 Validation result (greedy) at epoch 114, step    15100: bleu:   9.00, loss: 67873.6172, ppl:  14.5666, duration: 46.0702s\n",
            "2020-01-26 03:14:07,285 Epoch 114 Step:    15200 Batch Loss:     1.171884 Tokens per Sec:    12434, Lr: 0.000001\n",
            "2020-01-26 03:14:53,438 Example #0\n",
            "2020-01-26 03:14:53,440 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:14:53,440 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:14:53,440 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:14:53,440 Example #1\n",
            "2020-01-26 03:14:53,441 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:14:53,441 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:14:53,441 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:14:53,441 Example #2\n",
            "2020-01-26 03:14:53,442 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:14:53,442 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:14:53,442 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:14:53,442 Example #3\n",
            "2020-01-26 03:14:53,442 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:14:53,443 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:14:53,443 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:14:53,443 Validation result (greedy) at epoch 114, step    15200: bleu:   8.98, loss: 67869.6250, ppl:  14.5643, duration: 46.1571s\n",
            "2020-01-26 03:14:54,058 Epoch 114: total training loss 188.03\n",
            "2020-01-26 03:14:54,058 EPOCH 115\n",
            "2020-01-26 03:15:08,914 Epoch 115 Step:    15300 Batch Loss:     1.644526 Tokens per Sec:    12718, Lr: 0.000001\n",
            "2020-01-26 03:15:55,029 Example #0\n",
            "2020-01-26 03:15:55,030 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:15:55,030 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:15:55,031 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:15:55,031 Example #1\n",
            "2020-01-26 03:15:55,031 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:15:55,031 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:15:55,031 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:15:55,032 Example #2\n",
            "2020-01-26 03:15:55,032 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:15:55,032 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:15:55,032 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:15:55,032 Example #3\n",
            "2020-01-26 03:15:55,033 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:15:55,033 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:15:55,033 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:15:55,034 Validation result (greedy) at epoch 115, step    15300: bleu:   8.92, loss: 67872.2734, ppl:  14.5658, duration: 46.1188s\n",
            "2020-01-26 03:16:00,716 Epoch 115: total training loss 188.88\n",
            "2020-01-26 03:16:00,717 EPOCH 116\n",
            "2020-01-26 03:16:10,367 Epoch 116 Step:    15400 Batch Loss:     1.487834 Tokens per Sec:    12594, Lr: 0.000001\n",
            "2020-01-26 03:16:56,361 Example #0\n",
            "2020-01-26 03:16:56,362 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:16:56,362 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:16:56,363 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:16:56,363 Example #1\n",
            "2020-01-26 03:16:56,363 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:16:56,364 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:16:56,364 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:16:56,364 Example #2\n",
            "2020-01-26 03:16:56,364 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:16:56,364 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:16:56,365 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:16:56,365 Example #3\n",
            "2020-01-26 03:16:56,365 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:16:56,365 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:16:56,366 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:16:56,366 Validation result (greedy) at epoch 116, step    15400: bleu:   8.96, loss: 67871.4297, ppl:  14.5653, duration: 45.9987s\n",
            "2020-01-26 03:17:07,417 Epoch 116: total training loss 191.59\n",
            "2020-01-26 03:17:07,418 EPOCH 117\n",
            "2020-01-26 03:17:11,809 Epoch 117 Step:    15500 Batch Loss:     0.709412 Tokens per Sec:    12491, Lr: 0.000001\n",
            "2020-01-26 03:17:58,039 Example #0\n",
            "2020-01-26 03:17:58,041 \tSource:     Reading and meditating on real - life Bible accounts can help us to do what ?\n",
            "2020-01-26 03:17:58,041 \tReference:  De vbene okha ni rre Baibol ya ru iyobọ ne ima hẹ ?\n",
            "2020-01-26 03:17:58,041 \tHypothesis: De emwi nọ gha ru iyobọ ne ima ya gha ru erria yan emwi ne ima tie vbe Baibol ?\n",
            "2020-01-26 03:17:58,041 Example #1\n",
            "2020-01-26 03:17:58,042 \tSource:     What are the rewards for being humble ?\n",
            "2020-01-26 03:17:58,042 \tReference:  Ma ghaa mu egbe rriotọ , de afiangbe na lae miẹn ?\n",
            "2020-01-26 03:17:58,042 \tHypothesis: De emwi ọvbehe ne emwa ni mu egbe rriotọ ?\n",
            "2020-01-26 03:17:58,042 Example #2\n",
            "2020-01-26 03:17:58,043 \tSource:     In his younger years , Jesus no doubt had to deal with the deaths of family members and acquaintances .\n",
            "2020-01-26 03:17:58,043 \tReference:  Vbe Jesu rre ọvbokhan , ughaghe emwa eso ne ọ hoẹmwẹ ọnrẹn wulo .\n",
            "2020-01-26 03:17:58,043 \tHypothesis: Vbe ọ rre ukpo iyisẹn ne Jesu gbe vbe agbọn , ọ ma zẹdẹ gha khuẹrhẹ ne ẹgbẹe ọre kevbe ẹgbẹe ọre .\n",
            "2020-01-26 03:17:58,043 Example #3\n",
            "2020-01-26 03:17:58,044 \tSource:     One reason is that he rules with love . Indeed , our hearts are touched by how he chooses to exercise his sovereignty .\n",
            "2020-01-26 03:17:58,044 \tReference:  Ọna ẹre ọ si ẹre ne Arriọba ọghẹe na maan sẹ ọghe emwa ọvbehe , ere ọ vbe si ẹre ne emwa ne irẹn kha yan na mwẹ agbẹkunsotọ .\n",
            "2020-01-26 03:17:58,044 \tHypothesis: Ọ keghi re odẹ ọkpa ne ọ ya rẹn odẹ ne irẹn ya khaevbisẹ , ọ keghi ru iyobọ ne ima ya rẹn odẹ ne irẹn ya sinmwi oseghe ne ọdakha ọghẹe .\n",
            "2020-01-26 03:17:58,044 Validation result (greedy) at epoch 117, step    15500: bleu:   8.97, loss: 67873.5156, ppl:  14.5665, duration: 46.2353s\n",
            "2020-01-26 03:18:13,239 Epoch 117 Step:    15600 Batch Loss:     0.666670 Tokens per Sec:    12533, Lr: 0.000001\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/lib/python3.6/runpy.py\", line 193, in _run_module_as_main\n",
            "    \"__main__\", mod_spec)\n",
            "  File \"/usr/lib/python3.6/runpy.py\", line 85, in _run_code\n",
            "    exec(code, run_globals)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/__main__.py\", line 41, in <module>\n",
            "    main()\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/__main__.py\", line 29, in main\n",
            "    train(cfg_file=args.config_path)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/training.py\", line 596, in train\n",
            "    trainer.train_and_validate(train_data=train_data, valid_data=dev_data)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/training.py\", line 338, in train_and_validate\n",
            "    batch_type=self.eval_batch_type\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/prediction.py\", line 106, in validate_on_data\n",
            "    max_output_length=max_output_length)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/model.py\", line 170, in run_batch\n",
            "    max_output_length=max_output_length)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/search.py\", line 43, in greedy\n",
            "    decoder, encoder_output, encoder_hidden)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/search.py\", line 139, in transformer_greedy\n",
            "    trg_mask=trg_mask\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\", line 541, in __call__\n",
            "    result = self.forward(*input, **kwargs)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/joeynmt/decoders.py\", line 514, in forward\n",
            "    trg_embed.size(1)).type_as(trg_mask)\n",
            "KeyboardInterrupt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "n94wlrCjVc17",
        "outputId": "a76c4794-7ece-4ddb-a6a2-c6ee750f5175",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "# Output our validation accuracy epoch 1-30\n",
        "! cat \"$gdrive_path/models/${src}${tgt}_transformer/validations.txt\""
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Steps: 100\tLoss: 125476.36719\tPPL: 141.47205\tbleu: 0.00000\tLR: 0.00030000\t*\n",
            "Steps: 200\tLoss: 118648.96875\tPPL: 108.05595\tbleu: 0.00000\tLR: 0.00030000\t*\n",
            "Steps: 300\tLoss: 109354.53906\tPPL: 74.87583\tbleu: 0.00000\tLR: 0.00030000\t*\n",
            "Steps: 400\tLoss: 101641.44531\tPPL: 55.22541\tbleu: 0.19764\tLR: 0.00030000\t*\n",
            "Steps: 500\tLoss: 95679.29688\tPPL: 43.64631\tbleu: 0.78639\tLR: 0.00030000\t*\n",
            "Steps: 600\tLoss: 91839.13281\tPPL: 37.50826\tbleu: 2.80675\tLR: 0.00030000\t*\n",
            "Steps: 700\tLoss: 89375.11719\tPPL: 34.03248\tbleu: 2.75541\tLR: 0.00030000\t*\n",
            "Steps: 800\tLoss: 86692.92188\tPPL: 30.61404\tbleu: 2.64025\tLR: 0.00030000\t*\n",
            "Steps: 900\tLoss: 84652.37500\tPPL: 28.24526\tbleu: 2.71768\tLR: 0.00030000\t*\n",
            "Steps: 1000\tLoss: 83337.43750\tPPL: 26.81684\tbleu: 2.70694\tLR: 0.00030000\t*\n",
            "Steps: 1100\tLoss: 82218.52344\tPPL: 25.65838\tbleu: 2.64497\tLR: 0.00030000\t*\n",
            "Steps: 1200\tLoss: 80442.40625\tPPL: 23.92139\tbleu: 3.31051\tLR: 0.00030000\t*\n",
            "Steps: 1300\tLoss: 79197.82031\tPPL: 22.77478\tbleu: 3.33019\tLR: 0.00030000\t*\n",
            "Steps: 1400\tLoss: 78018.17188\tPPL: 21.73877\tbleu: 3.11730\tLR: 0.00030000\t*\n",
            "Steps: 1500\tLoss: 76878.25000\tPPL: 20.78245\tbleu: 3.73138\tLR: 0.00030000\t*\n",
            "Steps: 1600\tLoss: 76290.89844\tPPL: 20.30624\tbleu: 3.68220\tLR: 0.00030000\t*\n",
            "Steps: 1700\tLoss: 75560.96875\tPPL: 19.72961\tbleu: 3.62098\tLR: 0.00030000\t*\n",
            "Steps: 1800\tLoss: 74449.51562\tPPL: 18.88287\tbleu: 4.01199\tLR: 0.00030000\t*\n",
            "Steps: 1900\tLoss: 73871.99219\tPPL: 18.45735\tbleu: 4.46457\tLR: 0.00030000\t*\n",
            "Steps: 2000\tLoss: 73189.66406\tPPL: 17.96694\tbleu: 4.05053\tLR: 0.00030000\t*\n",
            "Steps: 2100\tLoss: 72345.94531\tPPL: 17.37852\tbleu: 4.21949\tLR: 0.00030000\t*\n",
            "Steps: 2200\tLoss: 72096.81250\tPPL: 17.20848\tbleu: 4.73951\tLR: 0.00030000\t*\n",
            "Steps: 2300\tLoss: 71423.82031\tPPL: 16.75743\tbleu: 4.52387\tLR: 0.00030000\t*\n",
            "Steps: 2400\tLoss: 71028.25000\tPPL: 16.49785\tbleu: 4.73609\tLR: 0.00030000\t*\n",
            "Steps: 2500\tLoss: 70475.75781\tPPL: 16.14202\tbleu: 4.46355\tLR: 0.00030000\t*\n",
            "Steps: 2600\tLoss: 70019.71094\tPPL: 15.85408\tbleu: 4.99101\tLR: 0.00030000\t*\n",
            "Steps: 2700\tLoss: 69720.86719\tPPL: 15.66819\tbleu: 5.01876\tLR: 0.00030000\t*\n",
            "Steps: 2800\tLoss: 69225.85156\tPPL: 15.36506\tbleu: 5.04857\tLR: 0.00030000\t*\n",
            "Steps: 2900\tLoss: 68906.17969\tPPL: 15.17243\tbleu: 5.00235\tLR: 0.00030000\t*\n",
            "Steps: 3000\tLoss: 68791.19531\tPPL: 15.10373\tbleu: 5.31593\tLR: 0.00030000\t*\n",
            "Steps: 3100\tLoss: 69157.22656\tPPL: 15.32350\tbleu: 4.99978\tLR: 0.00030000\t\n",
            "Steps: 3200\tLoss: 68055.75781\tPPL: 14.67164\tbleu: 5.67794\tLR: 0.00030000\t*\n",
            "Steps: 3300\tLoss: 68056.85938\tPPL: 14.67228\tbleu: 5.66833\tLR: 0.00030000\t\n",
            "Steps: 3400\tLoss: 68085.71875\tPPL: 14.68900\tbleu: 5.16024\tLR: 0.00030000\t\n",
            "Steps: 3500\tLoss: 67995.09375\tPPL: 14.63656\tbleu: 5.70646\tLR: 0.00030000\t*\n",
            "Steps: 3600\tLoss: 67231.42188\tPPL: 14.20200\tbleu: 5.60858\tLR: 0.00030000\t*\n",
            "Steps: 3700\tLoss: 67388.32812\tPPL: 14.29023\tbleu: 5.60374\tLR: 0.00030000\t\n",
            "Steps: 3800\tLoss: 67441.69531\tPPL: 14.32035\tbleu: 6.31954\tLR: 0.00030000\t\n",
            "Steps: 3900\tLoss: 67131.84375\tPPL: 14.14630\tbleu: 5.78611\tLR: 0.00030000\t*\n",
            "Steps: 4000\tLoss: 66808.63281\tPPL: 13.96700\tbleu: 6.28374\tLR: 0.00030000\t*\n",
            "Steps: 4100\tLoss: 66764.74219\tPPL: 13.94282\tbleu: 6.07741\tLR: 0.00030000\t*\n",
            "Steps: 4200\tLoss: 66829.39062\tPPL: 13.97844\tbleu: 6.32046\tLR: 0.00030000\t\n",
            "Steps: 4300\tLoss: 66457.75781\tPPL: 13.77492\tbleu: 6.37299\tLR: 0.00030000\t*\n",
            "Steps: 4400\tLoss: 66373.86719\tPPL: 13.72939\tbleu: 6.72327\tLR: 0.00030000\t*\n",
            "Steps: 4500\tLoss: 66614.52344\tPPL: 13.86040\tbleu: 6.94250\tLR: 0.00030000\t\n",
            "Steps: 4600\tLoss: 66329.39062\tPPL: 13.70531\tbleu: 7.05851\tLR: 0.00030000\t*\n",
            "Steps: 4700\tLoss: 66241.23438\tPPL: 13.65771\tbleu: 6.92800\tLR: 0.00030000\t*\n",
            "Steps: 4800\tLoss: 66121.75781\tPPL: 13.59346\tbleu: 7.04838\tLR: 0.00030000\t*\n",
            "Steps: 4900\tLoss: 66205.37500\tPPL: 13.63839\tbleu: 7.30122\tLR: 0.00030000\t\n",
            "Steps: 5000\tLoss: 66186.03906\tPPL: 13.62799\tbleu: 7.54020\tLR: 0.00030000\t\n",
            "Steps: 5100\tLoss: 66085.09375\tPPL: 13.57380\tbleu: 7.57227\tLR: 0.00030000\t*\n",
            "Steps: 5200\tLoss: 65724.94531\tPPL: 13.38223\tbleu: 7.67888\tLR: 0.00030000\t*\n",
            "Steps: 5300\tLoss: 66018.11719\tPPL: 13.53797\tbleu: 7.24353\tLR: 0.00030000\t\n",
            "Steps: 5400\tLoss: 66345.33594\tPPL: 13.71394\tbleu: 7.81080\tLR: 0.00030000\t\n",
            "Steps: 5500\tLoss: 66119.89844\tPPL: 13.59246\tbleu: 7.77553\tLR: 0.00030000\t\n",
            "Steps: 5600\tLoss: 66039.36719\tPPL: 13.54933\tbleu: 7.77225\tLR: 0.00030000\t\n",
            "Steps: 5700\tLoss: 65805.81250\tPPL: 13.42501\tbleu: 7.76459\tLR: 0.00030000\t\n",
            "Steps: 5800\tLoss: 66231.10156\tPPL: 13.65225\tbleu: 8.19801\tLR: 0.00021000\t\n",
            "Steps: 5900\tLoss: 65995.32031\tPPL: 13.52579\tbleu: 8.61758\tLR: 0.00021000\t\n",
            "Steps: 6000\tLoss: 66038.66406\tPPL: 13.54895\tbleu: 8.39876\tLR: 0.00021000\t\n",
            "Steps: 6100\tLoss: 66013.30469\tPPL: 13.53540\tbleu: 8.24765\tLR: 0.00021000\t\n",
            "Steps: 6200\tLoss: 66171.49219\tPPL: 13.62017\tbleu: 8.23300\tLR: 0.00021000\t\n",
            "Steps: 6300\tLoss: 66477.57812\tPPL: 13.78570\tbleu: 8.27286\tLR: 0.00021000\t\n",
            "Steps: 6400\tLoss: 66052.28125\tPPL: 13.55623\tbleu: 8.16124\tLR: 0.00014700\t\n",
            "Steps: 6500\tLoss: 66314.07812\tPPL: 13.69703\tbleu: 8.64006\tLR: 0.00014700\t\n",
            "Steps: 6600\tLoss: 66391.75000\tPPL: 13.73908\tbleu: 8.60272\tLR: 0.00014700\t\n",
            "Steps: 6700\tLoss: 66471.28125\tPPL: 13.78227\tbleu: 8.52694\tLR: 0.00014700\t\n",
            "Steps: 6800\tLoss: 66387.07812\tPPL: 13.73654\tbleu: 8.53749\tLR: 0.00014700\t\n",
            "Steps: 6900\tLoss: 66540.19531\tPPL: 13.81981\tbleu: 8.64762\tLR: 0.00014700\t\n",
            "Steps: 7000\tLoss: 66639.53906\tPPL: 13.87410\tbleu: 8.79795\tLR: 0.00010290\t\n",
            "Steps: 7100\tLoss: 66615.69531\tPPL: 13.86105\tbleu: 8.47606\tLR: 0.00010290\t\n",
            "Steps: 7200\tLoss: 66593.60938\tPPL: 13.84897\tbleu: 8.66779\tLR: 0.00010290\t\n",
            "Steps: 7300\tLoss: 66560.39062\tPPL: 13.83083\tbleu: 8.49449\tLR: 0.00010290\t\n",
            "Steps: 7400\tLoss: 66691.01562\tPPL: 13.90231\tbleu: 8.89157\tLR: 0.00010290\t\n",
            "Steps: 7500\tLoss: 66849.81250\tPPL: 13.98971\tbleu: 8.73623\tLR: 0.00010290\t\n",
            "Steps: 7600\tLoss: 66799.52344\tPPL: 13.96198\tbleu: 8.73055\tLR: 0.00007203\t\n",
            "Steps: 7700\tLoss: 66923.10938\tPPL: 14.03024\tbleu: 8.51567\tLR: 0.00007203\t\n",
            "Steps: 7800\tLoss: 66943.30469\tPPL: 14.04143\tbleu: 8.47803\tLR: 0.00007203\t\n",
            "Steps: 7900\tLoss: 66957.14062\tPPL: 14.04910\tbleu: 9.12226\tLR: 0.00007203\t\n",
            "Steps: 8000\tLoss: 67023.50781\tPPL: 14.08594\tbleu: 8.57921\tLR: 0.00007203\t\n",
            "Steps: 8100\tLoss: 67116.28125\tPPL: 14.13761\tbleu: 8.94585\tLR: 0.00007203\t\n",
            "Steps: 8200\tLoss: 67135.51562\tPPL: 14.14835\tbleu: 8.85090\tLR: 0.00005042\t\n",
            "Steps: 8300\tLoss: 67236.57031\tPPL: 14.20489\tbleu: 8.85404\tLR: 0.00005042\t\n",
            "Steps: 8400\tLoss: 67196.65625\tPPL: 14.18253\tbleu: 9.00315\tLR: 0.00005042\t\n",
            "Steps: 8500\tLoss: 67229.11719\tPPL: 14.20071\tbleu: 8.71434\tLR: 0.00005042\t\n",
            "Steps: 8600\tLoss: 67359.89062\tPPL: 14.27419\tbleu: 8.92282\tLR: 0.00005042\t\n",
            "Steps: 8700\tLoss: 67260.87500\tPPL: 14.21852\tbleu: 8.69390\tLR: 0.00005042\t\n",
            "Steps: 8800\tLoss: 67368.38281\tPPL: 14.27898\tbleu: 8.90604\tLR: 0.00003529\t\n",
            "Steps: 8900\tLoss: 67394.55469\tPPL: 14.29373\tbleu: 8.84061\tLR: 0.00003529\t\n",
            "Steps: 9000\tLoss: 67415.03125\tPPL: 14.30529\tbleu: 8.71909\tLR: 0.00003529\t\n",
            "Steps: 9100\tLoss: 67438.45312\tPPL: 14.31852\tbleu: 8.63892\tLR: 0.00003529\t\n",
            "Steps: 9200\tLoss: 67391.43750\tPPL: 14.29198\tbleu: 8.84657\tLR: 0.00003529\t\n",
            "Steps: 9300\tLoss: 67535.19531\tPPL: 14.37329\tbleu: 8.77897\tLR: 0.00003529\t\n",
            "Steps: 9400\tLoss: 67509.25781\tPPL: 14.35859\tbleu: 8.78664\tLR: 0.00002471\t\n",
            "Steps: 9500\tLoss: 67497.96094\tPPL: 14.35219\tbleu: 9.03824\tLR: 0.00002471\t\n",
            "Steps: 9600\tLoss: 67494.06250\tPPL: 14.34998\tbleu: 9.03799\tLR: 0.00002471\t\n",
            "Steps: 9700\tLoss: 67517.01562\tPPL: 14.36299\tbleu: 8.83709\tLR: 0.00002471\t\n",
            "Steps: 9800\tLoss: 67603.02344\tPPL: 14.41182\tbleu: 8.87709\tLR: 0.00002471\t\n",
            "Steps: 9900\tLoss: 67640.26562\tPPL: 14.43302\tbleu: 9.19885\tLR: 0.00002471\t\n",
            "Steps: 10000\tLoss: 67654.82031\tPPL: 14.44131\tbleu: 8.76662\tLR: 0.00001729\t\n",
            "Steps: 10100\tLoss: 67677.29688\tPPL: 14.45413\tbleu: 8.88496\tLR: 0.00001729\t\n",
            "Steps: 10200\tLoss: 67688.96094\tPPL: 14.46079\tbleu: 8.77931\tLR: 0.00001729\t\n",
            "Steps: 10300\tLoss: 67644.16406\tPPL: 14.43524\tbleu: 8.90649\tLR: 0.00001729\t\n",
            "Steps: 10400\tLoss: 67676.10938\tPPL: 14.45345\tbleu: 8.93158\tLR: 0.00001729\t\n",
            "Steps: 10500\tLoss: 67697.59375\tPPL: 14.46571\tbleu: 8.96955\tLR: 0.00001729\t\n",
            "Steps: 10600\tLoss: 67717.81250\tPPL: 14.47726\tbleu: 8.87830\tLR: 0.00001211\t\n",
            "Steps: 10700\tLoss: 67700.49219\tPPL: 14.46737\tbleu: 8.82774\tLR: 0.00001211\t\n",
            "Steps: 10800\tLoss: 67689.53125\tPPL: 14.46111\tbleu: 9.04687\tLR: 0.00001211\t\n",
            "Steps: 10900\tLoss: 67698.17188\tPPL: 14.46604\tbleu: 8.89640\tLR: 0.00001211\t\n",
            "Steps: 11000\tLoss: 67727.96875\tPPL: 14.48306\tbleu: 8.99521\tLR: 0.00001211\t\n",
            "Steps: 11100\tLoss: 67785.21875\tPPL: 14.51582\tbleu: 8.89295\tLR: 0.00001211\t\n",
            "Steps: 11200\tLoss: 67723.72656\tPPL: 14.48064\tbleu: 8.87474\tLR: 0.00000847\t\n",
            "Steps: 11300\tLoss: 67743.71875\tPPL: 14.49207\tbleu: 8.97991\tLR: 0.00000847\t\n",
            "Steps: 11400\tLoss: 67796.80469\tPPL: 14.52246\tbleu: 8.99560\tLR: 0.00000847\t\n",
            "Steps: 11500\tLoss: 67790.00781\tPPL: 14.51857\tbleu: 8.80995\tLR: 0.00000847\t\n",
            "Steps: 11600\tLoss: 67802.51562\tPPL: 14.52574\tbleu: 8.91001\tLR: 0.00000847\t\n",
            "Steps: 11700\tLoss: 67794.90625\tPPL: 14.52138\tbleu: 9.03064\tLR: 0.00000847\t\n",
            "Steps: 11800\tLoss: 67818.75000\tPPL: 14.53505\tbleu: 8.96232\tLR: 0.00000593\t\n",
            "Steps: 11900\tLoss: 67795.46094\tPPL: 14.52169\tbleu: 8.82956\tLR: 0.00000593\t\n",
            "Steps: 12000\tLoss: 67799.28125\tPPL: 14.52388\tbleu: 8.90506\tLR: 0.00000593\t\n",
            "Steps: 12100\tLoss: 67809.50000\tPPL: 14.52974\tbleu: 8.97888\tLR: 0.00000593\t\n",
            "Steps: 12200\tLoss: 67820.19531\tPPL: 14.53588\tbleu: 9.00638\tLR: 0.00000593\t\n",
            "Steps: 12300\tLoss: 67823.73438\tPPL: 14.53791\tbleu: 9.02727\tLR: 0.00000593\t\n",
            "Steps: 12400\tLoss: 67829.23438\tPPL: 14.54106\tbleu: 8.97700\tLR: 0.00000415\t\n",
            "Steps: 12500\tLoss: 67826.43750\tPPL: 14.53946\tbleu: 8.94803\tLR: 0.00000415\t\n",
            "Steps: 12600\tLoss: 67836.68750\tPPL: 14.54534\tbleu: 8.93616\tLR: 0.00000415\t\n",
            "Steps: 12700\tLoss: 67840.24219\tPPL: 14.54738\tbleu: 8.95289\tLR: 0.00000415\t\n",
            "Steps: 12800\tLoss: 67824.23438\tPPL: 14.53819\tbleu: 8.88773\tLR: 0.00000415\t\n",
            "Steps: 12900\tLoss: 67827.57812\tPPL: 14.54011\tbleu: 8.88984\tLR: 0.00000415\t\n",
            "Steps: 13000\tLoss: 67821.24219\tPPL: 14.53648\tbleu: 8.85695\tLR: 0.00000291\t\n",
            "Steps: 13100\tLoss: 67852.63281\tPPL: 14.55450\tbleu: 8.89511\tLR: 0.00000291\t\n",
            "Steps: 13200\tLoss: 67843.18750\tPPL: 14.54907\tbleu: 8.89024\tLR: 0.00000291\t\n",
            "Steps: 13300\tLoss: 67852.89062\tPPL: 14.55464\tbleu: 8.94270\tLR: 0.00000291\t\n",
            "Steps: 13400\tLoss: 67857.59375\tPPL: 14.55735\tbleu: 8.93678\tLR: 0.00000291\t\n",
            "Steps: 13500\tLoss: 67852.08594\tPPL: 14.55418\tbleu: 8.96115\tLR: 0.00000291\t\n",
            "Steps: 13600\tLoss: 67856.67969\tPPL: 14.55682\tbleu: 8.96623\tLR: 0.00000203\t\n",
            "Steps: 13700\tLoss: 67857.40625\tPPL: 14.55724\tbleu: 8.96343\tLR: 0.00000203\t\n",
            "Steps: 13800\tLoss: 67857.50781\tPPL: 14.55730\tbleu: 9.02738\tLR: 0.00000203\t\n",
            "Steps: 13900\tLoss: 67860.92188\tPPL: 14.55926\tbleu: 8.97965\tLR: 0.00000203\t\n",
            "Steps: 14000\tLoss: 67856.62500\tPPL: 14.55679\tbleu: 8.97418\tLR: 0.00000203\t\n",
            "Steps: 14100\tLoss: 67863.09375\tPPL: 14.56051\tbleu: 8.90866\tLR: 0.00000203\t\n",
            "Steps: 14200\tLoss: 67850.30469\tPPL: 14.55316\tbleu: 8.96697\tLR: 0.00000142\t\n",
            "Steps: 14300\tLoss: 67855.58594\tPPL: 14.55619\tbleu: 8.97200\tLR: 0.00000142\t\n",
            "Steps: 14400\tLoss: 67851.48438\tPPL: 14.55384\tbleu: 8.98772\tLR: 0.00000142\t\n",
            "Steps: 14500\tLoss: 67862.16406\tPPL: 14.55997\tbleu: 8.95805\tLR: 0.00000142\t\n",
            "Steps: 14600\tLoss: 67869.96875\tPPL: 14.56446\tbleu: 8.91694\tLR: 0.00000142\t\n",
            "Steps: 14700\tLoss: 67875.72656\tPPL: 14.56777\tbleu: 8.93576\tLR: 0.00000142\t\n",
            "Steps: 14800\tLoss: 67876.41406\tPPL: 14.56816\tbleu: 8.94185\tLR: 0.00000100\t\n",
            "Steps: 14900\tLoss: 67873.53125\tPPL: 14.56651\tbleu: 8.99964\tLR: 0.00000100\t\n",
            "Steps: 15000\tLoss: 67872.60156\tPPL: 14.56597\tbleu: 8.95616\tLR: 0.00000100\t\n",
            "Steps: 15100\tLoss: 67873.61719\tPPL: 14.56655\tbleu: 9.00390\tLR: 0.00000100\t\n",
            "Steps: 15200\tLoss: 67869.62500\tPPL: 14.56426\tbleu: 8.97571\tLR: 0.00000100\t\n",
            "Steps: 15300\tLoss: 67872.27344\tPPL: 14.56578\tbleu: 8.91668\tLR: 0.00000100\t\n",
            "Steps: 15400\tLoss: 67871.42969\tPPL: 14.56530\tbleu: 8.95969\tLR: 0.00000070\t\n",
            "Steps: 15500\tLoss: 67873.51562\tPPL: 14.56650\tbleu: 8.96878\tLR: 0.00000070\t\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "66WhRE9lIhoD",
        "outputId": "426f9105-3a82-45a0-cc6f-6e979076f5be",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Test our model\n",
        "! cd joeynmt; python3 -m joeynmt test \"$gdrive_path/models/${src}${tgt}_transformer/config.yaml\""
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2020-01-26 03:18:48,811 Hello! This is Joey-NMT.\n",
            "2020-01-26 03:19:11,960  dev bleu:   7.92 [Beam search decoding with beam size = 5 and alpha = 1.0]\n",
            "2020-01-26 03:19:25,058 test bleu:  12.49 [Beam search decoding with beam size = 5 and alpha = 1.0]\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}