{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/Kabongosalomon/masakhane/blob/tshiluba/en-lua/jw300-baseline/salomon_en_lua_notebook.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Igc5itf-xMGj"
   },
   "source": [
    "# Masakhane - Machine Translation for African Languages (Using JoeyNMT)\n",
    "\n",
    "### Languages: English-Tshiluba\n",
    "\n",
    "### Author: Salomon KABONGO KABENAMUALU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "l929HimrxS0a"
   },
   "source": [
    "## Retrieve your data & make a parallel corpus\n",
    "\n",
    "If you are wanting to use the JW300 data referenced on the Masakhane website or in our GitHub repo, you can use `opus-tools` to convert the data into a convenient format. `opus_read` from that package provides a convenient tool for reading the native aligned XML files and to convert them to TMX format. The tool can also be used to fetch relevant files from OPUS on the fly and to filter the data as necessary. [Read the documentation](https://pypi.org/project/opustools-pkg/) for more details.\n",
    "\n",
    "Once you have your corpus files in TMX format (an xml structure which will include the sentences in your target language and your source language in a single file), we recommend reading them into a pandas dataframe. Thankfully, Jade wrote a silly `tmx2dataframe` package which converts your tmx file to a pandas dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 120
    },
    "colab_type": "code",
    "id": "oGRmDELn7Az0",
    "outputId": "cab1986b-af47-430b-c5b1-9639e3807e2c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
      "\n",
      "Enter your authorization code:\n",
      "··········\n",
      "Mounted at /content/drive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Cn3tgQLzUxwn"
   },
   "outputs": [],
   "source": [
    "# TODO: Set your source and target languages. Keep in mind, these traditionally use language codes as found here:\n",
    "# These will also become the suffix's of all vocab and corpus files used throughout\n",
    "import os\n",
    "source_language = \"en\"\n",
    "target_language = \"lua\" \n",
    "lc = False  # If True, lowercase the data.\n",
    "seed = 42  # Random seed for shuffling.\n",
    "tag = \"baseline\" # Give a unique name to your folder - this is to ensure you don't rewrite any models you've already submitted\n",
    "\n",
    "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n",
    "os.environ[\"tgt\"] = target_language\n",
    "os.environ[\"tag\"] = tag\n",
    "\n",
    "# This will save it to a folder in our gdrive instead!\n",
    "!mkdir -p \"/content/drive/My Drive/masakhane/$src-$tgt-$tag\"\n",
    "os.environ[\"gdrive_path\"] = \"/content/drive/My Drive/masakhane/%s-%s-%s\" % (source_language, target_language, tag)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 33
    },
    "colab_type": "code",
    "id": "kBSgJHEw7Nvx",
    "outputId": "a090e69e-9174-4790-a695-853b97c11c2e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/content/drive/My Drive/masakhane/en-lua-baseline\n"
     ]
    }
   ],
   "source": [
    "!echo $gdrive_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 100
    },
    "colab_type": "code",
    "id": "gA75Fs9ys8Y9",
    "outputId": "f4c18fa7-9452-4fd1-b32c-edd84645ee00"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting opustools-pkg\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/6c/9f/e829a0cceccc603450cd18e1ff80807b6237a88d9a8df2c0bb320796e900/opustools_pkg-0.0.52-py3-none-any.whl (80kB)\n",
      "\u001b[K     |████████████████████████████████| 81kB 5.6MB/s eta 0:00:011\n",
      "\u001b[?25hInstalling collected packages: opustools-pkg\n",
      "Successfully installed opustools-pkg-0.0.52\n"
     ]
    }
   ],
   "source": [
    "# Install opus-tools\n",
    "! pip install opustools-pkg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 200
    },
    "colab_type": "code",
    "id": "xq-tDZVks7ZD",
    "outputId": "b2838aa0-05c3-40d5-c26d-190f9366ff43"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Alignment file /proj/nlpl/data/OPUS/JW300/latest/xml/en-lua.xml.gz not found. The following files are available for downloading:\n",
      "\n",
      "   3 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en-lua.xml.gz\n",
      " 263 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en.zip\n",
      "  32 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/lua.zip\n",
      "\n",
      " 298 MB Total size\n",
      "./JW300_latest_xml_en-lua.xml.gz ... 100% of 3 MB\n",
      "./JW300_latest_xml_en.zip ... 100% of 263 MB\n",
      "./JW300_latest_xml_lua.zip ... 100% of 32 MB\n"
     ]
    }
   ],
   "source": [
    "# Downloading our corpus\n",
    "! opus_read -d JW300 -s $src -t $tgt -wm moses -w jw300.$src jw300.$tgt -q\n",
    "\n",
    "# extract the corpus file\n",
    "! gunzip JW300_latest_xml_$src-$tgt.xml.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 566
    },
    "colab_type": "code",
    "id": "n48GDRnP8y2G",
    "outputId": "b7bdafa9-2756-4df4-ad09-bf371126b9af"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-01-02 08:48:18--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 277791 (271K) [text/plain]\n",
      "Saving to: ‘test.en-any.en’\n",
      "\n",
      "test.en-any.en      100%[===================>] 271.28K  --.-KB/s    in 0.02s   \n",
      "\n",
      "2020-01-02 08:48:18 (16.5 MB/s) - ‘test.en-any.en’ saved [277791/277791]\n",
      "\n",
      "--2020-01-02 08:48:22--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-lua.en\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 204380 (200K) [text/plain]\n",
      "Saving to: ‘test.en-lua.en’\n",
      "\n",
      "test.en-lua.en      100%[===================>] 199.59K  --.-KB/s    in 0.02s   \n",
      "\n",
      "2020-01-02 08:48:22 (10.9 MB/s) - ‘test.en-lua.en’ saved [204380/204380]\n",
      "\n",
      "--2020-01-02 08:48:28--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-lua.lua\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 236174 (231K) [text/plain]\n",
      "Saving to: ‘test.en-lua.lua’\n",
      "\n",
      "test.en-lua.lua     100%[===================>] 230.64K  --.-KB/s    in 0.02s   \n",
      "\n",
      "2020-01-02 08:48:29 (14.7 MB/s) - ‘test.en-lua.lua’ saved [236174/236174]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Download the global test set.\n",
    "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n",
    "  \n",
    "# And the specific test set for this language pair.\n",
    "os.environ[\"trg\"] = target_language \n",
    "os.environ[\"src\"] = source_language \n",
    "\n",
    "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.en \n",
    "! mv test.en-$trg.en test.en\n",
    "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.$trg \n",
    "! mv test.en-$trg.$trg test.$trg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 33
    },
    "colab_type": "code",
    "id": "NqDG-CI28y2L",
    "outputId": "54c30eff-00d1-4c6c-954a-a9e86d73bc05"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 3571 global test sentences to filter from the training/dev data.\n"
     ]
    }
   ],
   "source": [
    "# Read the test data to filter from train and dev splits.\n",
    "# Store english portion in set for quick filtering checks.\n",
    "en_test_sents = set()\n",
    "filter_test_sents = \"test.en-any.en\"\n",
    "j = 0\n",
    "with open(filter_test_sents) as f:\n",
    "  for line in f:\n",
    "    en_test_sents.add(line.strip())\n",
    "    j += 1\n",
    "print('Loaded {} global test sentences to filter from the training/dev data.'.format(j))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 66
    },
    "colab_type": "code",
    "id": "6D-_PqdXGJiB",
    "outputId": "dde6c0b8-d957-46fa-90ac-f65f11eaeb7a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drive\t\t\t     JW300_latest_xml_en.zip   sample_data     test.lua\n",
      "jw300.en\t\t     JW300_latest_xml_lua.zip  test.en\n",
      "JW300_latest_xml_en-lua.xml  jw300.lua\t\t       test.en-any.en\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 364
    },
    "colab_type": "code",
    "id": "3CNdwLBCfSIl",
    "outputId": "c27b04a0-217d-41c2-8b0d-6834b31e3aef"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded data and skipped 5835/330019 lines since contained in test set.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source_sentence</th>\n",
       "      <th>target_sentence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Our Planet ​ — What Is Its Future ?</td>\n",
       "      <td>Nditekemena kayi didiku bua buloba buetu ebu ?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>“ While humans for millennia have feared ‘ act...</td>\n",
       "      <td>Bilondeshile tshikandakanda kampanda tshia mu ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The United Nations Environment Programme ( UNE...</td>\n",
       "      <td>( Globe and Mail ) Tshibambalu tshia Ndongamu ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Executive director of UNEP , Klaus Toepfer , s...</td>\n",
       "      <td>Klaus Toepfer , mulombodi wa ndongamu eu udi w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Some environmental progress has been made sinc...</td>\n",
       "      <td>Katshia benza Tshibambalu etshi mu 1972 bua ku...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>As reported in The Toronto Star , “ the qualit...</td>\n",
       "      <td>Nunku , anu mudi tshinga tshikandakanda tshile...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Also , forest management programs , such as th...</td>\n",
       "      <td>Kabidi , ndongamu ya malu a mêtu bu mudi ya mu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Even so , the UNEP report says that if economi...</td>\n",
       "      <td>Nansha nanku , luapolo lua Tshibambalu etshi l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>The Globe stated : “ About half the world’s ri...</td>\n",
       "      <td>( Toronto Star ) Tshinga tshikandakanda tshiak...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Eighty countries holding 40 per cent of the wo...</td>\n",
       "      <td>Mu matunga 80 mudi bia pa lukama 40 bia bantu ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     source_sentence                                    target_sentence\n",
       "0                Our Planet ​ — What Is Its Future ?     Nditekemena kayi didiku bua buloba buetu ebu ?\n",
       "1  “ While humans for millennia have feared ‘ act...  Bilondeshile tshikandakanda kampanda tshia mu ...\n",
       "2  The United Nations Environment Programme ( UNE...  ( Globe and Mail ) Tshibambalu tshia Ndongamu ...\n",
       "3  Executive director of UNEP , Klaus Toepfer , s...  Klaus Toepfer , mulombodi wa ndongamu eu udi w...\n",
       "4  Some environmental progress has been made sinc...  Katshia benza Tshibambalu etshi mu 1972 bua ku...\n",
       "5  As reported in The Toronto Star , “ the qualit...  Nunku , anu mudi tshinga tshikandakanda tshile...\n",
       "6  Also , forest management programs , such as th...  Kabidi , ndongamu ya malu a mêtu bu mudi ya mu...\n",
       "7  Even so , the UNEP report says that if economi...  Nansha nanku , luapolo lua Tshibambalu etshi l...\n",
       "8  The Globe stated : “ About half the world’s ri...  ( Toronto Star ) Tshinga tshikandakanda tshiak...\n",
       "9  Eighty countries holding 40 per cent of the wo...  Mu matunga 80 mudi bia pa lukama 40 bia bantu ..."
      ]
     },
     "execution_count": 9,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# TMX file to dataframe\n",
    "source_file = 'jw300.' + source_language\n",
    "target_file = 'jw300.' + target_language\n",
    "\n",
    "source = []\n",
    "target = []\n",
    "skip_lines = []  # Collect the line numbers of the source portion to skip the same lines for the target portion.\n",
    "with open(source_file) as f:\n",
    "    for i, line in enumerate(f):\n",
    "        # Skip sentences that are contained in the test set.\n",
    "        if line.strip() not in en_test_sents:\n",
    "            source.append(line.strip())\n",
    "        else:\n",
    "            skip_lines.append(i)             \n",
    "with open(target_file) as f:\n",
    "    for j, line in enumerate(f):\n",
    "        # Only add to corpus if corresponding source was not skipped.\n",
    "        if j not in skip_lines:\n",
    "            target.append(line.strip())\n",
    "    \n",
    "print('Loaded data and skipped {}/{} lines since contained in test set.'.format(len(skip_lines), i))\n",
    "    \n",
    "df = pd.DataFrame(zip(source, target), columns=['source_sentence', 'target_sentence'])\n",
    "# if you get TypeError: data argument can't be an iterator is because of your zip version run this below\n",
    "#df = pd.DataFrame(list(zip(source, target)), columns=['source_sentence', 'target_sentence'])\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 33
    },
    "colab_type": "code",
    "id": "hvgFJhMy6178",
    "outputId": "86726c92-981a-4c17-a4e4-ac3f3b0a1a3d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'“ A lover of silver will never be satisfied with silver , nor a lover of wealth with income . ”'"
      ]
     },
     "execution_count": 10,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.source_sentence[100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 33
    },
    "colab_type": "code",
    "id": "luzCebRY7FP0",
    "outputId": "e63b248c-faf1-4498-ae44-b4e18ba75271"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'“ Muntu udi munange arjan kakukuta arjan nansha , anyi muntu udi munange biuma kakukuta tshidiye upeta to . ”'"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.target_sentence[100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 33
    },
    "colab_type": "code",
    "id": "TM6Uy1-W7Pxw",
    "outputId": "403dab91-0fcf-491c-ce28-3f5b98a5ce19"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "324185"
      ]
     },
     "execution_count": 12,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YkuK3B4p2AkN"
   },
   "source": [
    "## Pre-processing and export\n",
    "\n",
    "It is generally a good idea to remove duplicate translations and conflicting translations from the corpus. In practice, these public corpora include some number of these that need to be cleaned.\n",
    "\n",
    "In addition we will split our data into dev/test/train and export to the filesystem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 183
    },
    "colab_type": "code",
    "id": "M_2ouEOH1_1q",
    "outputId": "721019e8-7d01-4e2f-97db-031f0b3036b0"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:6: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  \n",
      "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:7: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  import sys\n"
     ]
    }
   ],
   "source": [
    "# drop duplicate translations\n",
    "df_pp = df.drop_duplicates()\n",
    "\n",
    "# drop conflicting translations\n",
    "# (this is optional and something that you might want to comment out \n",
    "# depending on the size of your corpus)\n",
    "df_pp.drop_duplicates(subset='source_sentence', inplace=True)\n",
    "df_pp.drop_duplicates(subset='target_sentence', inplace=True)\n",
    "\n",
    "# Shuffle the data to remove bias in dev set selection.\n",
    "df_pp = df_pp.sample(frac=1, random_state=seed).reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "Z_1BwAApEtMk",
    "outputId": "c6be289d-01ab-47ca-91fa-ba9d0bde3575"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting fuzzywuzzy\n",
      "  Downloading https://files.pythonhosted.org/packages/d8/f1/5a267addb30ab7eaa1beab2b9323073815da4551076554ecc890a3595ec9/fuzzywuzzy-0.17.0-py2.py3-none-any.whl\n",
      "Installing collected packages: fuzzywuzzy\n",
      "Successfully installed fuzzywuzzy-0.17.0\n",
      "Collecting python-Levenshtein\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/42/a9/d1785c85ebf9b7dfacd08938dd028209c34a0ea3b1bcdb895208bd40a67d/python-Levenshtein-0.12.0.tar.gz (48kB)\n",
      "\u001b[K     |████████████████████████████████| 51kB 4.3MB/s \n",
      "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from python-Levenshtein) (42.0.2)\n",
      "Building wheels for collected packages: python-Levenshtein\n",
      "  Building wheel for python-Levenshtein (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for python-Levenshtein: filename=python_Levenshtein-0.12.0-cp36-cp36m-linux_x86_64.whl size=144665 sha256=22f887c0a65f3e17aff31af915510aaca660b1c7640faab25f7e06d481010a6c\n",
      "  Stored in directory: /root/.cache/pip/wheels/de/c2/93/660fd5f7559049268ad2dc6d81c4e39e9e36518766eaf7e342\n",
      "Successfully built python-Levenshtein\n",
      "Installing collected packages: python-Levenshtein\n",
      "Successfully installed python-Levenshtein-0.12.0\n",
      "00:00:00.11 0.00 percent complete\n",
      "00:00:19.82 0.34 percent complete\n",
      "00:00:39.20 0.68 percent complete\n",
      "00:00:58.69 1.02 percent complete\n",
      "00:01:17.74 1.36 percent complete\n",
      "00:01:36.53 1.70 percent complete\n",
      "00:01:55.66 2.04 percent complete\n",
      "00:02:15.04 2.38 percent complete\n",
      "00:02:34.23 2.72 percent complete\n",
      "00:02:53.52 3.06 percent complete\n",
      "00:03:12.97 3.40 percent complete\n",
      "00:03:32.06 3.74 percent complete\n",
      "00:03:52.09 4.08 percent complete\n",
      "00:04:11.10 4.42 percent complete\n",
      "00:04:30.15 4.76 percent complete\n",
      "00:04:49.32 5.10 percent complete\n",
      "00:05:09.39 5.44 percent complete\n",
      "00:05:28.23 5.78 percent complete\n",
      "00:05:47.63 6.12 percent complete\n",
      "00:06:07.17 6.46 percent complete\n",
      "00:06:26.03 6.80 percent complete\n",
      "00:06:45.21 7.14 percent complete\n",
      "00:07:03.45 7.48 percent complete\n",
      "00:07:22.51 7.82 percent complete\n",
      "00:07:42.13 8.16 percent complete\n",
      "00:08:01.39 8.50 percent complete\n",
      "00:08:20.11 8.84 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '⇩']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00:08:39.05 9.18 percent complete\n",
      "00:08:58.16 9.52 percent complete\n",
      "00:09:17.02 9.86 percent complete\n",
      "00:09:35.73 10.20 percent complete\n",
      "00:09:54.60 10.54 percent complete\n",
      "00:10:14.97 10.88 percent complete\n",
      "00:10:33.72 11.22 percent complete\n",
      "00:10:52.61 11.56 percent complete\n",
      "00:11:11.66 11.89 percent complete\n",
      "00:11:30.51 12.23 percent complete\n",
      "00:11:49.01 12.57 percent complete\n",
      "00:12:07.46 12.91 percent complete\n",
      "00:12:26.61 13.25 percent complete\n",
      "00:12:46.55 13.59 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '*']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00:13:05.73 13.93 percent complete\n",
      "00:13:24.66 14.27 percent complete\n",
      "00:13:43.44 14.61 percent complete\n",
      "00:14:02.20 14.95 percent complete\n",
      "00:14:20.63 15.29 percent complete\n",
      "00:14:39.87 15.63 percent complete\n",
      "00:14:58.84 15.97 percent complete\n",
      "00:15:19.03 16.31 percent complete\n",
      "00:15:38.00 16.65 percent complete\n",
      "00:15:57.15 16.99 percent complete\n",
      "00:16:16.45 17.33 percent complete\n",
      "00:16:35.21 17.67 percent complete\n",
      "00:16:53.90 18.01 percent complete\n",
      "00:17:12.40 18.35 percent complete\n",
      "00:17:31.51 18.69 percent complete\n",
      "00:17:51.08 19.03 percent complete\n",
      "00:18:09.93 19.37 percent complete\n",
      "00:18:28.96 19.71 percent complete\n",
      "00:18:47.58 20.05 percent complete\n",
      "00:19:06.15 20.39 percent complete\n",
      "00:19:25.12 20.73 percent complete\n",
      "00:19:44.31 21.07 percent complete\n",
      "00:20:03.58 21.41 percent complete\n",
      "00:20:24.21 21.75 percent complete\n",
      "00:20:42.98 22.09 percent complete\n",
      "00:21:01.78 22.43 percent complete\n",
      "00:21:21.01 22.77 percent complete\n",
      "00:21:40.49 23.11 percent complete\n",
      "00:21:59.30 23.45 percent complete\n",
      "00:22:18.16 23.79 percent complete\n",
      "00:22:37.12 24.13 percent complete\n",
      "00:22:57.33 24.47 percent complete\n",
      "00:23:16.33 24.81 percent complete\n",
      "00:23:35.47 25.15 percent complete\n",
      "00:23:54.38 25.49 percent complete\n",
      "00:24:14.39 25.83 percent complete\n",
      "00:24:34.62 26.17 percent complete\n",
      "00:24:54.74 26.51 percent complete\n",
      "00:25:14.43 26.85 percent complete\n",
      "00:25:35.59 27.19 percent complete\n",
      "00:25:55.59 27.53 percent complete\n",
      "00:26:15.47 27.87 percent complete\n",
      "00:26:35.02 28.21 percent complete\n",
      "00:26:55.09 28.55 percent complete\n",
      "00:27:14.72 28.89 percent complete\n",
      "00:27:33.79 29.23 percent complete\n",
      "00:27:53.78 29.57 percent complete\n",
      "00:28:13.97 29.91 percent complete\n",
      "00:28:34.04 30.25 percent complete\n",
      "00:28:53.30 30.59 percent complete\n",
      "00:29:13.15 30.93 percent complete\n",
      "00:29:32.89 31.27 percent complete\n",
      "00:29:52.38 31.61 percent complete\n",
      "00:30:11.72 31.95 percent complete\n",
      "00:30:31.19 32.29 percent complete\n",
      "00:30:50.88 32.63 percent complete\n",
      "00:31:11.51 32.97 percent complete\n",
      "00:31:31.48 33.31 percent complete\n",
      "00:31:50.33 33.65 percent complete\n",
      "00:32:10.11 33.99 percent complete\n",
      "00:32:29.81 34.33 percent complete\n",
      "00:32:49.18 34.67 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '․ ․ ․ ․ ․ ․']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00:33:09.44 35.00 percent complete\n",
      "00:33:28.66 35.34 percent complete\n",
      "00:33:49.08 35.68 percent complete\n",
      "00:34:08.21 36.02 percent complete\n",
      "00:34:28.07 36.36 percent complete\n",
      "00:34:47.97 36.70 percent complete\n",
      "00:35:07.64 37.04 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '․ ․']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00:35:27.34 37.38 percent complete\n",
      "00:35:47.06 37.72 percent complete\n",
      "00:36:07.03 38.06 percent complete\n",
      "00:36:28.11 38.40 percent complete\n",
      "00:36:47.71 38.74 percent complete\n",
      "00:37:07.12 39.08 percent complete\n",
      "00:37:26.57 39.42 percent complete\n",
      "00:37:46.25 39.76 percent complete\n",
      "00:38:06.24 40.10 percent complete\n",
      "00:38:25.59 40.44 percent complete\n",
      "00:38:45.46 40.78 percent complete\n",
      "00:39:06.48 41.12 percent complete\n",
      "00:39:26.11 41.46 percent complete\n",
      "00:39:46.05 41.80 percent complete\n",
      "00:40:05.21 42.14 percent complete\n",
      "00:40:24.72 42.48 percent complete\n",
      "00:40:44.27 42.82 percent complete\n",
      "00:41:03.51 43.16 percent complete\n",
      "00:41:22.69 43.50 percent complete\n",
      "00:41:44.08 43.84 percent complete\n",
      "00:42:03.53 44.18 percent complete\n",
      "00:42:23.17 44.52 percent complete\n",
      "00:42:42.10 44.86 percent complete\n",
      "00:43:01.06 45.20 percent complete\n",
      "00:43:20.28 45.54 percent complete\n",
      "00:43:39.35 45.88 percent complete\n",
      "00:43:58.47 46.22 percent complete\n",
      "00:44:19.55 46.56 percent complete\n",
      "00:44:39.13 46.90 percent complete\n",
      "00:44:58.78 47.24 percent complete\n",
      "00:45:18.11 47.58 percent complete\n",
      "00:45:37.52 47.92 percent complete\n",
      "00:45:56.88 48.26 percent complete\n",
      "00:46:15.72 48.60 percent complete\n",
      "00:46:34.97 48.94 percent complete\n",
      "00:46:56.48 49.28 percent complete\n",
      "00:47:16.62 49.62 percent complete\n",
      "00:47:36.12 49.96 percent complete\n",
      "00:47:55.56 50.30 percent complete\n",
      "00:48:16.12 50.64 percent complete\n",
      "00:48:35.25 50.98 percent complete\n",
      "00:48:54.95 51.32 percent complete\n",
      "00:49:14.63 51.66 percent complete\n",
      "00:49:35.42 52.00 percent complete\n",
      "00:49:55.37 52.34 percent complete\n",
      "00:50:14.64 52.68 percent complete\n",
      "00:50:34.60 53.02 percent complete\n",
      "00:50:54.06 53.36 percent complete\n",
      "00:51:13.23 53.70 percent complete\n",
      "00:51:32.73 54.04 percent complete\n",
      "00:51:51.87 54.38 percent complete\n",
      "00:52:12.43 54.72 percent complete\n",
      "00:52:32.15 55.06 percent complete\n",
      "00:52:51.41 55.40 percent complete\n",
      "00:53:11.13 55.74 percent complete\n",
      "00:53:30.68 56.08 percent complete\n",
      "00:53:50.46 56.42 percent complete\n",
      "00:54:10.59 56.76 percent complete\n",
      "00:54:30.00 57.10 percent complete\n",
      "00:54:50.61 57.44 percent complete\n",
      "00:55:11.06 57.78 percent complete\n",
      "00:55:30.39 58.12 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '․ ․ ․ ․ ․']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00:55:50.50 58.45 percent complete\n",
      "00:56:10.14 58.79 percent complete\n",
      "00:56:30.12 59.13 percent complete\n",
      "00:56:49.61 59.47 percent complete\n",
      "00:57:09.14 59.81 percent complete\n",
      "00:57:29.30 60.15 percent complete\n",
      "00:57:49.25 60.49 percent complete\n",
      "00:58:09.50 60.83 percent complete\n",
      "00:58:29.97 61.17 percent complete\n",
      "00:58:50.03 61.51 percent complete\n",
      "00:59:09.70 61.85 percent complete\n",
      "00:59:30.00 62.19 percent complete\n",
      "00:59:49.27 62.53 percent complete\n",
      "01:00:09.48 62.87 percent complete\n",
      "01:00:30.53 63.21 percent complete\n",
      "01:00:49.93 63.55 percent complete\n",
      "01:01:09.33 63.89 percent complete\n",
      "01:01:28.69 64.23 percent complete\n",
      "01:01:48.34 64.57 percent complete\n",
      "01:02:08.75 64.91 percent complete\n",
      "01:02:27.80 65.25 percent complete\n",
      "01:02:48.18 65.59 percent complete\n",
      "01:03:09.48 65.93 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '↓']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "01:03:29.10 66.27 percent complete\n",
      "01:03:48.20 66.61 percent complete\n",
      "01:04:07.81 66.95 percent complete\n",
      "01:04:27.66 67.29 percent complete\n",
      "01:04:47.38 67.63 percent complete\n",
      "01:05:07.33 67.97 percent complete\n",
      "01:05:26.41 68.31 percent complete\n",
      "01:05:47.61 68.65 percent complete\n",
      "01:06:06.71 68.99 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '↓ ↓']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "01:06:26.54 69.33 percent complete\n",
      "01:06:47.00 69.67 percent complete\n",
      "01:07:06.34 70.01 percent complete\n",
      "01:07:26.26 70.35 percent complete\n",
      "01:07:45.25 70.69 percent complete\n",
      "01:08:05.32 71.03 percent complete\n",
      "01:08:26.28 71.37 percent complete\n",
      "01:08:45.19 71.71 percent complete\n",
      "01:09:03.98 72.05 percent complete\n",
      "01:09:24.04 72.39 percent complete\n",
      "01:09:43.90 72.73 percent complete\n",
      "01:10:03.51 73.07 percent complete\n",
      "01:10:22.93 73.41 percent complete\n",
      "01:10:42.70 73.75 percent complete\n",
      "01:11:03.95 74.09 percent complete\n",
      "01:11:22.98 74.43 percent complete\n",
      "01:11:42.22 74.77 percent complete\n",
      "01:12:01.77 75.11 percent complete\n",
      "01:12:22.58 75.45 percent complete\n",
      "01:12:41.59 75.79 percent complete\n",
      "01:13:01.23 76.13 percent complete\n",
      "01:13:21.02 76.47 percent complete\n",
      "01:13:41.38 76.81 percent complete\n",
      "01:14:00.23 77.15 percent complete\n",
      "01:14:19.54 77.49 percent complete\n",
      "01:14:38.85 77.83 percent complete\n",
      "01:14:59.10 78.17 percent complete\n",
      "01:15:19.31 78.51 percent complete\n",
      "01:15:38.36 78.85 percent complete\n",
      "01:15:57.95 79.19 percent complete\n",
      "01:16:19.35 79.53 percent complete\n",
      "01:16:38.47 79.87 percent complete\n",
      "01:16:57.88 80.21 percent complete\n",
      "01:17:17.23 80.55 percent complete\n",
      "01:17:37.39 80.89 percent complete\n",
      "01:17:57.47 81.23 percent complete\n",
      "01:18:16.85 81.56 percent complete\n",
      "01:18:36.05 81.90 percent complete\n",
      "01:18:57.43 82.24 percent complete\n",
      "01:19:16.66 82.58 percent complete\n",
      "01:19:35.92 82.92 percent complete\n",
      "01:19:56.35 83.26 percent complete\n",
      "01:20:16.90 83.60 percent complete\n",
      "01:20:36.32 83.94 percent complete\n",
      "01:20:56.05 84.28 percent complete\n",
      "01:21:15.89 84.62 percent complete\n",
      "01:21:37.36 84.96 percent complete\n",
      "01:21:57.37 85.30 percent complete\n",
      "01:22:17.18 85.64 percent complete\n",
      "01:22:36.67 85.98 percent complete\n",
      "01:22:56.35 86.32 percent complete\n",
      "01:23:16.85 86.66 percent complete\n",
      "01:23:36.26 87.00 percent complete\n",
      "01:23:56.04 87.34 percent complete\n",
      "01:24:17.23 87.68 percent complete\n",
      "01:24:37.35 88.02 percent complete\n",
      "01:24:57.00 88.36 percent complete\n",
      "01:25:16.82 88.70 percent complete\n",
      "01:25:36.62 89.04 percent complete\n",
      "01:25:56.12 89.38 percent complete\n",
      "01:26:15.84 89.72 percent complete\n",
      "01:26:35.89 90.06 percent complete\n",
      "01:26:56.04 90.40 percent complete\n",
      "01:27:15.84 90.74 percent complete\n",
      "01:27:35.29 91.08 percent complete\n",
      "01:27:55.14 91.42 percent complete\n",
      "01:28:14.47 91.76 percent complete\n",
      "01:28:34.40 92.10 percent complete\n",
      "01:28:53.86 92.44 percent complete\n",
      "01:29:13.57 92.78 percent complete\n",
      "01:29:33.90 93.12 percent complete\n",
      "01:29:54.96 93.46 percent complete\n",
      "01:30:15.11 93.80 percent complete\n",
      "01:30:34.77 94.14 percent complete\n",
      "01:30:53.85 94.48 percent complete\n",
      "01:31:13.14 94.82 percent complete\n",
      "01:31:32.80 95.16 percent complete\n",
      "01:31:52.12 95.50 percent complete\n",
      "01:32:11.44 95.84 percent complete\n",
      "01:32:32.95 96.18 percent complete\n",
      "01:32:52.57 96.52 percent complete\n",
      "01:33:12.12 96.86 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "01:33:31.97 97.20 percent complete\n",
      "01:33:51.65 97.54 percent complete\n",
      "01:34:11.63 97.88 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '”']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "01:34:31.02 98.22 percent complete\n",
      "01:34:51.22 98.56 percent complete\n",
      "01:35:12.27 98.90 percent complete\n",
      "01:35:31.84 99.24 percent complete\n",
      "01:35:51.85 99.58 percent complete\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Applied processor reduces input query to empty string, all comparisons will have score 0. [Query: '↓ ↓ ↓ ↓']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "01:36:11.29 99.92 percent complete\n"
     ]
    }
   ],
   "source": [
    "# Install fuzzy wuzzy to remove \"almost duplicate\" sentences in the\n",
    "# test and training sets.\n",
    "! pip install fuzzywuzzy\n",
    "! pip install python-Levenshtein\n",
    "import time\n",
    "from fuzzywuzzy import process\n",
    "import numpy as np\n",
    "\n",
    "# reset the index of the training set after previous filtering\n",
    "df_pp.reset_index(drop=False, inplace=True)\n",
    "\n",
    "# Remove samples from the training data set if they \"almost overlap\" with the\n",
    "# samples in the test set.\n",
    "\n",
    "# Filtering function. Adjust pad to narrow down the candidate matches to\n",
    "# within a certain length of characters of the given sample.\n",
    "def fuzzfilter(sample, candidates, pad):\n",
    "  candidates = [x for x in candidates if len(x) <= len(sample)+pad and len(x) >= len(sample)-pad] \n",
    "  if len(candidates) > 0:\n",
    "    return process.extractOne(sample, candidates)[1]\n",
    "  else:\n",
    "    return np.nan\n",
    "\n",
    "# NOTE - This might run slow depending on the size of your training set. We are\n",
    "# printing some information to help you track how long it would take. \n",
    "scores = []\n",
    "start_time = time.time()\n",
    "for idx, row in df_pp.iterrows():\n",
    "  scores.append(fuzzfilter(row['source_sentence'], list(en_test_sents), 5))\n",
    "  if idx % 1000 == 0:\n",
    "    hours, rem = divmod(time.time() - start_time, 3600)\n",
    "    minutes, seconds = divmod(rem, 60)\n",
    "    print(\"{:0>2}:{:0>2}:{:05.2f}\".format(int(hours),int(minutes),seconds), \"%0.2f percent complete\" % (100.0*float(idx)/float(len(df_pp))))\n",
    "\n",
    "# Filter out \"almost overlapping samples\"\n",
    "df_pp['scores'] = scores\n",
    "df_pp = df_pp[df_pp['scores'] < 95]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 803
    },
    "colab_type": "code",
    "id": "hxxBOCA-xXhy",
    "outputId": "fb058d77-0b84-4803-ac49-43b1b9e05e74"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==> train.en <==\n",
      "He was orderly , industrious , patient , and protective of all those under his care .\n",
      "After giving a thorough witness , he said to the conscience - stricken Jews : “ Repent , and let each one of you be baptized in the name of Jesus Christ for forgiveness of your sins . ”\n",
      "In the original Greek , the word translated “ readjust ” can also refer to the realignment or setting of broken bones , a painful procedure .\n",
      "Why are so many today in need of “ a strengthening aid ” ?\n",
      "As we can see from the case of the Mount Fugen eruption , we need to act on the warning .\n",
      "Submitting to Jesus ’ teachings as his disciples will give our life real meaning and satisfaction .\n",
      "What makes a woman truly praiseworthy ?\n",
      "“ I can’t simply erase memories of my first marriage , especially when we travel to the same places where my ex - husband and I went on vacation , ” says Ellen , who lives in South Africa .\n",
      "It took me a while to see how all the pieces fit together too .\n",
      "How did Jehovah react ?\n",
      "\n",
      "==> train.lua <==\n",
      "Noa uvua ne bulongame , muenji wa mudimu , uvua ne lutulu ne uvua ukuba bantu bonso bavuaye utangila .\n",
      "Kunyima kua Petelo mumane kuyisha kuyishilamu , wakambila bena Yuda ( bakumvua tuondo tuabu tua muoyo tusama ) ne : ‘ Kudimunayi mitshima yenu , nubatijibue buonso buenu mu dîna dia Yezu Kristo bualu bua kunujimijilabu mibi yenu . ’\n",
      "Mu tshiena - Gelika tshia kale , muaku udibu bakudimune ne : “ kupingaja mu njila , ” udi mua kuleja kabidi kululamija anyi kuakaja mifuba idi mitshibuke , bidi bisama bibi .\n",
      "Bua tshinyi bantu bapite bungi lelu badi dijinga ne “ diambuluisha didi dikolesha ” ?\n",
      "Anu mutudi mua kubimuena ku ditudika dia Mukuna wa Fugen , tudi ne bua kutumikila malu adibu batudimuija nawu .\n",
      "Bu mutudi bayidi ba Yezu , kutumikila malongesha ende kudi kutupetesha nsombelu wa nsongo ne wa disanka .\n",
      "Tshidi mua kuvuija mukaji mukumbane kutumbishibua menemene ntshinyi ?\n",
      "Ellen , udi musombele mu Afrique du Sud udi wamba ne : “ Ndi ne lutatu lua kupua muoyo dibaka dianyi dia kumpala , nangananga tuetu baye mu muaba umvua nya ne bayanyi wa kumpala mu dikisha .\n",
      "Nansha meme bivua bingangate matuku bua kumvua mulayi eu bimpe ne dikumbana diawu .\n",
      "Yehowa wakenza tshinyi bua bualu ebu ?\n",
      "==> dev.en <==\n",
      "JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "Such a complete change can be brought about only with the help of God and his spirit .\n",
      "As a forced recruit , he went on food raids and burned down houses .\n",
      "“ Because of the people’s volunteering , praise Jehovah ! ”\n",
      "Are you ready to defend your beliefs ?\n",
      "After the third time , God revealed to Paul that the “ thorn in the flesh ” would not be removed miraculously .\n",
      "Do we resort to human thinking by trying to solve them in our own way ?\n",
      "\n",
      "==> dev.lua <==\n",
      "YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "Bua tuetu kushintuluka bikole mu mushindu eu bidi bikengela bua Nzambi atuambuluishe ne nyuma wende .\n",
      "Pakaluaye musalayi , uvua uya mu dinyenga dia biakudia ne diosha dia nzubu ya bantu .\n",
      "Debola ne Balaka bakatuadija kuimba musambu wa ditshimuna bua kutumbisha Yehowa , kadi ki bantu to .\n",
      "Udi mudilongolole bua kubingisha mayisha enu anyi ?\n",
      "Kunyima kua musangu muisatu , Nzambi wakaleja Paulo ne : ‘ dieba didi mu mubidi wende ’ kadivua ne bua kumbuka mu tshishima to .\n",
      "Mbimpe tuele meji bua bualu ebu : Tutuku imue misangu tudieyemena , tukeba bua kujikija tshilumbu kampanda mu mushindu utudi tumona ne : mmuimpe anyi ?\n"
     ]
    }
   ],
   "source": [
    "# This section does the split between train/dev for the parallel corpora then saves them as separate files\n",
    "# We use 1000 dev test and the given test set.\n",
    "import csv\n",
    "\n",
    "# Do the split between dev/train and create parallel corpora\n",
    "num_dev_patterns = 1000\n",
    "\n",
    "# Optional: lower case the corpora - this will make it easier to generalize, but without proper casing.\n",
    "if lc:  # Julia: making lowercasing optional\n",
    "    df_pp[\"source_sentence\"] = df_pp[\"source_sentence\"].str.lower()\n",
    "    df_pp[\"target_sentence\"] = df_pp[\"target_sentence\"].str.lower()\n",
    "\n",
    "# Julia: test sets are already generated\n",
    "dev = df_pp.tail(num_dev_patterns) # Herman: Error in original\n",
    "stripped = df_pp.drop(df_pp.tail(num_dev_patterns).index)\n",
    "\n",
    "with open(\"train.\"+source_language, \"w\") as src_file, open(\"train.\"+target_language, \"w\") as trg_file:\n",
    "  for index, row in stripped.iterrows():\n",
    "    src_file.write(row[\"source_sentence\"]+\"\\n\")\n",
    "    trg_file.write(row[\"target_sentence\"]+\"\\n\")\n",
    "    \n",
    "with open(\"dev.\"+source_language, \"w\") as src_file, open(\"dev.\"+target_language, \"w\") as trg_file:\n",
    "  for index, row in dev.iterrows():\n",
    "    src_file.write(row[\"source_sentence\"]+\"\\n\")\n",
    "    trg_file.write(row[\"target_sentence\"]+\"\\n\")\n",
    "\n",
    "#stripped[[\"source_sentence\"]].to_csv(\"train.\"+source_language, header=False, index=False)  # Herman: Added `header=False` everywhere\n",
    "#stripped[[\"target_sentence\"]].to_csv(\"train.\"+target_language, header=False, index=False)  # Julia: Problematic handling of quotation marks.\n",
    "\n",
    "#dev[[\"source_sentence\"]].to_csv(\"dev.\"+source_language, header=False, index=False)\n",
    "#dev[[\"target_sentence\"]].to_csv(\"dev.\"+target_language, header=False, index=False)\n",
    "\n",
    "# Doublecheck the format below. There should be no extra quotation marks or weird characters.\n",
    "! head train.*\n",
    "! head dev.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "epeCydmCyS8X"
   },
   "source": [
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "## Installation of JoeyNMT\n",
    "\n",
    "JoeyNMT is a simple, minimalist NMT package which is useful for learning and teaching. Check out the documentation for JoeyNMT [here](https://joeynmt.readthedocs.io)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "iBRMm4kMxZ8L",
    "outputId": "494f3c4d-c639-482a-e725-d066ae2eff10"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fatal: destination path 'joeynmt' already exists and is not an empty directory.\n",
      "Processing /content/joeynmt\n",
      "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.16.0)\n",
      "Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (4.3.0)\n",
      "Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.17.4)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (42.0.2)\n",
      "Requirement already satisfied: torch>=1.1 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.3.1)\n",
      "Requirement already satisfied: tensorflow>=1.14 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.15.0)\n",
      "Requirement already satisfied: torchtext in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.3.1)\n",
      "Collecting sacrebleu>=1.3.6\n",
      "  Downloading https://files.pythonhosted.org/packages/45/31/1a135b964c169984b27fb2f7a50280fa7f8e6d9d404d8a9e596180487fd1/sacrebleu-1.4.3-py3-none-any.whl\n",
      "Collecting subword-nmt\n",
      "  Downloading https://files.pythonhosted.org/packages/74/60/6600a7bc09e7ab38bc53a48a20d8cae49b837f93f5842a41fe513a694912/subword_nmt-0.3.7-py2.py3-none-any.whl\n",
      "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (3.1.2)\n",
      "Requirement already satisfied: seaborn in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.9.0)\n",
      "Collecting pyyaml>=5.1\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB)\n",
      "\u001b[K     |████████████████████████████████| 266kB 7.7MB/s \n",
      "\u001b[?25hCollecting pylint\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl (302kB)\n",
      "\u001b[K     |████████████████████████████████| 307kB 13.2MB/s \n",
      "\u001b[?25hRequirement already satisfied: six==1.12 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.12.0)\n",
      "Requirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow->joeynmt==0.0.1) (0.46)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n",
      "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.0.8)\n",
      "Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n",
      "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n",
      "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.8.1)\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.1.0)\n",
      "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.11.2)\n",
      "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.33.6)\n",
      "Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.1)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.10.0)\n",
      "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.8.1)\n",
      "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.2.2)\n",
      "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.1.8)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (2.21.0)\n",
      "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (4.28.1)\n",
      "Collecting portalocker\n",
      "  Downloading https://files.pythonhosted.org/packages/91/db/7bc703c0760df726839e0699b7f78a4d8217fdc9c7fcb1b51b39c5a22a4e/portalocker-1.5.2-py2.py3-none-any.whl\n",
      "Requirement already satisfied: typing in /usr/local/lib/python3.6/dist-packages (from sacrebleu>=1.3.6->joeynmt==0.0.1) (3.6.6)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.6.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (0.10.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (1.1.0)\n",
      "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.4.5)\n",
      "Requirement already satisfied: scipy>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (1.3.3)\n",
      "Requirement already satisfied: pandas>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (0.25.3)\n",
      "Collecting isort<5,>=4.2.5\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl (42kB)\n",
      "\u001b[K     |████████████████████████████████| 51kB 8.0MB/s \n",
      "\u001b[?25hCollecting mccabe<0.7,>=0.6\n",
      "  Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl\n",
      "Collecting astroid<2.4,>=2.3.0\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ad/ae/86734823047962e7b8c8529186a1ac4a7ca19aaf1aa0c7713c022ef593fd/astroid-2.3.3-py3-none-any.whl (205kB)\n",
      "\u001b[K     |████████████████████████████████| 215kB 21.6MB/s \n",
      "\u001b[?25hRequirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow>=1.14->joeynmt==0.0.1) (2.8.0)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (0.16.0)\n",
      "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (3.1.1)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2.8)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (3.0.4)\n",
      "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (1.24.3)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2019.11.28)\n",
      "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.15.2->seaborn->joeynmt==0.0.1) (2018.9)\n",
      "Collecting typed-ast<1.5,>=1.4.0; implementation_name == \"cpython\" and python_version < \"3.8\"\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/31/d3/9d1802c161626d0278bafb1ffb32f76b9d01e123881bbf9d91e8ccf28e18/typed_ast-1.4.0-cp36-cp36m-manylinux1_x86_64.whl (736kB)\n",
      "\u001b[K     |████████████████████████████████| 737kB 19.9MB/s \n",
      "\u001b[?25hCollecting lazy-object-proxy==1.4.*\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/0b/dd/b1e3407e9e6913cf178e506cd0dee818e58694d9a5cd1984e3f6a8b9a10f/lazy_object_proxy-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (55kB)\n",
      "\u001b[K     |████████████████████████████████| 61kB 8.0MB/s \n",
      "\u001b[?25hBuilding wheels for collected packages: joeynmt, pyyaml\n",
      "  Building wheel for joeynmt (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for joeynmt: filename=joeynmt-0.0.1-cp36-none-any.whl size=72136 sha256=2d995a2a774d7ca37f210debdc7b722704c02354dbfab15bbae9aec40d82a4ee\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-oqdckzxn/wheels/db/01/db/751cc9f3e7f6faec127c43644ba250a3ea7ad200594aeda70a\n",
      "  Building wheel for pyyaml (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for pyyaml: filename=PyYAML-5.2-cp36-cp36m-linux_x86_64.whl size=44209 sha256=b3c4b27e927be9e22911b8d7cd5f57a071b26c84561317d031ee2486ffe701d3\n",
      "  Stored in directory: /root/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67\n",
      "Successfully built joeynmt pyyaml\n",
      "Installing collected packages: portalocker, sacrebleu, subword-nmt, pyyaml, isort, mccabe, typed-ast, lazy-object-proxy, astroid, pylint, joeynmt\n",
      "  Found existing installation: PyYAML 3.13\n",
      "    Uninstalling PyYAML-3.13:\n",
      "      Successfully uninstalled PyYAML-3.13\n",
      "Successfully installed astroid-2.3.3 isort-4.3.21 joeynmt-0.0.1 lazy-object-proxy-1.4.3 mccabe-0.6.1 portalocker-1.5.2 pylint-2.4.4 pyyaml-5.2 sacrebleu-1.4.3 subword-nmt-0.3.7 typed-ast-1.4.0\n"
     ]
    }
   ],
   "source": [
    "# Install JoeyNMT\n",
    "! git clone https://github.com/joeynmt/joeynmt.git\n",
    "! cd joeynmt; pip3 install ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AaE77Tcppex9"
   },
   "source": [
    "# Preprocessing the Data into Subword BPE Tokens\n",
    "\n",
    "- One of the most powerful improvements for agglutinative languages (a feature of most Bantu languages) is using BPE tokenization [ (Sennrich, 2015) ](https://arxiv.org/abs/1508.07909).\n",
    "\n",
    "- It was also shown that by optimizing the umber of BPE codes we significantly improve results for low-resourced languages [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021) [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)\n",
    "\n",
    "- Below we have the scripts for doing BPE tokenization of our data. We use 4000 tokens as recommended by [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021). You do not need to change anything. Simply running the below will be suitable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 400
    },
    "colab_type": "code",
    "id": "H-TyjtmXB1mL",
    "outputId": "401eaee3-a2f7-4860-c6f2-871ada9141e9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bpe.codes.4000\tdev.en\t     test.bpe.lua    test.lua\t    train.en\n",
      "dev.bpe.en\tdev.lua      test.en\t     train.bpe.en   train.lua\n",
      "dev.bpe.lua\ttest.bpe.en  test.en-any.en  train.bpe.lua\n",
      "bpe.codes.4000\tdev.en\t     test.bpe.lua    test.en-any.en.1  train.bpe.lua\n",
      "dev.bpe.en\tdev.lua      test.en\t     test.lua\t       train.en\n",
      "dev.bpe.lua\ttest.bpe.en  test.en-any.en  train.bpe.en      train.lua\n",
      "BPE Tshiluba Sentences\n",
      "N@@ di@@ sungula kayi didi yonso wa kutudi ne bua kuenza ?\n",
      "( Bala Nsumuinu 14 : 5 ; Efeso 4 : 25 . )\n",
      "Badi bapeta “ nyuma ” bu bantu badi ‘ bin@@ dile ne muoyo mujima bua Nzambi ab@@ angate bu bana , ba@@ pi@@ kudi@@ bue ku mi@@ bidi yabu . ’\n",
      "( Bala Lomo 4 : 20 - 22 . )\n",
      "( Bala Bienzedi 18 : 2 - 4 ; 20 : 20 , 21 , 34 , 35 . )\n",
      "Combined BPE Vocab\n",
      "xt\n",
      "›\n",
      "ʼ\n",
      "ḥ\n",
      "ondeshile\n",
      "Â\n",
      "Î\n",
      "ʺ\n",
      "œ@@\n",
      "ansha\n"
     ]
    }
   ],
   "source": [
    "# One of the huge boosts in NMT performance was to use a different method of tokenizing. \n",
    "# Usually, NMT would tokenize by words. However, using a method called BPE gave amazing boosts to performance\n",
    "\n",
    "# Do subword NMT\n",
    "from os import path\n",
    "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n",
    "os.environ[\"tgt\"] = target_language\n",
    "\n",
    "# Learn BPEs on the training data.\n",
    "os.environ[\"data_path\"] = path.join(\"joeynmt\", \"data\", source_language + target_language) # Herman! \n",
    "! subword-nmt learn-joint-bpe-and-vocab --input train.$src train.$tgt -s 4000 -o bpe.codes.4000 --write-vocabulary vocab.$src vocab.$tgt\n",
    "\n",
    "# Apply BPE splits to the development and test data.\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < train.$src > train.bpe.$src\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < train.$tgt > train.bpe.$tgt\n",
    "\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < dev.$src > dev.bpe.$src\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < dev.$tgt > dev.bpe.$tgt\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < test.$src > test.bpe.$src\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt\n",
    "\n",
    "# Create directory, move everyone we care about to the correct location\n",
    "! mkdir -p $data_path\n",
    "! cp train.* $data_path\n",
    "! cp test.* $data_path\n",
    "! cp dev.* $data_path\n",
    "! cp bpe.codes.4000 $data_path\n",
    "! ls $data_path\n",
    "\n",
    "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n",
    "! cp train.* \"$gdrive_path\"\n",
    "! cp test.* \"$gdrive_path\"\n",
    "! cp dev.* \"$gdrive_path\"\n",
    "! cp bpe.codes.4000 \"$gdrive_path\"\n",
    "! ls \"$gdrive_path\"\n",
    "\n",
    "# Create that vocab using build_vocab\n",
    "! sudo chmod 777 joeynmt/scripts/build_vocab.py\n",
    "! joeynmt/scripts/build_vocab.py joeynmt/data/$src$tgt/train.bpe.$src joeynmt/data/$src$tgt/train.bpe.$tgt --output_path joeynmt/data/$src$tgt/vocab.txt\n",
    "\n",
    "# Some output\n",
    "! echo \"BPE Tshiluba Sentences\"\n",
    "! tail -n 5 test.bpe.$tgt\n",
    "! echo \"Combined BPE Vocab\"\n",
    "! tail -n 10 joeynmt/data/$src$tgt/vocab.txt  # Herman"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 66
    },
    "colab_type": "code",
    "id": "IlMitUHR8Qy-",
    "outputId": "d323bcb5-0ae3-4374-b9aa-7c176af1d884"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bpe.codes.4000\tdev.en\t     test.bpe.lua    test.en-any.en.1  train.bpe.lua\n",
      "dev.bpe.en\tdev.lua      test.en\t     test.lua\t       train.en\n",
      "dev.bpe.lua\ttest.bpe.en  test.en-any.en  train.bpe.en      train.lua\n"
     ]
    }
   ],
   "source": [
    "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n",
    "! cp train.* \"$gdrive_path\"\n",
    "! cp test.* \"$gdrive_path\"\n",
    "! cp dev.* \"$gdrive_path\"\n",
    "! cp bpe.codes.4000 \"$gdrive_path\"\n",
    "! ls \"$gdrive_path\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Ixmzi60WsUZ8"
   },
   "source": [
    "# Creating the JoeyNMT Config\n",
    "\n",
    "JoeyNMT requires a yaml config. We provide a template below. We've also set a number of defaults with it, that you may play with!\n",
    "\n",
    "- We used Transformer architecture \n",
    "- We set our dropout to reasonably high: 0.3 (recommended in  [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021))\n",
    "\n",
    "Things worth playing with:\n",
    "- The batch size (also recommended to change for low-resourced languages)\n",
    "- The number of epochs (we've set it at 30 just so it runs in about an hour, for testing purposes)\n",
    "- The decoder options (beam_size, alpha)\n",
    "- Evaluation metrics (BLEU versus Crhf4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PIs1lY2hxMsl"
   },
   "outputs": [],
   "source": [
    "# This creates the config file for our JoeyNMT system. It might seem overwhelming so we've provided a couple of useful parameters you'll need to update\n",
    "# (You can of course play with all the parameters if you'd like!)\n",
    "\n",
    "name = '%s%s' % (source_language, target_language)\n",
    "gdrive_path = os.environ[\"gdrive_path\"]\n",
    "\n",
    "# Create the config\n",
    "config = \"\"\"\n",
    "name: \"{name}_transformer\"\n",
    "\n",
    "data:\n",
    "    src: \"{source_language}\"\n",
    "    trg: \"{target_language}\"\n",
    "    train: \"data/{name}/train.bpe\"\n",
    "    dev:   \"data/{name}/dev.bpe\"\n",
    "    test:  \"data/{name}/test.bpe\"\n",
    "    level: \"bpe\"\n",
    "    lowercase: False\n",
    "    max_sent_length: 100\n",
    "    src_vocab: \"data/{name}/vocab.txt\"\n",
    "    trg_vocab: \"data/{name}/vocab.txt\"\n",
    "\n",
    "testing:\n",
    "    beam_size: 5\n",
    "    alpha: 1.0\n",
    "\n",
    "training:\n",
    "    #load_model: \"{gdrive_path}/models/{name}_transformer/1.ckpt\" # if uncommented, load a pre-trained model from this checkpoint\n",
    "    random_seed: 42\n",
    "    optimizer: \"adam\"\n",
    "    normalization: \"tokens\"\n",
    "    adam_betas: [0.9, 0.999] \n",
    "    scheduling: \"plateau\"           # TODO: try switching from plateau to Noam scheduling\n",
    "    patience: 5                     # For plateau: decrease learning rate by decrease_factor if validation score has not improved for this many validation rounds.\n",
    "    learning_rate_factor: 0.5       # factor for Noam scheduler (used with Transformer)\n",
    "    learning_rate_warmup: 1000      # warmup steps for Noam scheduler (used with Transformer)\n",
    "    decrease_factor: 0.7\n",
    "    loss: \"crossentropy\"\n",
    "    learning_rate: 0.0003\n",
    "    learning_rate_min: 0.00000001\n",
    "    weight_decay: 0.0\n",
    "    label_smoothing: 0.1\n",
    "    batch_size: 4096\n",
    "    batch_type: \"token\"\n",
    "    eval_batch_size: 3600\n",
    "    eval_batch_type: \"token\"\n",
    "    batch_multiplier: 1\n",
    "    early_stopping_metric: \"ppl\"\n",
    "    epochs: 30                     # TODO: Decrease for when playing around and checking of working. Around 30 is sufficient to check if its working at all\n",
    "    validation_freq: 1000          # TODO: Set to at least once per epoch.\n",
    "    logging_freq: 100\n",
    "    eval_metric: \"bleu\"\n",
    "    model_dir: \"models/{name}_transformer\"\n",
    "    overwrite: False               # TODO: Set to True if you want to overwrite possibly existing models. \n",
    "    shuffle: True\n",
    "    use_cuda: True\n",
    "    max_output_length: 100\n",
    "    print_valid_sents: [0, 1, 2, 3]\n",
    "    keep_last_ckpts: 3\n",
    "\n",
    "model:\n",
    "    initializer: \"xavier\"\n",
    "    bias_initializer: \"zeros\"\n",
    "    init_gain: 1.0\n",
    "    embed_initializer: \"xavier\"\n",
    "    embed_init_gain: 1.0\n",
    "    tied_embeddings: True\n",
    "    tied_softmax: True\n",
    "    encoder:\n",
    "        type: \"transformer\"\n",
    "        num_layers: 6\n",
    "        num_heads: 4             # TODO: Increase to 8 for larger data.\n",
    "        embeddings:\n",
    "            embedding_dim: 256   # TODO: Increase to 512 for larger data.\n",
    "            scale: True\n",
    "            dropout: 0.2\n",
    "        # typically ff_size = 4 x hidden_size\n",
    "        hidden_size: 256         # TODO: Increase to 512 for larger data.\n",
    "        ff_size: 1024            # TODO: Increase to 2048 for larger data.\n",
    "        dropout: 0.3\n",
    "    decoder:\n",
    "        type: \"transformer\"\n",
    "        num_layers: 6\n",
    "        num_heads: 4              # TODO: Increase to 8 for larger data.\n",
    "        embeddings:\n",
    "            embedding_dim: 256    # TODO: Increase to 512 for larger data.\n",
    "            scale: True\n",
    "            dropout: 0.2\n",
    "        # typically ff_size = 4 x hidden_size\n",
    "        hidden_size: 256         # TODO: Increase to 512 for larger data.\n",
    "        ff_size: 1024            # TODO: Increase to 2048 for larger data.\n",
    "        dropout: 0.3\n",
    "\"\"\".format(name=name, gdrive_path=os.environ[\"gdrive_path\"], source_language=source_language, target_language=target_language)\n",
    "with open(\"joeynmt/configs/transformer_{name}.yaml\".format(name=name),'w') as f:\n",
    "    f.write(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pIifxE3Qzuvs"
   },
   "source": [
    "# Train the Model\n",
    "\n",
    "This single line of joeynmt runs the training using the config we made above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "6ZBPFwT94WpI",
    "outputId": "b980ae18-66b3-4b27-e4a4-7815ac14ea5f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01-02 10:37:34,915 Hello! This is Joey-NMT.\n",
      "2020-01-02 10:37:36,732 Total params: 12164864\n",
      "2020-01-02 10:37:36,733 Trainable parameters: ['decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.dec_layer_norm.bias', 'decoder.layers.0.dec_layer_norm.weight', 'decoder.layers.0.feed_forward.layer_norm.bias', 'decoder.layers.0.feed_forward.layer_norm.weight', 'decoder.layers.0.feed_forward.pwff_layer.0.bias', 'decoder.layers.0.feed_forward.pwff_layer.0.weight', 'decoder.layers.0.feed_forward.pwff_layer.3.bias', 'decoder.layers.0.feed_forward.pwff_layer.3.weight', 'decoder.layers.0.src_trg_att.k_layer.bias', 'decoder.layers.0.src_trg_att.k_layer.weight', 'decoder.layers.0.src_trg_att.output_layer.bias', 'decoder.layers.0.src_trg_att.output_layer.weight', 'decoder.layers.0.src_trg_att.q_layer.bias', 'decoder.layers.0.src_trg_att.q_layer.weight', 'decoder.layers.0.src_trg_att.v_layer.bias', 'decoder.layers.0.src_trg_att.v_layer.weight', 'decoder.layers.0.trg_trg_att.k_layer.bias', 'decoder.layers.0.trg_trg_att.k_layer.weight', 'decoder.layers.0.trg_trg_att.output_layer.bias', 'decoder.layers.0.trg_trg_att.output_layer.weight', 'decoder.layers.0.trg_trg_att.q_layer.bias', 'decoder.layers.0.trg_trg_att.q_layer.weight', 'decoder.layers.0.trg_trg_att.v_layer.bias', 'decoder.layers.0.trg_trg_att.v_layer.weight', 'decoder.layers.0.x_layer_norm.bias', 'decoder.layers.0.x_layer_norm.weight', 'decoder.layers.1.dec_layer_norm.bias', 'decoder.layers.1.dec_layer_norm.weight', 'decoder.layers.1.feed_forward.layer_norm.bias', 'decoder.layers.1.feed_forward.layer_norm.weight', 'decoder.layers.1.feed_forward.pwff_layer.0.bias', 'decoder.layers.1.feed_forward.pwff_layer.0.weight', 'decoder.layers.1.feed_forward.pwff_layer.3.bias', 'decoder.layers.1.feed_forward.pwff_layer.3.weight', 'decoder.layers.1.src_trg_att.k_layer.bias', 'decoder.layers.1.src_trg_att.k_layer.weight', 'decoder.layers.1.src_trg_att.output_layer.bias', 'decoder.layers.1.src_trg_att.output_layer.weight', 'decoder.layers.1.src_trg_att.q_layer.bias', 'decoder.layers.1.src_trg_att.q_layer.weight', 'decoder.layers.1.src_trg_att.v_layer.bias', 'decoder.layers.1.src_trg_att.v_layer.weight', 'decoder.layers.1.trg_trg_att.k_layer.bias', 'decoder.layers.1.trg_trg_att.k_layer.weight', 'decoder.layers.1.trg_trg_att.output_layer.bias', 'decoder.layers.1.trg_trg_att.output_layer.weight', 'decoder.layers.1.trg_trg_att.q_layer.bias', 'decoder.layers.1.trg_trg_att.q_layer.weight', 'decoder.layers.1.trg_trg_att.v_layer.bias', 'decoder.layers.1.trg_trg_att.v_layer.weight', 'decoder.layers.1.x_layer_norm.bias', 'decoder.layers.1.x_layer_norm.weight', 'decoder.layers.2.dec_layer_norm.bias', 'decoder.layers.2.dec_layer_norm.weight', 'decoder.layers.2.feed_forward.layer_norm.bias', 'decoder.layers.2.feed_forward.layer_norm.weight', 'decoder.layers.2.feed_forward.pwff_layer.0.bias', 'decoder.layers.2.feed_forward.pwff_layer.0.weight', 'decoder.layers.2.feed_forward.pwff_layer.3.bias', 'decoder.layers.2.feed_forward.pwff_layer.3.weight', 'decoder.layers.2.src_trg_att.k_layer.bias', 'decoder.layers.2.src_trg_att.k_layer.weight', 'decoder.layers.2.src_trg_att.output_layer.bias', 'decoder.layers.2.src_trg_att.output_layer.weight', 'decoder.layers.2.src_trg_att.q_layer.bias', 'decoder.layers.2.src_trg_att.q_layer.weight', 'decoder.layers.2.src_trg_att.v_layer.bias', 'decoder.layers.2.src_trg_att.v_layer.weight', 'decoder.layers.2.trg_trg_att.k_layer.bias', 'decoder.layers.2.trg_trg_att.k_layer.weight', 'decoder.layers.2.trg_trg_att.output_layer.bias', 'decoder.layers.2.trg_trg_att.output_layer.weight', 'decoder.layers.2.trg_trg_att.q_layer.bias', 'decoder.layers.2.trg_trg_att.q_layer.weight', 'decoder.layers.2.trg_trg_att.v_layer.bias', 'decoder.layers.2.trg_trg_att.v_layer.weight', 'decoder.layers.2.x_layer_norm.bias', 'decoder.layers.2.x_layer_norm.weight', 'decoder.layers.3.dec_layer_norm.bias', 'decoder.layers.3.dec_layer_norm.weight', 'decoder.layers.3.feed_forward.layer_norm.bias', 'decoder.layers.3.feed_forward.layer_norm.weight', 'decoder.layers.3.feed_forward.pwff_layer.0.bias', 'decoder.layers.3.feed_forward.pwff_layer.0.weight', 'decoder.layers.3.feed_forward.pwff_layer.3.bias', 'decoder.layers.3.feed_forward.pwff_layer.3.weight', 'decoder.layers.3.src_trg_att.k_layer.bias', 'decoder.layers.3.src_trg_att.k_layer.weight', 'decoder.layers.3.src_trg_att.output_layer.bias', 'decoder.layers.3.src_trg_att.output_layer.weight', 'decoder.layers.3.src_trg_att.q_layer.bias', 'decoder.layers.3.src_trg_att.q_layer.weight', 'decoder.layers.3.src_trg_att.v_layer.bias', 'decoder.layers.3.src_trg_att.v_layer.weight', 'decoder.layers.3.trg_trg_att.k_layer.bias', 'decoder.layers.3.trg_trg_att.k_layer.weight', 'decoder.layers.3.trg_trg_att.output_layer.bias', 'decoder.layers.3.trg_trg_att.output_layer.weight', 'decoder.layers.3.trg_trg_att.q_layer.bias', 'decoder.layers.3.trg_trg_att.q_layer.weight', 'decoder.layers.3.trg_trg_att.v_layer.bias', 'decoder.layers.3.trg_trg_att.v_layer.weight', 'decoder.layers.3.x_layer_norm.bias', 'decoder.layers.3.x_layer_norm.weight', 'decoder.layers.4.dec_layer_norm.bias', 'decoder.layers.4.dec_layer_norm.weight', 'decoder.layers.4.feed_forward.layer_norm.bias', 'decoder.layers.4.feed_forward.layer_norm.weight', 'decoder.layers.4.feed_forward.pwff_layer.0.bias', 'decoder.layers.4.feed_forward.pwff_layer.0.weight', 'decoder.layers.4.feed_forward.pwff_layer.3.bias', 'decoder.layers.4.feed_forward.pwff_layer.3.weight', 'decoder.layers.4.src_trg_att.k_layer.bias', 'decoder.layers.4.src_trg_att.k_layer.weight', 'decoder.layers.4.src_trg_att.output_layer.bias', 'decoder.layers.4.src_trg_att.output_layer.weight', 'decoder.layers.4.src_trg_att.q_layer.bias', 'decoder.layers.4.src_trg_att.q_layer.weight', 'decoder.layers.4.src_trg_att.v_layer.bias', 'decoder.layers.4.src_trg_att.v_layer.weight', 'decoder.layers.4.trg_trg_att.k_layer.bias', 'decoder.layers.4.trg_trg_att.k_layer.weight', 'decoder.layers.4.trg_trg_att.output_layer.bias', 'decoder.layers.4.trg_trg_att.output_layer.weight', 'decoder.layers.4.trg_trg_att.q_layer.bias', 'decoder.layers.4.trg_trg_att.q_layer.weight', 'decoder.layers.4.trg_trg_att.v_layer.bias', 'decoder.layers.4.trg_trg_att.v_layer.weight', 'decoder.layers.4.x_layer_norm.bias', 'decoder.layers.4.x_layer_norm.weight', 'decoder.layers.5.dec_layer_norm.bias', 'decoder.layers.5.dec_layer_norm.weight', 'decoder.layers.5.feed_forward.layer_norm.bias', 'decoder.layers.5.feed_forward.layer_norm.weight', 'decoder.layers.5.feed_forward.pwff_layer.0.bias', 'decoder.layers.5.feed_forward.pwff_layer.0.weight', 'decoder.layers.5.feed_forward.pwff_layer.3.bias', 'decoder.layers.5.feed_forward.pwff_layer.3.weight', 'decoder.layers.5.src_trg_att.k_layer.bias', 'decoder.layers.5.src_trg_att.k_layer.weight', 'decoder.layers.5.src_trg_att.output_layer.bias', 'decoder.layers.5.src_trg_att.output_layer.weight', 'decoder.layers.5.src_trg_att.q_layer.bias', 'decoder.layers.5.src_trg_att.q_layer.weight', 'decoder.layers.5.src_trg_att.v_layer.bias', 'decoder.layers.5.src_trg_att.v_layer.weight', 'decoder.layers.5.trg_trg_att.k_layer.bias', 'decoder.layers.5.trg_trg_att.k_layer.weight', 'decoder.layers.5.trg_trg_att.output_layer.bias', 'decoder.layers.5.trg_trg_att.output_layer.weight', 'decoder.layers.5.trg_trg_att.q_layer.bias', 'decoder.layers.5.trg_trg_att.q_layer.weight', 'decoder.layers.5.trg_trg_att.v_layer.bias', 'decoder.layers.5.trg_trg_att.v_layer.weight', 'decoder.layers.5.x_layer_norm.bias', 'decoder.layers.5.x_layer_norm.weight', 'encoder.layer_norm.bias', 'encoder.layer_norm.weight', 'encoder.layers.0.feed_forward.layer_norm.bias', 'encoder.layers.0.feed_forward.layer_norm.weight', 'encoder.layers.0.feed_forward.pwff_layer.0.bias', 'encoder.layers.0.feed_forward.pwff_layer.0.weight', 'encoder.layers.0.feed_forward.pwff_layer.3.bias', 'encoder.layers.0.feed_forward.pwff_layer.3.weight', 'encoder.layers.0.layer_norm.bias', 'encoder.layers.0.layer_norm.weight', 'encoder.layers.0.src_src_att.k_layer.bias', 'encoder.layers.0.src_src_att.k_layer.weight', 'encoder.layers.0.src_src_att.output_layer.bias', 'encoder.layers.0.src_src_att.output_layer.weight', 'encoder.layers.0.src_src_att.q_layer.bias', 'encoder.layers.0.src_src_att.q_layer.weight', 'encoder.layers.0.src_src_att.v_layer.bias', 'encoder.layers.0.src_src_att.v_layer.weight', 'encoder.layers.1.feed_forward.layer_norm.bias', 'encoder.layers.1.feed_forward.layer_norm.weight', 'encoder.layers.1.feed_forward.pwff_layer.0.bias', 'encoder.layers.1.feed_forward.pwff_layer.0.weight', 'encoder.layers.1.feed_forward.pwff_layer.3.bias', 'encoder.layers.1.feed_forward.pwff_layer.3.weight', 'encoder.layers.1.layer_norm.bias', 'encoder.layers.1.layer_norm.weight', 'encoder.layers.1.src_src_att.k_layer.bias', 'encoder.layers.1.src_src_att.k_layer.weight', 'encoder.layers.1.src_src_att.output_layer.bias', 'encoder.layers.1.src_src_att.output_layer.weight', 'encoder.layers.1.src_src_att.q_layer.bias', 'encoder.layers.1.src_src_att.q_layer.weight', 'encoder.layers.1.src_src_att.v_layer.bias', 'encoder.layers.1.src_src_att.v_layer.weight', 'encoder.layers.2.feed_forward.layer_norm.bias', 'encoder.layers.2.feed_forward.layer_norm.weight', 'encoder.layers.2.feed_forward.pwff_layer.0.bias', 'encoder.layers.2.feed_forward.pwff_layer.0.weight', 'encoder.layers.2.feed_forward.pwff_layer.3.bias', 'encoder.layers.2.feed_forward.pwff_layer.3.weight', 'encoder.layers.2.layer_norm.bias', 'encoder.layers.2.layer_norm.weight', 'encoder.layers.2.src_src_att.k_layer.bias', 'encoder.layers.2.src_src_att.k_layer.weight', 'encoder.layers.2.src_src_att.output_layer.bias', 'encoder.layers.2.src_src_att.output_layer.weight', 'encoder.layers.2.src_src_att.q_layer.bias', 'encoder.layers.2.src_src_att.q_layer.weight', 'encoder.layers.2.src_src_att.v_layer.bias', 'encoder.layers.2.src_src_att.v_layer.weight', 'encoder.layers.3.feed_forward.layer_norm.bias', 'encoder.layers.3.feed_forward.layer_norm.weight', 'encoder.layers.3.feed_forward.pwff_layer.0.bias', 'encoder.layers.3.feed_forward.pwff_layer.0.weight', 'encoder.layers.3.feed_forward.pwff_layer.3.bias', 'encoder.layers.3.feed_forward.pwff_layer.3.weight', 'encoder.layers.3.layer_norm.bias', 'encoder.layers.3.layer_norm.weight', 'encoder.layers.3.src_src_att.k_layer.bias', 'encoder.layers.3.src_src_att.k_layer.weight', 'encoder.layers.3.src_src_att.output_layer.bias', 'encoder.layers.3.src_src_att.output_layer.weight', 'encoder.layers.3.src_src_att.q_layer.bias', 'encoder.layers.3.src_src_att.q_layer.weight', 'encoder.layers.3.src_src_att.v_layer.bias', 'encoder.layers.3.src_src_att.v_layer.weight', 'encoder.layers.4.feed_forward.layer_norm.bias', 'encoder.layers.4.feed_forward.layer_norm.weight', 'encoder.layers.4.feed_forward.pwff_layer.0.bias', 'encoder.layers.4.feed_forward.pwff_layer.0.weight', 'encoder.layers.4.feed_forward.pwff_layer.3.bias', 'encoder.layers.4.feed_forward.pwff_layer.3.weight', 'encoder.layers.4.layer_norm.bias', 'encoder.layers.4.layer_norm.weight', 'encoder.layers.4.src_src_att.k_layer.bias', 'encoder.layers.4.src_src_att.k_layer.weight', 'encoder.layers.4.src_src_att.output_layer.bias', 'encoder.layers.4.src_src_att.output_layer.weight', 'encoder.layers.4.src_src_att.q_layer.bias', 'encoder.layers.4.src_src_att.q_layer.weight', 'encoder.layers.4.src_src_att.v_layer.bias', 'encoder.layers.4.src_src_att.v_layer.weight', 'encoder.layers.5.feed_forward.layer_norm.bias', 'encoder.layers.5.feed_forward.layer_norm.weight', 'encoder.layers.5.feed_forward.pwff_layer.0.bias', 'encoder.layers.5.feed_forward.pwff_layer.0.weight', 'encoder.layers.5.feed_forward.pwff_layer.3.bias', 'encoder.layers.5.feed_forward.pwff_layer.3.weight', 'encoder.layers.5.layer_norm.bias', 'encoder.layers.5.layer_norm.weight', 'encoder.layers.5.src_src_att.k_layer.bias', 'encoder.layers.5.src_src_att.k_layer.weight', 'encoder.layers.5.src_src_att.output_layer.bias', 'encoder.layers.5.src_src_att.output_layer.weight', 'encoder.layers.5.src_src_att.q_layer.bias', 'encoder.layers.5.src_src_att.q_layer.weight', 'encoder.layers.5.src_src_att.v_layer.bias', 'encoder.layers.5.src_src_att.v_layer.weight', 'src_embed.lut.weight']\n",
      "2020-01-02 10:37:45,727 cfg.name                           : enlua_transformer\n",
      "2020-01-02 10:37:45,727 cfg.data.src                       : en\n",
      "2020-01-02 10:37:45,727 cfg.data.trg                       : lua\n",
      "2020-01-02 10:37:45,728 cfg.data.train                     : data/enlua/train.bpe\n",
      "2020-01-02 10:37:45,728 cfg.data.dev                       : data/enlua/dev.bpe\n",
      "2020-01-02 10:37:45,728 cfg.data.test                      : data/enlua/test.bpe\n",
      "2020-01-02 10:37:45,728 cfg.data.level                     : bpe\n",
      "2020-01-02 10:37:45,728 cfg.data.lowercase                 : False\n",
      "2020-01-02 10:37:45,728 cfg.data.max_sent_length           : 100\n",
      "2020-01-02 10:37:45,728 cfg.data.src_vocab                 : data/enlua/vocab.txt\n",
      "2020-01-02 10:37:45,728 cfg.data.trg_vocab                 : data/enlua/vocab.txt\n",
      "2020-01-02 10:37:45,728 cfg.testing.beam_size              : 5\n",
      "2020-01-02 10:37:45,728 cfg.testing.alpha                  : 1.0\n",
      "2020-01-02 10:37:45,728 cfg.training.random_seed           : 42\n",
      "2020-01-02 10:37:45,728 cfg.training.optimizer             : adam\n",
      "2020-01-02 10:37:45,728 cfg.training.normalization         : tokens\n",
      "2020-01-02 10:37:45,728 cfg.training.adam_betas            : [0.9, 0.999]\n",
      "2020-01-02 10:37:45,728 cfg.training.scheduling            : plateau\n",
      "2020-01-02 10:37:45,728 cfg.training.patience              : 5\n",
      "2020-01-02 10:37:45,728 cfg.training.learning_rate_factor  : 0.5\n",
      "2020-01-02 10:37:45,728 cfg.training.learning_rate_warmup  : 1000\n",
      "2020-01-02 10:37:45,728 cfg.training.decrease_factor       : 0.7\n",
      "2020-01-02 10:37:45,729 cfg.training.loss                  : crossentropy\n",
      "2020-01-02 10:37:45,729 cfg.training.learning_rate         : 0.0003\n",
      "2020-01-02 10:37:45,729 cfg.training.learning_rate_min     : 1e-08\n",
      "2020-01-02 10:37:45,729 cfg.training.weight_decay          : 0.0\n",
      "2020-01-02 10:37:45,729 cfg.training.label_smoothing       : 0.1\n",
      "2020-01-02 10:37:45,729 cfg.training.batch_size            : 4096\n",
      "2020-01-02 10:37:45,729 cfg.training.batch_type            : token\n",
      "2020-01-02 10:37:45,729 cfg.training.eval_batch_size       : 3600\n",
      "2020-01-02 10:37:45,729 cfg.training.eval_batch_type       : token\n",
      "2020-01-02 10:37:45,729 cfg.training.batch_multiplier      : 1\n",
      "2020-01-02 10:37:45,729 cfg.training.early_stopping_metric : ppl\n",
      "2020-01-02 10:37:45,729 cfg.training.epochs                : 30\n",
      "2020-01-02 10:37:45,729 cfg.training.validation_freq       : 1000\n",
      "2020-01-02 10:37:45,729 cfg.training.logging_freq          : 100\n",
      "2020-01-02 10:37:45,729 cfg.training.eval_metric           : bleu\n",
      "2020-01-02 10:37:45,729 cfg.training.model_dir             : models/enlua_transformer\n",
      "2020-01-02 10:37:45,729 cfg.training.overwrite             : False\n",
      "2020-01-02 10:37:45,729 cfg.training.shuffle               : True\n",
      "2020-01-02 10:37:45,729 cfg.training.use_cuda              : True\n",
      "2020-01-02 10:37:45,730 cfg.training.max_output_length     : 100\n",
      "2020-01-02 10:37:45,730 cfg.training.print_valid_sents     : [0, 1, 2, 3]\n",
      "2020-01-02 10:37:45,730 cfg.training.keep_last_ckpts       : 3\n",
      "2020-01-02 10:37:45,730 cfg.model.initializer              : xavier\n",
      "2020-01-02 10:37:45,730 cfg.model.bias_initializer         : zeros\n",
      "2020-01-02 10:37:45,730 cfg.model.init_gain                : 1.0\n",
      "2020-01-02 10:37:45,730 cfg.model.embed_initializer        : xavier\n",
      "2020-01-02 10:37:45,730 cfg.model.embed_init_gain          : 1.0\n",
      "2020-01-02 10:37:45,730 cfg.model.tied_embeddings          : True\n",
      "2020-01-02 10:37:45,730 cfg.model.tied_softmax             : True\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.type             : transformer\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.num_layers       : 6\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.num_heads        : 4\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.embeddings.embedding_dim : 256\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.embeddings.scale : True\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.embeddings.dropout : 0.2\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.hidden_size      : 256\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.ff_size          : 1024\n",
      "2020-01-02 10:37:45,730 cfg.model.encoder.dropout          : 0.3\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.type             : transformer\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.num_layers       : 6\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.num_heads        : 4\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.embeddings.embedding_dim : 256\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.embeddings.scale : True\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.embeddings.dropout : 0.2\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.hidden_size      : 256\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.ff_size          : 1024\n",
      "2020-01-02 10:37:45,731 cfg.model.decoder.dropout          : 0.3\n",
      "2020-01-02 10:37:45,731 Data set sizes: \n",
      "\ttrain 291883,\n",
      "\tvalid 1000,\n",
      "\ttest 2699\n",
      "2020-01-02 10:37:45,731 First training example:\n",
      "\t[SRC] He was or@@ der@@ ly , in@@ du@@ stri@@ ous , pati@@ ent , and prote@@ ctive of all those under his care .\n",
      "\t[TRG] Noa uvua ne bul@@ ong@@ ame , mu@@ enji wa mudimu , uvua ne lutulu ne uvua u@@ ku@@ ba bantu bonso ba@@ vuaye utangila .\n",
      "2020-01-02 10:37:45,731 First 10 words (src): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) . (5) , (6) ne (7) the (8) to (9) bua\n",
      "2020-01-02 10:37:45,732 First 10 words (trg): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) . (5) , (6) ne (7) the (8) to (9) bua\n",
      "2020-01-02 10:37:45,732 Number of Src words (types): 4315\n",
      "2020-01-02 10:37:45,732 Number of Trg words (types): 4315\n",
      "2020-01-02 10:37:45,732 Model(\n",
      "\tencoder=TransformerEncoder(num_layers=6, num_heads=4),\n",
      "\tdecoder=TransformerDecoder(num_layers=6, num_heads=4),\n",
      "\tsrc_embed=Embeddings(embedding_dim=256, vocab_size=4315),\n",
      "\ttrg_embed=Embeddings(embedding_dim=256, vocab_size=4315))\n",
      "2020-01-02 10:37:45,735 EPOCH 1\n",
      "2020-01-02 10:37:57,996 Epoch   1 Step:      100 Batch Loss:     5.568820 Tokens per Sec:    18596, Lr: 0.000300\n",
      "2020-01-02 10:38:09,739 Epoch   1 Step:      200 Batch Loss:     5.543124 Tokens per Sec:    19594, Lr: 0.000300\n",
      "2020-01-02 10:38:21,498 Epoch   1 Step:      300 Batch Loss:     5.335598 Tokens per Sec:    19706, Lr: 0.000300\n",
      "2020-01-02 10:38:33,205 Epoch   1 Step:      400 Batch Loss:     4.973787 Tokens per Sec:    19327, Lr: 0.000300\n",
      "2020-01-02 10:38:44,935 Epoch   1 Step:      500 Batch Loss:     4.586766 Tokens per Sec:    19781, Lr: 0.000300\n",
      "2020-01-02 10:38:56,668 Epoch   1 Step:      600 Batch Loss:     4.488767 Tokens per Sec:    19844, Lr: 0.000300\n",
      "2020-01-02 10:39:08,351 Epoch   1 Step:      700 Batch Loss:     4.129436 Tokens per Sec:    19261, Lr: 0.000300\n",
      "2020-01-02 10:39:20,106 Epoch   1 Step:      800 Batch Loss:     4.127619 Tokens per Sec:    19713, Lr: 0.000300\n",
      "2020-01-02 10:39:31,836 Epoch   1 Step:      900 Batch Loss:     4.124986 Tokens per Sec:    19673, Lr: 0.000300\n",
      "2020-01-02 10:39:43,558 Epoch   1 Step:     1000 Batch Loss:     4.010236 Tokens per Sec:    19012, Lr: 0.000300\n",
      "2020-01-02 10:40:12,878 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:40:12,878 Saving new checkpoint.\n",
      "2020-01-02 10:40:13,113 Example #0\n",
      "2020-01-02 10:40:13,114 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:40:13,114 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:40:13,114 \tHypothesis: ( MU : “ NUUUUUUUUUU udi wamba ne : “ NUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU@@\n",
      "2020-01-02 10:40:13,114 Example #1\n",
      "2020-01-02 10:40:13,114 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:40:13,114 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:40:13,114 \tHypothesis: Kadi , wakamumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumumuudi umumumumumumumumumumumumumumumumumumumumumumumuudi umuudi umumumumumumumumumumumumumumumumumumumumumumumu@@\n",
      "2020-01-02 10:40:13,114 Example #2\n",
      "2020-01-02 10:40:13,114 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:40:13,114 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:40:13,114 \tHypothesis: Kadi tudi ne bua kuikala ne : Yehowa udi ne bua kukusankisha malu a bungi .\n",
      "2020-01-02 10:40:13,114 Example #3\n",
      "2020-01-02 10:40:13,115 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:40:13,115 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:40:13,115 \tHypothesis: Kadi tudi ne bua kukusankisha malu a bungi .\n",
      "2020-01-02 10:40:13,115 Validation result (greedy) at epoch   1, step     1000: bleu:   1.13, loss: 102899.7891, ppl:  53.8499, duration: 29.5559s\n",
      "2020-01-02 10:40:24,829 Epoch   1 Step:     1100 Batch Loss:     3.886719 Tokens per Sec:    19671, Lr: 0.000300\n",
      "2020-01-02 10:40:36,593 Epoch   1 Step:     1200 Batch Loss:     3.979998 Tokens per Sec:    19356, Lr: 0.000300\n",
      "2020-01-02 10:40:48,203 Epoch   1 Step:     1300 Batch Loss:     3.723805 Tokens per Sec:    19080, Lr: 0.000300\n",
      "2020-01-02 10:41:00,009 Epoch   1 Step:     1400 Batch Loss:     3.545810 Tokens per Sec:    19340, Lr: 0.000300\n",
      "2020-01-02 10:41:11,797 Epoch   1 Step:     1500 Batch Loss:     3.405194 Tokens per Sec:    19436, Lr: 0.000300\n",
      "2020-01-02 10:41:23,536 Epoch   1 Step:     1600 Batch Loss:     3.514859 Tokens per Sec:    19857, Lr: 0.000300\n",
      "2020-01-02 10:41:35,348 Epoch   1 Step:     1700 Batch Loss:     3.266917 Tokens per Sec:    19763, Lr: 0.000300\n",
      "2020-01-02 10:41:47,120 Epoch   1 Step:     1800 Batch Loss:     3.467505 Tokens per Sec:    20093, Lr: 0.000300\n",
      "2020-01-02 10:41:58,899 Epoch   1 Step:     1900 Batch Loss:     3.545438 Tokens per Sec:    19721, Lr: 0.000300\n",
      "2020-01-02 10:42:10,513 Epoch   1 Step:     2000 Batch Loss:     3.089291 Tokens per Sec:    19290, Lr: 0.000300\n",
      "2020-01-02 10:42:39,717 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:42:39,717 Saving new checkpoint.\n",
      "2020-01-02 10:42:39,929 Example #0\n",
      "2020-01-02 10:42:39,929 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:42:39,929 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:42:39,929 \tHypothesis: DIBAKU bena Kristo ba mu tshisumbu tshia bena Kristo ba mu diulu , kadi ‘ kuenu , kadi nuikale ne muoyo , kadi nuikale ne bua kuitabuja . ’\n",
      "2020-01-02 10:42:39,930 Example #1\n",
      "2020-01-02 10:42:39,930 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:42:39,930 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:42:39,930 \tHypothesis: Bantu ba bungi bavua babababababababaye ne : bavua babababababapeshe .\n",
      "2020-01-02 10:42:39,930 Example #2\n",
      "2020-01-02 10:42:39,930 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:42:39,930 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:42:39,930 \tHypothesis: Nunku , Yezu uvua ne bua kuvuluka ne : Yehowa uvua mupeshe muntu udi ujinga bua kukokesha , kadi udi ujinga bua kukokesha .\n",
      "2020-01-02 10:42:39,930 Example #3\n",
      "2020-01-02 10:42:39,931 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:42:39,931 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:42:39,931 \tHypothesis: Mvua ne bua kuikala ne disanka dia bungi bua kukokesha mu diulu .\n",
      "2020-01-02 10:42:39,931 Validation result (greedy) at epoch   1, step     2000: bleu:   2.73, loss: 84754.0547, ppl:  26.6625, duration: 29.4180s\n",
      "2020-01-02 10:42:51,653 Epoch   1 Step:     2100 Batch Loss:     3.841846 Tokens per Sec:    19616, Lr: 0.000300\n",
      "2020-01-02 10:43:03,442 Epoch   1 Step:     2200 Batch Loss:     3.531554 Tokens per Sec:    19607, Lr: 0.000300\n",
      "2020-01-02 10:43:15,172 Epoch   1 Step:     2300 Batch Loss:     3.364511 Tokens per Sec:    19320, Lr: 0.000300\n",
      "2020-01-02 10:43:26,880 Epoch   1 Step:     2400 Batch Loss:     2.885530 Tokens per Sec:    19526, Lr: 0.000300\n",
      "2020-01-02 10:43:38,688 Epoch   1 Step:     2500 Batch Loss:     2.853201 Tokens per Sec:    19567, Lr: 0.000300\n",
      "2020-01-02 10:43:50,375 Epoch   1 Step:     2600 Batch Loss:     2.759443 Tokens per Sec:    19433, Lr: 0.000300\n",
      "2020-01-02 10:44:02,152 Epoch   1 Step:     2700 Batch Loss:     2.676429 Tokens per Sec:    19597, Lr: 0.000300\n",
      "2020-01-02 10:44:13,962 Epoch   1 Step:     2800 Batch Loss:     3.233649 Tokens per Sec:    19609, Lr: 0.000300\n",
      "2020-01-02 10:44:25,678 Epoch   1 Step:     2900 Batch Loss:     3.197888 Tokens per Sec:    19341, Lr: 0.000300\n",
      "2020-01-02 10:44:37,311 Epoch   1 Step:     3000 Batch Loss:     3.828218 Tokens per Sec:    19235, Lr: 0.000300\n",
      "2020-01-02 10:45:06,625 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:45:06,625 Saving new checkpoint.\n",
      "2020-01-02 10:45:06,833 Example #0\n",
      "2020-01-02 10:45:06,833 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:45:06,834 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:45:06,834 \tHypothesis: DIKU TSHIA NE NE NE NE NE NNE NNNNE NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN@@\n",
      "2020-01-02 10:45:06,834 Example #1\n",
      "2020-01-02 10:45:06,834 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:45:06,834 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:45:06,834 \tHypothesis: Bu muvua bana bende bakulakaje bavuabu bababababababababababababababababababapeshe .\n",
      "2020-01-02 10:45:06,834 Example #2\n",
      "2020-01-02 10:45:06,834 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:45:06,834 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:45:06,835 \tHypothesis: Nunku , Mose uvua mumanye ne : Yehowa uvua ne bua kujingulula ne : uvua ne bua kubutula bantu bende to , kadi kavua ne bua kupepeja to .\n",
      "2020-01-02 10:45:06,835 Example #3\n",
      "2020-01-02 10:45:06,835 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:45:06,835 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:45:06,835 \tHypothesis: Mvua ne bua kuikala ne disanka dia bungi bua kupingana ku lufu lua Yehowa .\n",
      "2020-01-02 10:45:06,835 Validation result (greedy) at epoch   1, step     3000: bleu:   5.22, loss: 75980.5391, ppl:  18.9800, duration: 29.5233s\n",
      "2020-01-02 10:45:18,673 Epoch   1 Step:     3100 Batch Loss:     2.895209 Tokens per Sec:    19486, Lr: 0.000300\n",
      "2020-01-02 10:45:30,473 Epoch   1 Step:     3200 Batch Loss:     3.172625 Tokens per Sec:    19501, Lr: 0.000300\n",
      "2020-01-02 10:45:40,245 Epoch   1: total training loss 12640.40\n",
      "2020-01-02 10:45:40,245 EPOCH 2\n",
      "2020-01-02 10:45:42,622 Epoch   2 Step:     3300 Batch Loss:     3.111502 Tokens per Sec:    17353, Lr: 0.000300\n",
      "2020-01-02 10:45:54,436 Epoch   2 Step:     3400 Batch Loss:     3.205148 Tokens per Sec:    19518, Lr: 0.000300\n",
      "2020-01-02 10:46:06,167 Epoch   2 Step:     3500 Batch Loss:     3.197549 Tokens per Sec:    19328, Lr: 0.000300\n",
      "2020-01-02 10:46:17,962 Epoch   2 Step:     3600 Batch Loss:     3.011550 Tokens per Sec:    19227, Lr: 0.000300\n",
      "2020-01-02 10:46:29,781 Epoch   2 Step:     3700 Batch Loss:     2.917604 Tokens per Sec:    19655, Lr: 0.000300\n",
      "2020-01-02 10:46:41,547 Epoch   2 Step:     3800 Batch Loss:     2.838443 Tokens per Sec:    19354, Lr: 0.000300\n",
      "2020-01-02 10:46:53,311 Epoch   2 Step:     3900 Batch Loss:     2.614524 Tokens per Sec:    19353, Lr: 0.000300\n",
      "2020-01-02 10:47:05,084 Epoch   2 Step:     4000 Batch Loss:     3.049827 Tokens per Sec:    19562, Lr: 0.000300\n",
      "2020-01-02 10:47:34,378 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:47:34,378 Saving new checkpoint.\n",
      "2020-01-02 10:47:34,635 Example #0\n",
      "2020-01-02 10:47:34,635 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:47:34,636 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:47:34,637 \tHypothesis: YEJI udi wamba bua bena Kristo ba mu dîku diende ne : ‘ Nengenu , kadi ndi nengelela Tatu wanyi , kadi nudi utshina , kadi nudi utshina . ’\n",
      "2020-01-02 10:47:34,637 Example #1\n",
      "2020-01-02 10:47:34,637 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:47:34,637 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:47:34,637 \tHypothesis: Bu muvua bana bende bamulamate , bakamusaka bua kumulamate ku bupika bua kumulonda .\n",
      "2020-01-02 10:47:34,637 Example #2\n",
      "2020-01-02 10:47:34,637 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:47:34,637 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:47:34,637 \tHypothesis: Nunku , Mose uvua mua kuikala mumanye ne : Yehowa kêna mua kujingulula ne : uvua mupuangane , kadi udi utshina bua mpekatu wende to .\n",
      "2020-01-02 10:47:34,638 Example #3\n",
      "2020-01-02 10:47:34,638 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:47:34,638 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:47:34,638 \tHypothesis: Ndi ne bua kuvuluka ne : Yehowa udi uvuluka lufu lua lufu lua lufu lua lufu .\n",
      "2020-01-02 10:47:34,638 Validation result (greedy) at epoch   2, step     4000: bleu:   7.78, loss: 70149.6797, ppl:  15.1425, duration: 29.5533s\n",
      "2020-01-02 10:47:46,433 Epoch   2 Step:     4100 Batch Loss:     3.186476 Tokens per Sec:    19528, Lr: 0.000300\n",
      "2020-01-02 10:47:58,100 Epoch   2 Step:     4200 Batch Loss:     3.134429 Tokens per Sec:    19397, Lr: 0.000300\n",
      "2020-01-02 10:48:09,803 Epoch   2 Step:     4300 Batch Loss:     3.032674 Tokens per Sec:    19390, Lr: 0.000300\n",
      "2020-01-02 10:48:21,486 Epoch   2 Step:     4400 Batch Loss:     2.868320 Tokens per Sec:    19159, Lr: 0.000300\n",
      "2020-01-02 10:48:33,267 Epoch   2 Step:     4500 Batch Loss:     2.594971 Tokens per Sec:    19551, Lr: 0.000300\n",
      "2020-01-02 10:48:45,014 Epoch   2 Step:     4600 Batch Loss:     2.938017 Tokens per Sec:    19453, Lr: 0.000300\n",
      "2020-01-02 10:48:56,762 Epoch   2 Step:     4700 Batch Loss:     2.857719 Tokens per Sec:    19638, Lr: 0.000300\n",
      "2020-01-02 10:49:08,537 Epoch   2 Step:     4800 Batch Loss:     2.946003 Tokens per Sec:    19229, Lr: 0.000300\n",
      "2020-01-02 10:49:20,371 Epoch   2 Step:     4900 Batch Loss:     2.926871 Tokens per Sec:    19556, Lr: 0.000300\n",
      "2020-01-02 10:49:32,106 Epoch   2 Step:     5000 Batch Loss:     2.807736 Tokens per Sec:    19248, Lr: 0.000300\n",
      "2020-01-02 10:50:01,354 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:50:01,355 Saving new checkpoint.\n",
      "2020-01-02 10:50:01,610 Example #0\n",
      "2020-01-02 10:50:01,611 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:50:01,611 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:50:01,611 \tHypothesis: YEJI wakasambila bena Kristo bonso bua kusambila ne : ‘ Tatu wanyi , kadi Tatu wanyi , kadi Tatu wanyi , kadi yeye neakuambuluishe . ’\n",
      "2020-01-02 10:50:01,611 Example #1\n",
      "2020-01-02 10:50:01,611 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:50:01,612 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:50:01,612 \tHypothesis: Bu muvua bana bende ne bena Kristo ba mu buloba , bakamuambila ne : bavua bamuenzela mudimu ne muoyo mujima .\n",
      "2020-01-02 10:50:01,612 Example #2\n",
      "2020-01-02 10:50:01,612 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:50:01,612 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:50:01,612 \tHypothesis: Nunku Mose uvua mumanye ne : Yehowa uvua mua kujingulula malu mabi , kadi yeye ke yeye kayi kayi utshina bubi .\n",
      "2020-01-02 10:50:01,612 Example #3\n",
      "2020-01-02 10:50:01,612 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:50:01,612 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:50:01,613 \tHypothesis: Ndi ne dianyisha dia bungi bua mudi Yehowa uvuluka dibishibua dia bafue .\n",
      "2020-01-02 10:50:01,613 Validation result (greedy) at epoch   2, step     5000: bleu:   9.80, loss: 66669.0469, ppl:  13.2324, duration: 29.5063s\n",
      "2020-01-02 10:50:13,464 Epoch   2 Step:     5100 Batch Loss:     2.641332 Tokens per Sec:    19576, Lr: 0.000300\n",
      "2020-01-02 10:50:25,252 Epoch   2 Step:     5200 Batch Loss:     2.893947 Tokens per Sec:    19594, Lr: 0.000300\n",
      "2020-01-02 10:50:37,089 Epoch   2 Step:     5300 Batch Loss:     2.828562 Tokens per Sec:    19253, Lr: 0.000300\n",
      "2020-01-02 10:50:48,786 Epoch   2 Step:     5400 Batch Loss:     2.512304 Tokens per Sec:    19157, Lr: 0.000300\n",
      "2020-01-02 10:51:00,612 Epoch   2 Step:     5500 Batch Loss:     2.830184 Tokens per Sec:    19307, Lr: 0.000300\n",
      "2020-01-02 10:51:12,346 Epoch   2 Step:     5600 Batch Loss:     2.182584 Tokens per Sec:    19841, Lr: 0.000300\n",
      "2020-01-02 10:51:23,976 Epoch   2 Step:     5700 Batch Loss:     2.062052 Tokens per Sec:    18998, Lr: 0.000300\n",
      "2020-01-02 10:51:35,931 Epoch   2 Step:     5800 Batch Loss:     2.128184 Tokens per Sec:    19616, Lr: 0.000300\n",
      "2020-01-02 10:51:47,634 Epoch   2 Step:     5900 Batch Loss:     3.006864 Tokens per Sec:    19245, Lr: 0.000300\n",
      "2020-01-02 10:51:59,530 Epoch   2 Step:     6000 Batch Loss:     2.412691 Tokens per Sec:    19736, Lr: 0.000300\n",
      "2020-01-02 10:52:28,734 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:52:28,734 Saving new checkpoint.\n",
      "2020-01-02 10:52:29,009 Example #0\n",
      "2020-01-02 10:52:29,010 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:52:29,010 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:52:29,010 \tHypothesis: YEJI wakafila bena Kristo bonso bua kusambila Tatu wetu ne : ‘ Kanuakulekelayi Tatu wanyi , kadi nuakule . ’\n",
      "2020-01-02 10:52:29,010 Example #1\n",
      "2020-01-02 10:52:29,010 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:52:29,010 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:52:29,010 \tHypothesis: Bu muvua bana bende bamutombole , bakamupesha bena mudimu bende bua kupingana mu ditunga dia ditunga dia ditunga dia ditunga dia Bâla .\n",
      "2020-01-02 10:52:29,010 Example #2\n",
      "2020-01-02 10:52:29,011 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:52:29,011 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:52:29,011 \tHypothesis: Nunku Mose uvua mumanye ne : Yehowa uvua ne bua kujingulula ne : kavua mupangile bubi to .\n",
      "2020-01-02 10:52:29,011 Example #3\n",
      "2020-01-02 10:52:29,011 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:52:29,011 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:52:29,011 \tHypothesis: Ndi ne ditekemena dia se : badi bavuluka ne : Yehowa udi ufua .\n",
      "2020-01-02 10:52:29,011 Validation result (greedy) at epoch   2, step     6000: bleu:  11.32, loss: 62869.3203, ppl:  11.4212, duration: 29.4802s\n",
      "2020-01-02 10:52:40,777 Epoch   2 Step:     6100 Batch Loss:     2.715950 Tokens per Sec:    19268, Lr: 0.000300\n",
      "2020-01-02 10:52:52,471 Epoch   2 Step:     6200 Batch Loss:     2.395639 Tokens per Sec:    19452, Lr: 0.000300\n",
      "2020-01-02 10:53:04,257 Epoch   2 Step:     6300 Batch Loss:     2.491000 Tokens per Sec:    19531, Lr: 0.000300\n",
      "2020-01-02 10:53:16,135 Epoch   2 Step:     6400 Batch Loss:     2.853841 Tokens per Sec:    20057, Lr: 0.000300\n",
      "2020-01-02 10:53:27,784 Epoch   2 Step:     6500 Batch Loss:     2.201997 Tokens per Sec:    19061, Lr: 0.000300\n",
      "2020-01-02 10:53:36,387 Epoch   2: total training loss 9031.02\n",
      "2020-01-02 10:53:36,387 EPOCH 3\n",
      "2020-01-02 10:53:39,830 Epoch   3 Step:     6600 Batch Loss:     2.743274 Tokens per Sec:    16605, Lr: 0.000300\n",
      "2020-01-02 10:53:51,463 Epoch   3 Step:     6700 Batch Loss:     2.474597 Tokens per Sec:    19244, Lr: 0.000300\n",
      "2020-01-02 10:54:03,179 Epoch   3 Step:     6800 Batch Loss:     2.677408 Tokens per Sec:    19283, Lr: 0.000300\n",
      "2020-01-02 10:54:14,997 Epoch   3 Step:     6900 Batch Loss:     2.789459 Tokens per Sec:    20037, Lr: 0.000300\n",
      "2020-01-02 10:54:26,810 Epoch   3 Step:     7000 Batch Loss:     2.443068 Tokens per Sec:    19415, Lr: 0.000300\n",
      "2020-01-02 10:54:56,092 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:54:56,092 Saving new checkpoint.\n",
      "2020-01-02 10:54:56,358 Example #0\n",
      "2020-01-02 10:54:56,358 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:54:56,358 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:54:56,358 \tHypothesis: YEHOLE udi uyisha bena Kristo bonso bua kusambila ne : ‘ Tatumbishe Tatu wanyi , kadi Tatu wanyi , kadi udi ukuba . ’\n",
      "2020-01-02 10:54:56,358 Example #1\n",
      "2020-01-02 10:54:56,358 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:54:56,359 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:54:56,359 \tHypothesis: Bu muvua bana bende babenga , bakamulekela bua kupingana mu buloba , bakamulekela bua kupingana mu ditunga diabu .\n",
      "2020-01-02 10:54:56,359 Example #2\n",
      "2020-01-02 10:54:56,359 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:54:56,359 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:54:56,359 \tHypothesis: Nunku Mose uvua mumanye ne : Yehowa uvua mua kumvua malu avuaye muenze , nansha muvuaye kayi ubenga bua kushipa mpekatu , kadi udi ubenga bua kuteketa ku bubi .\n",
      "2020-01-02 10:54:56,359 Example #3\n",
      "2020-01-02 10:54:56,359 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:54:56,359 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:54:56,359 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa dikumbana dia dibishibua dia bafue .\n",
      "2020-01-02 10:54:56,360 Validation result (greedy) at epoch   3, step     7000: bleu:  12.77, loss: 60245.4219, ppl:  10.3174, duration: 29.5498s\n",
      "2020-01-02 10:55:08,185 Epoch   3 Step:     7100 Batch Loss:     2.527328 Tokens per Sec:    19635, Lr: 0.000300\n",
      "2020-01-02 10:55:20,019 Epoch   3 Step:     7200 Batch Loss:     2.461953 Tokens per Sec:    19576, Lr: 0.000300\n",
      "2020-01-02 10:55:31,806 Epoch   3 Step:     7300 Batch Loss:     2.469876 Tokens per Sec:    19482, Lr: 0.000300\n",
      "2020-01-02 10:55:43,555 Epoch   3 Step:     7400 Batch Loss:     2.736724 Tokens per Sec:    19480, Lr: 0.000300\n",
      "2020-01-02 10:55:55,479 Epoch   3 Step:     7500 Batch Loss:     2.558197 Tokens per Sec:    19711, Lr: 0.000300\n",
      "2020-01-02 10:56:07,334 Epoch   3 Step:     7600 Batch Loss:     2.338225 Tokens per Sec:    19684, Lr: 0.000300\n",
      "2020-01-02 10:56:19,099 Epoch   3 Step:     7700 Batch Loss:     2.423660 Tokens per Sec:    19344, Lr: 0.000300\n",
      "2020-01-02 10:56:30,872 Epoch   3 Step:     7800 Batch Loss:     2.550068 Tokens per Sec:    19289, Lr: 0.000300\n",
      "2020-01-02 10:56:42,674 Epoch   3 Step:     7900 Batch Loss:     2.443649 Tokens per Sec:    19782, Lr: 0.000300\n",
      "2020-01-02 10:56:54,483 Epoch   3 Step:     8000 Batch Loss:     2.099230 Tokens per Sec:    19508, Lr: 0.000300\n",
      "2020-01-02 10:57:23,735 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:57:23,735 Saving new checkpoint.\n",
      "2020-01-02 10:57:24,002 Example #0\n",
      "2020-01-02 10:57:24,003 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:57:24,003 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:57:24,003 \tHypothesis: YEHOWA udi mufile tshilejilu tshia bena Kristo bonso bua kusambila Tatu wende ne : ‘ Kanuena ne muaba wa kuendesha , kadi nudi nenu . ’\n",
      "2020-01-02 10:57:24,003 Example #1\n",
      "2020-01-02 10:57:24,003 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:57:24,003 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:57:24,003 \tHypothesis: Bu muvua bana babu bamutume ne bamutume , bakamulekela bua kupingana mu buloba bua kupingana mu ditunga diabu .\n",
      "2020-01-02 10:57:24,003 Example #2\n",
      "2020-01-02 10:57:24,003 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:57:24,003 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:57:24,003 \tHypothesis: Nunku , Mose uvua ne bua kumvua mishindu yende , Yehowa uvua ubenga bua kulumbuluisha mpekatu , kadi ki mmuntu udi ukengesha to .\n",
      "2020-01-02 10:57:24,003 Example #3\n",
      "2020-01-02 10:57:24,003 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:57:24,003 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:57:24,004 \tHypothesis: Ndi mushindike ne : badi bavuluka ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 10:57:24,004 Validation result (greedy) at epoch   3, step     8000: bleu:  14.21, loss: 58000.2734, ppl:   9.4579, duration: 29.5208s\n",
      "2020-01-02 10:57:35,840 Epoch   3 Step:     8100 Batch Loss:     2.578505 Tokens per Sec:    19581, Lr: 0.000300\n",
      "2020-01-02 10:57:47,603 Epoch   3 Step:     8200 Batch Loss:     2.170945 Tokens per Sec:    19574, Lr: 0.000300\n",
      "2020-01-02 10:57:59,439 Epoch   3 Step:     8300 Batch Loss:     2.492265 Tokens per Sec:    19493, Lr: 0.000300\n",
      "2020-01-02 10:58:11,228 Epoch   3 Step:     8400 Batch Loss:     2.684120 Tokens per Sec:    19564, Lr: 0.000300\n",
      "2020-01-02 10:58:23,000 Epoch   3 Step:     8500 Batch Loss:     2.216371 Tokens per Sec:    19608, Lr: 0.000300\n",
      "2020-01-02 10:58:34,858 Epoch   3 Step:     8600 Batch Loss:     2.020723 Tokens per Sec:    19280, Lr: 0.000300\n",
      "2020-01-02 10:58:46,617 Epoch   3 Step:     8700 Batch Loss:     2.666734 Tokens per Sec:    19556, Lr: 0.000300\n",
      "2020-01-02 10:58:58,393 Epoch   3 Step:     8800 Batch Loss:     2.312324 Tokens per Sec:    19189, Lr: 0.000300\n",
      "2020-01-02 10:59:10,050 Epoch   3 Step:     8900 Batch Loss:     2.605999 Tokens per Sec:    19310, Lr: 0.000300\n",
      "2020-01-02 10:59:21,742 Epoch   3 Step:     9000 Batch Loss:     2.378433 Tokens per Sec:    19194, Lr: 0.000300\n",
      "2020-01-02 10:59:51,024 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 10:59:51,024 Saving new checkpoint.\n",
      "2020-01-02 10:59:51,274 Example #0\n",
      "2020-01-02 10:59:51,274 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 10:59:51,274 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 10:59:51,274 \tHypothesis: YEHOWA wakafundila bena Kristo bonso bua kusambila Tatu wende ne : ‘ Kanuena nuenza mudimu , kadi nuikale muaba webe . ’\n",
      "2020-01-02 10:59:51,275 Example #1\n",
      "2020-01-02 10:59:51,275 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 10:59:51,275 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 10:59:51,275 \tHypothesis: Bu muvua bana babu babenga ne bamulekela , bakamulekela bua kupingana mu ditunga dia mbulamatadi .\n",
      "2020-01-02 10:59:51,275 Example #2\n",
      "2020-01-02 10:59:51,275 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 10:59:51,275 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 10:59:51,275 \tHypothesis: Nunku , Mose uvua mua kumvua mishindu yende , Yehowa wakamona ne : nansha muvuaye kayi ne bubi , udi utshina tshiji .\n",
      "2020-01-02 10:59:51,275 Example #3\n",
      "2020-01-02 10:59:51,275 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 10:59:51,275 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 10:59:51,276 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa mu dikumbana dia dibika dia bafue .\n",
      "2020-01-02 10:59:51,276 Validation result (greedy) at epoch   3, step     9000: bleu:  15.75, loss: 56034.4023, ppl:   8.7644, duration: 29.5338s\n",
      "2020-01-02 11:00:03,051 Epoch   3 Step:     9100 Batch Loss:     2.429463 Tokens per Sec:    19095, Lr: 0.000300\n",
      "2020-01-02 11:00:14,901 Epoch   3 Step:     9200 Batch Loss:     2.451626 Tokens per Sec:    19617, Lr: 0.000300\n",
      "2020-01-02 11:00:26,850 Epoch   3 Step:     9300 Batch Loss:     1.973258 Tokens per Sec:    19838, Lr: 0.000300\n",
      "2020-01-02 11:00:38,725 Epoch   3 Step:     9400 Batch Loss:     2.487381 Tokens per Sec:    19187, Lr: 0.000300\n",
      "2020-01-02 11:00:50,433 Epoch   3 Step:     9500 Batch Loss:     2.543725 Tokens per Sec:    19279, Lr: 0.000300\n",
      "2020-01-02 11:01:02,283 Epoch   3 Step:     9600 Batch Loss:     2.640833 Tokens per Sec:    19525, Lr: 0.000300\n",
      "2020-01-02 11:01:14,148 Epoch   3 Step:     9700 Batch Loss:     2.168578 Tokens per Sec:    19564, Lr: 0.000300\n",
      "2020-01-02 11:01:25,792 Epoch   3 Step:     9800 Batch Loss:     2.206785 Tokens per Sec:    19309, Lr: 0.000300\n",
      "2020-01-02 11:01:31,730 Epoch   3: total training loss 7891.85\n",
      "2020-01-02 11:01:31,730 EPOCH 4\n",
      "2020-01-02 11:01:37,960 Epoch   4 Step:     9900 Batch Loss:     2.155440 Tokens per Sec:    18287, Lr: 0.000300\n",
      "2020-01-02 11:01:49,621 Epoch   4 Step:    10000 Batch Loss:     2.506886 Tokens per Sec:    19540, Lr: 0.000300\n",
      "2020-01-02 11:02:18,879 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:02:18,880 Saving new checkpoint.\n",
      "2020-01-02 11:02:19,123 Example #0\n",
      "2020-01-02 11:02:19,123 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:02:19,123 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:02:19,123 \tHypothesis: YEZAMBI udi uteka bena Kristo bonso mu disambila ne : ‘ Tatuebe , kanuena nuenza mudimu , kadi nudi nuenza mudimu . ’\n",
      "2020-01-02 11:02:19,124 Example #1\n",
      "2020-01-02 11:02:19,124 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:02:19,124 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:02:19,124 \tHypothesis: Bu muvua bana babu babenga ne bamutume , bakamusaka bua kumushiya mu buloba bua kupingana kudiye .\n",
      "2020-01-02 11:02:19,124 Example #2\n",
      "2020-01-02 11:02:19,124 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:02:19,124 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:02:19,124 \tHypothesis: Nunku Mose uvua mua kumvua mishindu yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi utshina bubi , udi ulekela tshiji .\n",
      "2020-01-02 11:02:19,124 Example #3\n",
      "2020-01-02 11:02:19,124 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:02:19,124 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:02:19,124 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa dia dibishibua dia bafue .\n",
      "2020-01-02 11:02:19,125 Validation result (greedy) at epoch   4, step    10000: bleu:  16.10, loss: 54509.3945, ppl:   8.2616, duration: 29.5029s\n",
      "2020-01-02 11:02:30,857 Epoch   4 Step:    10100 Batch Loss:     2.533672 Tokens per Sec:    18806, Lr: 0.000300\n",
      "2020-01-02 11:02:42,566 Epoch   4 Step:    10200 Batch Loss:     2.244236 Tokens per Sec:    19410, Lr: 0.000300\n",
      "2020-01-02 11:02:54,350 Epoch   4 Step:    10300 Batch Loss:     2.362107 Tokens per Sec:    19687, Lr: 0.000300\n",
      "2020-01-02 11:03:06,111 Epoch   4 Step:    10400 Batch Loss:     2.362621 Tokens per Sec:    19569, Lr: 0.000300\n",
      "2020-01-02 11:03:17,796 Epoch   4 Step:    10500 Batch Loss:     2.358480 Tokens per Sec:    19307, Lr: 0.000300\n",
      "2020-01-02 11:03:29,671 Epoch   4 Step:    10600 Batch Loss:     1.874325 Tokens per Sec:    19860, Lr: 0.000300\n",
      "2020-01-02 11:03:41,420 Epoch   4 Step:    10700 Batch Loss:     2.311492 Tokens per Sec:    19271, Lr: 0.000300\n",
      "2020-01-02 11:03:53,210 Epoch   4 Step:    10800 Batch Loss:     2.349700 Tokens per Sec:    19742, Lr: 0.000300\n",
      "2020-01-02 11:04:05,026 Epoch   4 Step:    10900 Batch Loss:     2.122722 Tokens per Sec:    19082, Lr: 0.000300\n",
      "2020-01-02 11:04:16,776 Epoch   4 Step:    11000 Batch Loss:     2.292089 Tokens per Sec:    19568, Lr: 0.000300\n",
      "2020-01-02 11:04:46,079 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:04:46,079 Saving new checkpoint.\n",
      "2020-01-02 11:04:46,323 Example #0\n",
      "2020-01-02 11:04:46,323 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:04:46,323 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:04:46,323 \tHypothesis: YEHOLO wakafila tshilejilu tshia bena Kristo bonso bua kusambila Tatu ne : ‘ Kanulekelayi , kadi kanuena nuenza muaba uudi . ’\n",
      "2020-01-02 11:04:46,323 Example #1\n",
      "2020-01-02 11:04:46,323 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:04:46,323 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:04:46,323 \tHypothesis: Bu muvua bana babu babenga ne bamusungile , bakamusaka bua kupingana mu ditunga dia mbulamatadi bua kumutumbisha .\n",
      "2020-01-02 11:04:46,323 Example #2\n",
      "2020-01-02 11:04:46,324 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:04:46,324 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:04:46,324 \tHypothesis: Nunku , Mose uvua mumanye mishindu yende , Yehowa uvua mumueneke ne : nansha muvuaye kayi ne bubi , udi ukengesha tshiji .\n",
      "2020-01-02 11:04:46,324 Example #3\n",
      "2020-01-02 11:04:46,324 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:04:46,324 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:04:46,324 \tHypothesis: Ndi ngitaba ne : badi mu divuluka dia Yehowa dia dibika dia bafue .\n",
      "2020-01-02 11:04:46,324 Validation result (greedy) at epoch   4, step    11000: bleu:  17.35, loss: 53012.0664, ppl:   7.7960, duration: 29.5477s\n",
      "2020-01-02 11:04:57,989 Epoch   4 Step:    11100 Batch Loss:     2.153965 Tokens per Sec:    19051, Lr: 0.000300\n",
      "2020-01-02 11:05:09,737 Epoch   4 Step:    11200 Batch Loss:     2.307924 Tokens per Sec:    19250, Lr: 0.000300\n",
      "2020-01-02 11:05:21,393 Epoch   4 Step:    11300 Batch Loss:     2.384014 Tokens per Sec:    19180, Lr: 0.000300\n",
      "2020-01-02 11:05:33,198 Epoch   4 Step:    11400 Batch Loss:     1.921121 Tokens per Sec:    19521, Lr: 0.000300\n",
      "2020-01-02 11:05:45,002 Epoch   4 Step:    11500 Batch Loss:     1.802721 Tokens per Sec:    18877, Lr: 0.000300\n",
      "2020-01-02 11:05:56,742 Epoch   4 Step:    11600 Batch Loss:     2.404537 Tokens per Sec:    19394, Lr: 0.000300\n",
      "2020-01-02 11:06:08,518 Epoch   4 Step:    11700 Batch Loss:     2.380770 Tokens per Sec:    19436, Lr: 0.000300\n",
      "2020-01-02 11:06:20,310 Epoch   4 Step:    11800 Batch Loss:     2.376062 Tokens per Sec:    19499, Lr: 0.000300\n",
      "2020-01-02 11:06:32,142 Epoch   4 Step:    11900 Batch Loss:     2.005730 Tokens per Sec:    19232, Lr: 0.000300\n",
      "2020-01-02 11:06:44,049 Epoch   4 Step:    12000 Batch Loss:     2.240419 Tokens per Sec:    19838, Lr: 0.000300\n",
      "2020-01-02 11:07:13,276 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:07:13,276 Saving new checkpoint.\n",
      "2020-01-02 11:07:13,562 Example #0\n",
      "2020-01-02 11:07:13,562 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:07:13,562 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:07:13,562 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe bua kusambila Tatu wende ne : ‘ Kanulekedi , kadi nuenze bu muaba uudi . ’\n",
      "2020-01-02 11:07:13,562 Example #1\n",
      "2020-01-02 11:07:13,562 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:07:13,562 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:07:13,562 \tHypothesis: Bu muvua bana babu babenga ne bamushipe , bakamusaka bua kushiya ditunga dia mbulamatadi .\n",
      "2020-01-02 11:07:13,563 Example #2\n",
      "2020-01-02 11:07:13,563 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:07:13,563 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:07:13,563 \tHypothesis: Nunku Mose uvua mua kumvua malu avuaye mua kumvua , Yehowa wakaleja patoke ne : nansha yeye kayi ulua ku bubi , udi ukenga .\n",
      "2020-01-02 11:07:13,563 Example #3\n",
      "2020-01-02 11:07:13,563 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:07:13,563 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:07:13,563 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa dia dibishibua dia bafue .\n",
      "2020-01-02 11:07:13,563 Validation result (greedy) at epoch   4, step    12000: bleu:  18.26, loss: 51737.6211, ppl:   7.4205, duration: 29.5139s\n",
      "2020-01-02 11:07:25,383 Epoch   4 Step:    12100 Batch Loss:     2.045502 Tokens per Sec:    19562, Lr: 0.000300\n",
      "2020-01-02 11:07:37,218 Epoch   4 Step:    12200 Batch Loss:     1.955127 Tokens per Sec:    19358, Lr: 0.000300\n",
      "2020-01-02 11:07:49,007 Epoch   4 Step:    12300 Batch Loss:     2.211698 Tokens per Sec:    19511, Lr: 0.000300\n",
      "2020-01-02 11:08:00,844 Epoch   4 Step:    12400 Batch Loss:     2.284011 Tokens per Sec:    19391, Lr: 0.000300\n",
      "2020-01-02 11:08:12,530 Epoch   4 Step:    12500 Batch Loss:     2.268448 Tokens per Sec:    19476, Lr: 0.000300\n",
      "2020-01-02 11:08:24,360 Epoch   4 Step:    12600 Batch Loss:     2.213573 Tokens per Sec:    19572, Lr: 0.000300\n",
      "2020-01-02 11:08:36,071 Epoch   4 Step:    12700 Batch Loss:     2.164679 Tokens per Sec:    19269, Lr: 0.000300\n",
      "2020-01-02 11:08:47,744 Epoch   4 Step:    12800 Batch Loss:     2.346673 Tokens per Sec:    19400, Lr: 0.000300\n",
      "2020-01-02 11:08:59,565 Epoch   4 Step:    12900 Batch Loss:     2.278499 Tokens per Sec:    19465, Lr: 0.000300\n",
      "2020-01-02 11:09:11,409 Epoch   4 Step:    13000 Batch Loss:     2.114444 Tokens per Sec:    19721, Lr: 0.000300\n",
      "2020-01-02 11:09:40,739 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:09:40,739 Saving new checkpoint.\n",
      "2020-01-02 11:09:40,974 Example #0\n",
      "2020-01-02 11:09:40,975 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:09:40,975 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:09:40,975 \tHypothesis: YEZU wakafila tshilejilu tshia bena Kristo bonso bua kusambila Tatu wende ne : ‘ Kanulekedi muaba wanyi , kadi nenuikale muaba uudi . ’\n",
      "2020-01-02 11:09:40,975 Example #1\n",
      "2020-01-02 11:09:40,975 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:09:40,976 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:09:40,976 \tHypothesis: Bu muvua bana babu babenga ne bamushipe , bakamuambila bua kushiya buloba bua kupingana mu ditunga diabu .\n",
      "2020-01-02 11:09:40,976 Example #2\n",
      "2020-01-02 11:09:40,976 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:09:40,976 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:09:40,976 \tHypothesis: Nunku Mose uvua mua kumvua bimpe mishindu yende , Yehowa wakadileja patoke ne : nansha muvuaye kayi uya ku mpekatu , udi uya ku njiwu .\n",
      "2020-01-02 11:09:40,976 Example #3\n",
      "2020-01-02 11:09:40,976 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:09:40,976 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:09:40,976 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa diindila dia dibika dia bafue .\n",
      "2020-01-02 11:09:40,977 Validation result (greedy) at epoch   4, step    13000: bleu:  18.31, loss: 50633.3828, ppl:   7.1098, duration: 29.5673s\n",
      "2020-01-02 11:09:52,745 Epoch   4 Step:    13100 Batch Loss:     2.114905 Tokens per Sec:    19165, Lr: 0.000300\n",
      "2020-01-02 11:09:57,959 Epoch   4: total training loss 7284.45\n",
      "2020-01-02 11:09:57,959 EPOCH 5\n",
      "2020-01-02 11:10:04,929 Epoch   5 Step:    13200 Batch Loss:     2.241098 Tokens per Sec:    18233, Lr: 0.000300\n",
      "2020-01-02 11:10:16,707 Epoch   5 Step:    13300 Batch Loss:     1.801580 Tokens per Sec:    19608, Lr: 0.000300\n",
      "2020-01-02 11:10:28,406 Epoch   5 Step:    13400 Batch Loss:     2.243426 Tokens per Sec:    19357, Lr: 0.000300\n",
      "2020-01-02 11:10:40,248 Epoch   5 Step:    13500 Batch Loss:     2.112034 Tokens per Sec:    19746, Lr: 0.000300\n",
      "2020-01-02 11:10:52,065 Epoch   5 Step:    13600 Batch Loss:     2.214568 Tokens per Sec:    19195, Lr: 0.000300\n",
      "2020-01-02 11:11:03,913 Epoch   5 Step:    13700 Batch Loss:     1.733306 Tokens per Sec:    19382, Lr: 0.000300\n",
      "2020-01-02 11:11:15,546 Epoch   5 Step:    13800 Batch Loss:     2.410551 Tokens per Sec:    19457, Lr: 0.000300\n",
      "2020-01-02 11:11:27,289 Epoch   5 Step:    13900 Batch Loss:     1.919518 Tokens per Sec:    19324, Lr: 0.000300\n",
      "2020-01-02 11:11:39,122 Epoch   5 Step:    14000 Batch Loss:     1.743710 Tokens per Sec:    19810, Lr: 0.000300\n",
      "2020-01-02 11:12:08,395 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:12:08,395 Saving new checkpoint.\n",
      "2020-01-02 11:12:08,648 Example #0\n",
      "2020-01-02 11:12:08,649 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:12:08,649 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:12:08,649 \tHypothesis: YESUKU wakafundila bena Kristo bonso bua kusambila Tatu wende ne : ‘ Kanulekedi muaba wanyi , kadi nudi nuya muaba webe . ’\n",
      "2020-01-02 11:12:08,649 Example #1\n",
      "2020-01-02 11:12:08,649 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:12:08,649 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:12:08,649 \tHypothesis: Bu muvua bana babu babenga ne bamushikamemu , bakamuambila bua kushiya buloba bulaya .\n",
      "2020-01-02 11:12:08,650 Example #2\n",
      "2020-01-02 11:12:08,650 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:12:08,650 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:12:08,650 \tHypothesis: Nunku Mose uvua mua kumvua bimpe njila yende , Yehowa wakadileja patoke ne : nansha muvuaye kayi ne bua kulumbulula bubi , udi ulekela tshiji .\n",
      "2020-01-02 11:12:08,650 Example #3\n",
      "2020-01-02 11:12:08,650 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:12:08,650 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:12:08,650 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa diindile dia dibishibua dia bafue .\n",
      "2020-01-02 11:12:08,650 Validation result (greedy) at epoch   5, step    14000: bleu:  18.94, loss: 49630.2148, ppl:   6.8388, duration: 29.5284s\n",
      "2020-01-02 11:12:20,247 Epoch   5 Step:    14100 Batch Loss:     2.385367 Tokens per Sec:    19118, Lr: 0.000300\n",
      "2020-01-02 11:12:32,037 Epoch   5 Step:    14200 Batch Loss:     2.213422 Tokens per Sec:    19392, Lr: 0.000300\n",
      "2020-01-02 11:12:43,715 Epoch   5 Step:    14300 Batch Loss:     1.945031 Tokens per Sec:    19498, Lr: 0.000300\n",
      "2020-01-02 11:12:55,483 Epoch   5 Step:    14400 Batch Loss:     1.898693 Tokens per Sec:    19606, Lr: 0.000300\n",
      "2020-01-02 11:13:07,379 Epoch   5 Step:    14500 Batch Loss:     2.121178 Tokens per Sec:    19484, Lr: 0.000300\n",
      "2020-01-02 11:13:19,195 Epoch   5 Step:    14600 Batch Loss:     1.981629 Tokens per Sec:    19380, Lr: 0.000300\n",
      "2020-01-02 11:13:31,041 Epoch   5 Step:    14700 Batch Loss:     2.223627 Tokens per Sec:    19488, Lr: 0.000300\n",
      "2020-01-02 11:13:42,714 Epoch   5 Step:    14800 Batch Loss:     2.287854 Tokens per Sec:    19180, Lr: 0.000300\n",
      "2020-01-02 11:13:54,410 Epoch   5 Step:    14900 Batch Loss:     1.967321 Tokens per Sec:    19623, Lr: 0.000300\n",
      "2020-01-02 11:14:06,147 Epoch   5 Step:    15000 Batch Loss:     1.719248 Tokens per Sec:    19824, Lr: 0.000300\n",
      "2020-01-02 11:14:35,420 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:14:35,420 Saving new checkpoint.\n",
      "2020-01-02 11:14:35,696 Example #0\n",
      "2020-01-02 11:14:35,696 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:14:35,697 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:14:35,697 \tHypothesis: YESOLE mmutushile tshilejilu tshia bena Kristo bonso patudi tusambila Tatu wende ne : ‘ Kanulekelayi , kadi kanuena nuenza muaba webe . ’\n",
      "2020-01-02 11:14:35,697 Example #1\n",
      "2020-01-02 11:14:35,697 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:14:35,697 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:14:35,697 \tHypothesis: Bu muvua bana babu babenga ne bamushiminyina , bakamulekela bua kushiya buloba buvuabu bamupeshe .\n",
      "2020-01-02 11:14:35,697 Example #2\n",
      "2020-01-02 11:14:35,697 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:14:35,697 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:14:35,697 \tHypothesis: Nunku , Mose uvua mua kumvua malu avuaye muenze , ke bualu kayi Yehowa kavua ujinga bua kuenza mpekatu , kadi udi ulekela tshiji .\n",
      "2020-01-02 11:14:35,697 Example #3\n",
      "2020-01-02 11:14:35,698 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:14:35,698 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:14:35,698 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa diindila dia dibika dia bafue .\n",
      "2020-01-02 11:14:35,698 Validation result (greedy) at epoch   5, step    15000: bleu:  19.52, loss: 48967.0352, ppl:   6.6653, duration: 29.5508s\n",
      "2020-01-02 11:14:47,437 Epoch   5 Step:    15100 Batch Loss:     2.234443 Tokens per Sec:    19228, Lr: 0.000300\n",
      "2020-01-02 11:14:59,250 Epoch   5 Step:    15200 Batch Loss:     1.692213 Tokens per Sec:    19589, Lr: 0.000300\n",
      "2020-01-02 11:15:11,030 Epoch   5 Step:    15300 Batch Loss:     1.932638 Tokens per Sec:    19740, Lr: 0.000300\n",
      "2020-01-02 11:15:22,905 Epoch   5 Step:    15400 Batch Loss:     1.831988 Tokens per Sec:    19334, Lr: 0.000300\n",
      "2020-01-02 11:15:34,714 Epoch   5 Step:    15500 Batch Loss:     1.876834 Tokens per Sec:    19420, Lr: 0.000300\n",
      "2020-01-02 11:15:46,485 Epoch   5 Step:    15600 Batch Loss:     1.890089 Tokens per Sec:    19286, Lr: 0.000300\n",
      "2020-01-02 11:15:58,376 Epoch   5 Step:    15700 Batch Loss:     1.628464 Tokens per Sec:    19526, Lr: 0.000300\n",
      "2020-01-02 11:16:10,039 Epoch   5 Step:    15800 Batch Loss:     2.114320 Tokens per Sec:    18793, Lr: 0.000300\n",
      "2020-01-02 11:16:21,770 Epoch   5 Step:    15900 Batch Loss:     2.216446 Tokens per Sec:    19406, Lr: 0.000300\n",
      "2020-01-02 11:16:33,609 Epoch   5 Step:    16000 Batch Loss:     2.489284 Tokens per Sec:    19823, Lr: 0.000300\n",
      "2020-01-02 11:17:02,903 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:17:02,903 Saving new checkpoint.\n",
      "2020-01-02 11:17:03,147 Example #0\n",
      "2020-01-02 11:17:03,147 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:17:03,147 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:17:03,147 \tHypothesis: YEZU wakafila tshilejilu tshia bena Kristo bonso bua kusambila Tatu wende ne : ‘ Kanuenji malu anyi , kadi nuenze bu muaba . ’\n",
      "2020-01-02 11:17:03,147 Example #1\n",
      "2020-01-02 11:17:03,148 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:17:03,148 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:17:03,148 \tHypothesis: Bu muvua bana babu babenga ne bamushipe , bakamuambila bua kushiya buloba abu .\n",
      "2020-01-02 11:17:03,148 Example #2\n",
      "2020-01-02 11:17:03,148 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:17:03,148 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:17:03,148 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakafuanyikija ne : nansha muvuaye kayi uya ku mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:17:03,148 Example #3\n",
      "2020-01-02 11:17:03,148 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:17:03,148 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:17:03,148 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa diindila dia dibika dia bafue .\n",
      "2020-01-02 11:17:03,149 Validation result (greedy) at epoch   5, step    16000: bleu:  19.83, loss: 48093.6836, ppl:   6.4436, duration: 29.5391s\n",
      "2020-01-02 11:17:15,019 Epoch   5 Step:    16100 Batch Loss:     2.015493 Tokens per Sec:    19349, Lr: 0.000300\n",
      "2020-01-02 11:17:26,943 Epoch   5 Step:    16200 Batch Loss:     2.001325 Tokens per Sec:    19797, Lr: 0.000300\n",
      "2020-01-02 11:17:38,645 Epoch   5 Step:    16300 Batch Loss:     1.966933 Tokens per Sec:    19334, Lr: 0.000300\n",
      "2020-01-02 11:17:50,351 Epoch   5 Step:    16400 Batch Loss:     2.307684 Tokens per Sec:    19382, Lr: 0.000300\n",
      "2020-01-02 11:17:53,948 Epoch   5: total training loss 6831.91\n",
      "2020-01-02 11:17:53,948 EPOCH 6\n",
      "2020-01-02 11:18:02,460 Epoch   6 Step:    16500 Batch Loss:     2.188208 Tokens per Sec:    18636, Lr: 0.000300\n",
      "2020-01-02 11:18:14,113 Epoch   6 Step:    16600 Batch Loss:     1.937109 Tokens per Sec:    19350, Lr: 0.000300\n",
      "2020-01-02 11:18:25,890 Epoch   6 Step:    16700 Batch Loss:     2.181293 Tokens per Sec:    19508, Lr: 0.000300\n",
      "2020-01-02 11:18:37,563 Epoch   6 Step:    16800 Batch Loss:     2.087052 Tokens per Sec:    18864, Lr: 0.000300\n",
      "2020-01-02 11:18:49,435 Epoch   6 Step:    16900 Batch Loss:     1.868238 Tokens per Sec:    19710, Lr: 0.000300\n",
      "2020-01-02 11:19:01,271 Epoch   6 Step:    17000 Batch Loss:     2.092833 Tokens per Sec:    19627, Lr: 0.000300\n",
      "2020-01-02 11:19:30,535 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:19:30,536 Saving new checkpoint.\n",
      "2020-01-02 11:19:30,795 Example #0\n",
      "2020-01-02 11:19:30,796 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:19:30,796 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:19:30,796 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe tshia kusambila Tatu wende ne : ‘ Kanulekedi muaba wanyi , kadi wewe udi uya muaba . ’\n",
      "2020-01-02 11:19:30,796 Example #1\n",
      "2020-01-02 11:19:30,796 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:19:30,796 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:19:30,796 \tHypothesis: Bu muvua bana babu babenga ne bamukine , bakamuambila bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:19:30,796 Example #2\n",
      "2020-01-02 11:19:30,796 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:19:30,797 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:19:30,797 \tHypothesis: Nunku , Mose uvua ne bua kumvua mishindu yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 11:19:30,797 Example #3\n",
      "2020-01-02 11:19:30,797 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:19:30,797 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:19:30,797 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa dia dibika dia bafue .\n",
      "2020-01-02 11:19:30,797 Validation result (greedy) at epoch   6, step    17000: bleu:  19.95, loss: 47514.9023, ppl:   6.3007, duration: 29.5256s\n",
      "2020-01-02 11:19:42,662 Epoch   6 Step:    17100 Batch Loss:     2.641386 Tokens per Sec:    19614, Lr: 0.000300\n",
      "2020-01-02 11:19:54,538 Epoch   6 Step:    17200 Batch Loss:     2.039268 Tokens per Sec:    19241, Lr: 0.000300\n",
      "2020-01-02 11:20:06,379 Epoch   6 Step:    17300 Batch Loss:     1.997550 Tokens per Sec:    19430, Lr: 0.000300\n",
      "2020-01-02 11:20:18,030 Epoch   6 Step:    17400 Batch Loss:     1.986365 Tokens per Sec:    19148, Lr: 0.000300\n",
      "2020-01-02 11:20:29,854 Epoch   6 Step:    17500 Batch Loss:     1.946229 Tokens per Sec:    19276, Lr: 0.000300\n",
      "2020-01-02 11:20:41,711 Epoch   6 Step:    17600 Batch Loss:     1.488998 Tokens per Sec:    19529, Lr: 0.000300\n",
      "2020-01-02 11:20:53,418 Epoch   6 Step:    17700 Batch Loss:     1.870464 Tokens per Sec:    19262, Lr: 0.000300\n",
      "2020-01-02 11:21:05,278 Epoch   6 Step:    17800 Batch Loss:     1.650072 Tokens per Sec:    19556, Lr: 0.000300\n",
      "2020-01-02 11:21:17,168 Epoch   6 Step:    17900 Batch Loss:     1.767884 Tokens per Sec:    19655, Lr: 0.000300\n",
      "2020-01-02 11:21:28,955 Epoch   6 Step:    18000 Batch Loss:     2.209151 Tokens per Sec:    19531, Lr: 0.000300\n",
      "2020-01-02 11:21:58,286 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:21:58,286 Saving new checkpoint.\n",
      "2020-01-02 11:21:58,534 Example #0\n",
      "2020-01-02 11:21:58,534 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:21:58,534 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:21:58,534 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso bua kusambila Tatu wende ne : ‘ Kanulekedi , kadi nuikale muaba undi . ’\n",
      "2020-01-02 11:21:58,534 Example #1\n",
      "2020-01-02 11:21:58,535 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:21:58,535 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:21:58,535 \tHypothesis: Bu muvua bana babu babenga ne bamushipa , bakamusaka bua kumbuka mu ditunga adi .\n",
      "2020-01-02 11:21:58,535 Example #2\n",
      "2020-01-02 11:21:58,535 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:21:58,535 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:21:58,535 \tHypothesis: Nunku Mose uvua mua kumvua bimpe njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi uya kumpala kua mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:21:58,535 Example #3\n",
      "2020-01-02 11:21:58,535 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:21:58,535 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:21:58,536 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa diindila dia dibika dia bafue .\n",
      "2020-01-02 11:21:58,536 Validation result (greedy) at epoch   6, step    18000: bleu:  20.36, loss: 47023.1133, ppl:   6.1818, duration: 29.5803s\n",
      "2020-01-02 11:22:10,428 Epoch   6 Step:    18100 Batch Loss:     2.388353 Tokens per Sec:    19914, Lr: 0.000300\n",
      "2020-01-02 11:22:22,235 Epoch   6 Step:    18200 Batch Loss:     1.773120 Tokens per Sec:    19163, Lr: 0.000300\n",
      "2020-01-02 11:22:34,060 Epoch   6 Step:    18300 Batch Loss:     1.849570 Tokens per Sec:    19167, Lr: 0.000300\n",
      "2020-01-02 11:22:45,948 Epoch   6 Step:    18400 Batch Loss:     1.606249 Tokens per Sec:    19416, Lr: 0.000300\n",
      "2020-01-02 11:22:57,779 Epoch   6 Step:    18500 Batch Loss:     1.684443 Tokens per Sec:    19305, Lr: 0.000300\n",
      "2020-01-02 11:23:09,614 Epoch   6 Step:    18600 Batch Loss:     1.995659 Tokens per Sec:    19284, Lr: 0.000300\n",
      "2020-01-02 11:23:21,320 Epoch   6 Step:    18700 Batch Loss:     2.200078 Tokens per Sec:    19388, Lr: 0.000300\n",
      "2020-01-02 11:23:32,991 Epoch   6 Step:    18800 Batch Loss:     2.065585 Tokens per Sec:    19289, Lr: 0.000300\n",
      "2020-01-02 11:23:44,626 Epoch   6 Step:    18900 Batch Loss:     1.964380 Tokens per Sec:    18788, Lr: 0.000300\n",
      "2020-01-02 11:23:56,429 Epoch   6 Step:    19000 Batch Loss:     2.145277 Tokens per Sec:    19223, Lr: 0.000300\n",
      "2020-01-02 11:24:25,748 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:24:25,748 Saving new checkpoint.\n",
      "2020-01-02 11:24:25,992 Example #0\n",
      "2020-01-02 11:24:25,993 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:24:25,993 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:24:25,993 \tHypothesis: YEZU wakafila tshilejilu tshia bena Kristo bonso pa kusambila Tatu wende ne : ‘ Kanulekedi , kadi wewe udi ulua muaba . ’\n",
      "2020-01-02 11:24:25,993 Example #1\n",
      "2020-01-02 11:24:25,993 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:24:25,993 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:24:25,993 \tHypothesis: Bu muvua bana babu babenga ne bamukine , bakamuenzeja bua kumbuka mu ditunga adi .\n",
      "2020-01-02 11:24:25,993 Example #2\n",
      "2020-01-02 11:24:25,993 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:24:25,993 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:24:25,993 \tHypothesis: Nunku Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , udi ukenga .\n",
      "2020-01-02 11:24:25,993 Example #3\n",
      "2020-01-02 11:24:25,993 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:24:25,993 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:24:25,993 \tHypothesis: Ndi mujadike ne : badi bavuluka ne : Yehowa udi uindila dibika dia bafue .\n",
      "2020-01-02 11:24:25,994 Validation result (greedy) at epoch   6, step    19000: bleu:  21.08, loss: 46357.8203, ppl:   6.0245, duration: 29.5642s\n",
      "2020-01-02 11:24:37,936 Epoch   6 Step:    19100 Batch Loss:     1.474120 Tokens per Sec:    19222, Lr: 0.000300\n",
      "2020-01-02 11:24:49,804 Epoch   6 Step:    19200 Batch Loss:     1.961314 Tokens per Sec:    20036, Lr: 0.000300\n",
      "2020-01-02 11:25:01,580 Epoch   6 Step:    19300 Batch Loss:     2.005141 Tokens per Sec:    19194, Lr: 0.000300\n",
      "2020-01-02 11:25:13,338 Epoch   6 Step:    19400 Batch Loss:     2.031661 Tokens per Sec:    19323, Lr: 0.000300\n",
      "2020-01-02 11:25:25,212 Epoch   6 Step:    19500 Batch Loss:     1.674572 Tokens per Sec:    19605, Lr: 0.000300\n",
      "2020-01-02 11:25:37,066 Epoch   6 Step:    19600 Batch Loss:     1.742440 Tokens per Sec:    19432, Lr: 0.000300\n",
      "2020-01-02 11:25:48,930 Epoch   6 Step:    19700 Batch Loss:     2.088961 Tokens per Sec:    19748, Lr: 0.000300\n",
      "2020-01-02 11:25:50,908 Epoch   6: total training loss 6532.87\n",
      "2020-01-02 11:25:50,908 EPOCH 7\n",
      "2020-01-02 11:26:01,179 Epoch   7 Step:    19800 Batch Loss:     2.008860 Tokens per Sec:    19116, Lr: 0.000300\n",
      "2020-01-02 11:26:12,921 Epoch   7 Step:    19900 Batch Loss:     1.952571 Tokens per Sec:    19297, Lr: 0.000300\n",
      "2020-01-02 11:26:24,853 Epoch   7 Step:    20000 Batch Loss:     1.884014 Tokens per Sec:    19678, Lr: 0.000300\n",
      "2020-01-02 11:26:54,216 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:26:54,216 Saving new checkpoint.\n",
      "2020-01-02 11:26:54,461 Example #0\n",
      "2020-01-02 11:26:54,462 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:26:54,462 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:26:54,462 \tHypothesis: YEZU wakafila tshilejilu tshia bena Kristo bonso pavuaye usambila Tatuende ne : ‘ Kanulekedi tshintu tshianyi , kadi nengenze bu tshianyi . ’\n",
      "2020-01-02 11:26:54,462 Example #1\n",
      "2020-01-02 11:26:54,462 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:26:54,462 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:26:54,462 \tHypothesis: Bu muvua bana babu babenga ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:26:54,462 Example #2\n",
      "2020-01-02 11:26:54,462 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:26:54,462 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:26:54,462 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi uya kumpala , udi ukuata tshiji .\n",
      "2020-01-02 11:26:54,462 Example #3\n",
      "2020-01-02 11:26:54,463 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:26:54,463 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:26:54,463 \tHypothesis: Ndi mushindike ne : badi bavuluka Yehowa bua dibika dia bafue .\n",
      "2020-01-02 11:26:54,463 Validation result (greedy) at epoch   7, step    20000: bleu:  21.19, loss: 45906.8906, ppl:   5.9202, duration: 29.6096s\n",
      "2020-01-02 11:27:06,210 Epoch   7 Step:    20100 Batch Loss:     1.864169 Tokens per Sec:    19392, Lr: 0.000300\n",
      "2020-01-02 11:27:17,997 Epoch   7 Step:    20200 Batch Loss:     1.893431 Tokens per Sec:    19360, Lr: 0.000300\n",
      "2020-01-02 11:27:29,771 Epoch   7 Step:    20300 Batch Loss:     1.880956 Tokens per Sec:    19355, Lr: 0.000300\n",
      "2020-01-02 11:27:41,464 Epoch   7 Step:    20400 Batch Loss:     1.973069 Tokens per Sec:    19536, Lr: 0.000300\n",
      "2020-01-02 11:27:53,186 Epoch   7 Step:    20500 Batch Loss:     1.750376 Tokens per Sec:    19409, Lr: 0.000300\n",
      "2020-01-02 11:28:04,948 Epoch   7 Step:    20600 Batch Loss:     1.291761 Tokens per Sec:    19267, Lr: 0.000300\n",
      "2020-01-02 11:28:16,664 Epoch   7 Step:    20700 Batch Loss:     2.004114 Tokens per Sec:    19296, Lr: 0.000300\n",
      "2020-01-02 11:28:28,364 Epoch   7 Step:    20800 Batch Loss:     2.137767 Tokens per Sec:    19061, Lr: 0.000300\n",
      "2020-01-02 11:28:40,251 Epoch   7 Step:    20900 Batch Loss:     2.365898 Tokens per Sec:    19659, Lr: 0.000300\n",
      "2020-01-02 11:28:51,998 Epoch   7 Step:    21000 Batch Loss:     2.028100 Tokens per Sec:    19360, Lr: 0.000300\n",
      "2020-01-02 11:29:21,249 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:29:21,249 Saving new checkpoint.\n",
      "2020-01-02 11:29:21,512 Example #0\n",
      "2020-01-02 11:29:21,512 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:29:21,512 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:29:21,512 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso patudi tusambila Tatu wende ne : ‘ Kanulekedi , kadi nuenze bu muaba . ’\n",
      "2020-01-02 11:29:21,512 Example #1\n",
      "2020-01-02 11:29:21,513 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:29:21,513 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:29:21,513 \tHypothesis: Bu muvua bana babu babenga ne bamupambuisha , bakamusaka bua kumbuka mu ditunga adi divua dienda dienda dikola .\n",
      "2020-01-02 11:29:21,513 Example #2\n",
      "2020-01-02 11:29:21,513 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:29:21,513 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:29:21,513 \tHypothesis: Nunku Mose uvua ne bua kumvua njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi utshina bubi , udi ukuata tshiji .\n",
      "2020-01-02 11:29:21,513 Example #3\n",
      "2020-01-02 11:29:21,513 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:29:21,513 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:29:21,513 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa diindila dia dibika dia bafue .\n",
      "2020-01-02 11:29:21,514 Validation result (greedy) at epoch   7, step    21000: bleu:  21.89, loss: 45281.0781, ppl:   5.7784, duration: 29.5146s\n",
      "2020-01-02 11:29:33,311 Epoch   7 Step:    21100 Batch Loss:     1.938720 Tokens per Sec:    19126, Lr: 0.000300\n",
      "2020-01-02 11:29:45,033 Epoch   7 Step:    21200 Batch Loss:     1.900849 Tokens per Sec:    19570, Lr: 0.000300\n",
      "2020-01-02 11:29:56,797 Epoch   7 Step:    21300 Batch Loss:     1.684049 Tokens per Sec:    19782, Lr: 0.000300\n",
      "2020-01-02 11:30:08,603 Epoch   7 Step:    21400 Batch Loss:     1.883672 Tokens per Sec:    19213, Lr: 0.000300\n",
      "2020-01-02 11:30:20,226 Epoch   7 Step:    21500 Batch Loss:     1.350496 Tokens per Sec:    19177, Lr: 0.000300\n",
      "2020-01-02 11:30:31,846 Epoch   7 Step:    21600 Batch Loss:     2.030804 Tokens per Sec:    18942, Lr: 0.000300\n",
      "2020-01-02 11:30:43,700 Epoch   7 Step:    21700 Batch Loss:     2.194757 Tokens per Sec:    19609, Lr: 0.000300\n",
      "2020-01-02 11:30:55,356 Epoch   7 Step:    21800 Batch Loss:     1.721157 Tokens per Sec:    18919, Lr: 0.000300\n",
      "2020-01-02 11:31:07,151 Epoch   7 Step:    21900 Batch Loss:     1.974037 Tokens per Sec:    19459, Lr: 0.000300\n",
      "2020-01-02 11:31:18,937 Epoch   7 Step:    22000 Batch Loss:     2.005310 Tokens per Sec:    19857, Lr: 0.000300\n",
      "2020-01-02 11:31:48,301 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:31:48,301 Saving new checkpoint.\n",
      "2020-01-02 11:31:48,548 Example #0\n",
      "2020-01-02 11:31:48,548 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:31:48,548 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:31:48,548 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatu wende ne : ‘ Kanulekedi , kadi kanuena nuenza muaba . ’\n",
      "2020-01-02 11:31:48,548 Example #1\n",
      "2020-01-02 11:31:48,548 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:31:48,548 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:31:48,549 \tHypothesis: Bu muvua bana babu babenga ne bamukengesha , bakamuenzeja bua kumbuka mu buloba buvuabu bamupeshe .\n",
      "2020-01-02 11:31:48,549 Example #2\n",
      "2020-01-02 11:31:48,549 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:31:48,549 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:31:48,549 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi utshina bubi , udi ukenga .\n",
      "2020-01-02 11:31:48,549 Example #3\n",
      "2020-01-02 11:31:48,549 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:31:48,549 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:31:48,549 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa dibika dia bafue .\n",
      "2020-01-02 11:31:48,549 Validation result (greedy) at epoch   7, step    22000: bleu:  21.80, loss: 44715.1953, ppl:   5.6531, duration: 29.6117s\n",
      "2020-01-02 11:32:00,432 Epoch   7 Step:    22100 Batch Loss:     1.712698 Tokens per Sec:    19400, Lr: 0.000300\n",
      "2020-01-02 11:32:12,237 Epoch   7 Step:    22200 Batch Loss:     1.827039 Tokens per Sec:    19860, Lr: 0.000300\n",
      "2020-01-02 11:32:23,848 Epoch   7 Step:    22300 Batch Loss:     2.495296 Tokens per Sec:    19492, Lr: 0.000300\n",
      "2020-01-02 11:32:35,671 Epoch   7 Step:    22400 Batch Loss:     1.886254 Tokens per Sec:    18972, Lr: 0.000300\n",
      "2020-01-02 11:32:47,449 Epoch   7 Step:    22500 Batch Loss:     2.106150 Tokens per Sec:    19736, Lr: 0.000300\n",
      "2020-01-02 11:32:59,260 Epoch   7 Step:    22600 Batch Loss:     1.765030 Tokens per Sec:    19539, Lr: 0.000300\n",
      "2020-01-02 11:33:10,927 Epoch   7 Step:    22700 Batch Loss:     1.831388 Tokens per Sec:    19130, Lr: 0.000300\n",
      "2020-01-02 11:33:22,821 Epoch   7 Step:    22800 Batch Loss:     1.921616 Tokens per Sec:    19653, Lr: 0.000300\n",
      "2020-01-02 11:33:34,677 Epoch   7 Step:    22900 Batch Loss:     2.036633 Tokens per Sec:    19104, Lr: 0.000300\n",
      "2020-01-02 11:33:46,433 Epoch   7 Step:    23000 Batch Loss:     2.398711 Tokens per Sec:    19507, Lr: 0.000300\n",
      "2020-01-02 11:34:15,748 Example #0\n",
      "2020-01-02 11:34:15,749 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:34:15,749 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:34:15,749 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatu wende ne : ‘ Kanulekedi , kadi nuikale ne muaba . ’\n",
      "2020-01-02 11:34:15,749 Example #1\n",
      "2020-01-02 11:34:15,749 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:34:15,749 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:34:15,749 \tHypothesis: Bu muvua bana babu babenge ne bamukine , bakamusaka bua kumbuka mu ditunga divua dienda dienda dienda dienda dikola .\n",
      "2020-01-02 11:34:15,749 Example #2\n",
      "2020-01-02 11:34:15,749 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:34:15,749 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:34:15,749 \tHypothesis: Ke bualu kayi Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:34:15,749 Example #3\n",
      "2020-01-02 11:34:15,749 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:34:15,749 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:34:15,749 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa ku lufu .\n",
      "2020-01-02 11:34:15,750 Validation result (greedy) at epoch   7, step    23000: bleu:  21.69, loss: 45031.0820, ppl:   5.7227, duration: 29.3158s\n",
      "2020-01-02 11:34:17,152 Epoch   7: total training loss 6297.91\n",
      "2020-01-02 11:34:17,153 EPOCH 8\n",
      "2020-01-02 11:34:27,955 Epoch   8 Step:    23100 Batch Loss:     2.064579 Tokens per Sec:    18748, Lr: 0.000300\n",
      "2020-01-02 11:34:39,761 Epoch   8 Step:    23200 Batch Loss:     2.359523 Tokens per Sec:    19527, Lr: 0.000300\n",
      "2020-01-02 11:34:51,456 Epoch   8 Step:    23300 Batch Loss:     2.296349 Tokens per Sec:    19182, Lr: 0.000300\n",
      "2020-01-02 11:35:03,166 Epoch   8 Step:    23400 Batch Loss:     1.606494 Tokens per Sec:    19387, Lr: 0.000300\n",
      "2020-01-02 11:35:14,989 Epoch   8 Step:    23500 Batch Loss:     1.813542 Tokens per Sec:    19407, Lr: 0.000300\n",
      "2020-01-02 11:35:26,769 Epoch   8 Step:    23600 Batch Loss:     1.990092 Tokens per Sec:    19450, Lr: 0.000300\n",
      "2020-01-02 11:35:38,460 Epoch   8 Step:    23700 Batch Loss:     1.822804 Tokens per Sec:    19024, Lr: 0.000300\n",
      "2020-01-02 11:35:50,187 Epoch   8 Step:    23800 Batch Loss:     1.602755 Tokens per Sec:    19478, Lr: 0.000300\n",
      "2020-01-02 11:36:01,965 Epoch   8 Step:    23900 Batch Loss:     1.787577 Tokens per Sec:    19126, Lr: 0.000300\n",
      "2020-01-02 11:36:13,793 Epoch   8 Step:    24000 Batch Loss:     1.946015 Tokens per Sec:    19508, Lr: 0.000300\n",
      "2020-01-02 11:36:43,154 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:36:43,154 Saving new checkpoint.\n",
      "2020-01-02 11:36:43,404 Example #0\n",
      "2020-01-02 11:36:43,404 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:36:43,404 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:36:43,404 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanulekedi tshintu tshianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 11:36:43,404 Example #1\n",
      "2020-01-02 11:36:43,405 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:36:43,405 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:36:43,405 \tHypothesis: Bu muvua bana babu babenga ne bamushipe , bakamuambila bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:36:43,405 Example #2\n",
      "2020-01-02 11:36:43,405 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:36:43,405 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:36:43,405 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:36:43,405 Example #3\n",
      "2020-01-02 11:36:43,405 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:36:43,405 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:36:43,406 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa dia dibika dia bafue .\n",
      "2020-01-02 11:36:43,406 Validation result (greedy) at epoch   8, step    24000: bleu:  22.32, loss: 44329.7617, ppl:   5.5693, duration: 29.6122s\n",
      "2020-01-02 11:36:55,189 Epoch   8 Step:    24100 Batch Loss:     1.884475 Tokens per Sec:    19695, Lr: 0.000300\n",
      "2020-01-02 11:37:06,901 Epoch   8 Step:    24200 Batch Loss:     1.854593 Tokens per Sec:    19269, Lr: 0.000300\n",
      "2020-01-02 11:37:18,741 Epoch   8 Step:    24300 Batch Loss:     2.005785 Tokens per Sec:    19592, Lr: 0.000300\n",
      "2020-01-02 11:37:30,647 Epoch   8 Step:    24400 Batch Loss:     1.948807 Tokens per Sec:    19718, Lr: 0.000300\n",
      "2020-01-02 11:37:42,530 Epoch   8 Step:    24500 Batch Loss:     2.030665 Tokens per Sec:    19749, Lr: 0.000300\n",
      "2020-01-02 11:37:54,257 Epoch   8 Step:    24600 Batch Loss:     1.894539 Tokens per Sec:    19231, Lr: 0.000300\n",
      "2020-01-02 11:38:06,043 Epoch   8 Step:    24700 Batch Loss:     1.700870 Tokens per Sec:    19281, Lr: 0.000300\n",
      "2020-01-02 11:38:17,712 Epoch   8 Step:    24800 Batch Loss:     1.949090 Tokens per Sec:    19195, Lr: 0.000300\n",
      "2020-01-02 11:38:29,498 Epoch   8 Step:    24900 Batch Loss:     1.931039 Tokens per Sec:    19299, Lr: 0.000300\n",
      "2020-01-02 11:38:41,324 Epoch   8 Step:    25000 Batch Loss:     1.669127 Tokens per Sec:    19374, Lr: 0.000300\n",
      "2020-01-02 11:39:10,582 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:39:10,582 Saving new checkpoint.\n",
      "2020-01-02 11:39:10,820 Example #0\n",
      "2020-01-02 11:39:10,821 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:39:10,821 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:39:10,821 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanulekedi , kadi kanuena nuangata muaba webe . ’\n",
      "2020-01-02 11:39:10,821 Example #1\n",
      "2020-01-02 11:39:10,821 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:39:10,821 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:39:10,821 \tHypothesis: Bu muvua bana babu babenga ne bamushikile , bakamuenzeja bua kumbuka mu ditunga adi divua dimutumina .\n",
      "2020-01-02 11:39:10,821 Example #2\n",
      "2020-01-02 11:39:10,821 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:39:10,821 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:39:10,821 \tHypothesis: Nunku , Yehowa uvua mua kumvua njila yende , wakaleja patoke ne : nansha yeye kayi wanyisha mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:39:10,821 Example #3\n",
      "2020-01-02 11:39:10,821 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:39:10,821 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:39:10,822 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa bua dibika dia bafue .\n",
      "2020-01-02 11:39:10,822 Validation result (greedy) at epoch   8, step    25000: bleu:  22.45, loss: 43693.2930, ppl:   5.4337, duration: 29.4978s\n",
      "2020-01-02 11:39:22,637 Epoch   8 Step:    25100 Batch Loss:     2.355102 Tokens per Sec:    19834, Lr: 0.000300\n",
      "2020-01-02 11:39:34,457 Epoch   8 Step:    25200 Batch Loss:     2.020176 Tokens per Sec:    19188, Lr: 0.000300\n",
      "2020-01-02 11:39:46,157 Epoch   8 Step:    25300 Batch Loss:     1.629022 Tokens per Sec:    19414, Lr: 0.000300\n",
      "2020-01-02 11:39:58,006 Epoch   8 Step:    25400 Batch Loss:     1.869711 Tokens per Sec:    19652, Lr: 0.000300\n",
      "2020-01-02 11:40:09,808 Epoch   8 Step:    25500 Batch Loss:     1.653661 Tokens per Sec:    19253, Lr: 0.000300\n",
      "2020-01-02 11:40:21,445 Epoch   8 Step:    25600 Batch Loss:     1.553859 Tokens per Sec:    18878, Lr: 0.000300\n",
      "2020-01-02 11:40:33,193 Epoch   8 Step:    25700 Batch Loss:     1.465299 Tokens per Sec:    19448, Lr: 0.000300\n",
      "2020-01-02 11:40:44,934 Epoch   8 Step:    25800 Batch Loss:     1.785111 Tokens per Sec:    19546, Lr: 0.000300\n",
      "2020-01-02 11:40:56,785 Epoch   8 Step:    25900 Batch Loss:     1.658743 Tokens per Sec:    19598, Lr: 0.000300\n",
      "2020-01-02 11:41:08,478 Epoch   8 Step:    26000 Batch Loss:     1.841159 Tokens per Sec:    19209, Lr: 0.000300\n",
      "2020-01-02 11:41:37,766 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:41:37,766 Saving new checkpoint.\n",
      "2020-01-02 11:41:38,030 Example #0\n",
      "2020-01-02 11:41:38,031 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:41:38,031 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:41:38,031 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanulekedi , kanuena nuya . ’\n",
      "2020-01-02 11:41:38,031 Example #1\n",
      "2020-01-02 11:41:38,031 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:41:38,031 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:41:38,031 \tHypothesis: Bu muvua bana babu bamukine ne bamushipe , bakamusaka bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:41:38,031 Example #2\n",
      "2020-01-02 11:41:38,031 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:41:38,031 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:41:38,031 \tHypothesis: Nunku , Mose uvua mua kumvua bimpe menemene njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 11:41:38,031 Example #3\n",
      "2020-01-02 11:41:38,031 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:41:38,031 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:41:38,031 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 11:41:38,032 Validation result (greedy) at epoch   8, step    26000: bleu:  22.52, loss: 43213.6836, ppl:   5.3337, duration: 29.5535s\n",
      "2020-01-02 11:41:49,846 Epoch   8 Step:    26100 Batch Loss:     2.196059 Tokens per Sec:    19445, Lr: 0.000300\n",
      "2020-01-02 11:42:01,791 Epoch   8 Step:    26200 Batch Loss:     1.940081 Tokens per Sec:    19430, Lr: 0.000300\n",
      "2020-01-02 11:42:13,581 Epoch   8 Step:    26300 Batch Loss:     2.189533 Tokens per Sec:    19141, Lr: 0.000300\n",
      "2020-01-02 11:42:14,181 Epoch   8: total training loss 6112.25\n",
      "2020-01-02 11:42:14,182 EPOCH 9\n",
      "2020-01-02 11:42:25,703 Epoch   9 Step:    26400 Batch Loss:     1.674912 Tokens per Sec:    18731, Lr: 0.000300\n",
      "2020-01-02 11:42:37,647 Epoch   9 Step:    26500 Batch Loss:     1.570919 Tokens per Sec:    19647, Lr: 0.000300\n",
      "2020-01-02 11:42:49,440 Epoch   9 Step:    26600 Batch Loss:     1.583163 Tokens per Sec:    19886, Lr: 0.000300\n",
      "2020-01-02 11:43:01,355 Epoch   9 Step:    26700 Batch Loss:     1.775659 Tokens per Sec:    19370, Lr: 0.000300\n",
      "2020-01-02 11:43:13,034 Epoch   9 Step:    26800 Batch Loss:     1.251023 Tokens per Sec:    19620, Lr: 0.000300\n",
      "2020-01-02 11:43:24,867 Epoch   9 Step:    26900 Batch Loss:     1.909518 Tokens per Sec:    19923, Lr: 0.000300\n",
      "2020-01-02 11:43:36,689 Epoch   9 Step:    27000 Batch Loss:     1.821359 Tokens per Sec:    19184, Lr: 0.000300\n",
      "2020-01-02 11:44:05,993 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:44:05,994 Saving new checkpoint.\n",
      "2020-01-02 11:44:06,276 Example #0\n",
      "2020-01-02 11:44:06,277 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:44:06,277 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:44:06,277 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanulekedi , kadi kanuena nuenza . ’\n",
      "2020-01-02 11:44:06,277 Example #1\n",
      "2020-01-02 11:44:06,277 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:44:06,277 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:44:06,277 \tHypothesis: Bu muvua bana babu babenga ne bamukine , bakamuenzeja bua kumbuka mu ditunga divua dimutumine .\n",
      "2020-01-02 11:44:06,277 Example #2\n",
      "2020-01-02 11:44:06,277 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:44:06,277 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:44:06,277 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakajingulula patoke ne : nansha mudiye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 11:44:06,277 Example #3\n",
      "2020-01-02 11:44:06,277 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:44:06,277 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:44:06,277 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa dia dibika dia bafue .\n",
      "2020-01-02 11:44:06,277 Validation result (greedy) at epoch   9, step    27000: bleu:  22.80, loss: 42980.8711, ppl:   5.2858, duration: 29.5879s\n",
      "2020-01-02 11:44:18,130 Epoch   9 Step:    27100 Batch Loss:     1.749432 Tokens per Sec:    19795, Lr: 0.000300\n",
      "2020-01-02 11:44:29,949 Epoch   9 Step:    27200 Batch Loss:     2.093867 Tokens per Sec:    19562, Lr: 0.000300\n",
      "2020-01-02 11:44:41,694 Epoch   9 Step:    27300 Batch Loss:     1.587272 Tokens per Sec:    19039, Lr: 0.000300\n",
      "2020-01-02 11:44:53,511 Epoch   9 Step:    27400 Batch Loss:     1.628432 Tokens per Sec:    19590, Lr: 0.000300\n",
      "2020-01-02 11:45:05,340 Epoch   9 Step:    27500 Batch Loss:     1.645901 Tokens per Sec:    19373, Lr: 0.000300\n",
      "2020-01-02 11:45:17,050 Epoch   9 Step:    27600 Batch Loss:     1.951946 Tokens per Sec:    18966, Lr: 0.000300\n",
      "2020-01-02 11:45:28,981 Epoch   9 Step:    27700 Batch Loss:     1.730793 Tokens per Sec:    19364, Lr: 0.000300\n",
      "2020-01-02 11:45:40,812 Epoch   9 Step:    27800 Batch Loss:     1.893958 Tokens per Sec:    19596, Lr: 0.000300\n",
      "2020-01-02 11:45:52,470 Epoch   9 Step:    27900 Batch Loss:     2.104831 Tokens per Sec:    19340, Lr: 0.000300\n",
      "2020-01-02 11:46:04,362 Epoch   9 Step:    28000 Batch Loss:     1.995121 Tokens per Sec:    19446, Lr: 0.000300\n",
      "2020-01-02 11:46:33,642 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:46:33,642 Saving new checkpoint.\n",
      "2020-01-02 11:46:33,917 Example #0\n",
      "2020-01-02 11:46:33,918 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:46:33,918 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:46:33,918 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : “ Kanulekedi , kadi nuenze malu anudi nuenza . ”\n",
      "2020-01-02 11:46:33,918 Example #1\n",
      "2020-01-02 11:46:33,918 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:46:33,918 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:46:33,918 \tHypothesis: Bu muvua bana babu bamukine ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:46:33,918 Example #2\n",
      "2020-01-02 11:46:33,918 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:46:33,918 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:46:33,918 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene ne : nansha muvuaye kayi wanyisha mpekatu , udi ubenga bua kulumbuluisha .\n",
      "2020-01-02 11:46:33,918 Example #3\n",
      "2020-01-02 11:46:33,919 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:46:33,919 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:46:33,919 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 11:46:33,919 Validation result (greedy) at epoch   9, step    28000: bleu:  23.39, loss: 42610.9062, ppl:   5.2106, duration: 29.5569s\n",
      "2020-01-02 11:46:45,655 Epoch   9 Step:    28100 Batch Loss:     1.851619 Tokens per Sec:    19083, Lr: 0.000300\n",
      "2020-01-02 11:46:57,564 Epoch   9 Step:    28200 Batch Loss:     1.388702 Tokens per Sec:    19540, Lr: 0.000300\n",
      "2020-01-02 11:47:09,457 Epoch   9 Step:    28300 Batch Loss:     1.961200 Tokens per Sec:    19690, Lr: 0.000300\n",
      "2020-01-02 11:47:21,198 Epoch   9 Step:    28400 Batch Loss:     1.592588 Tokens per Sec:    19293, Lr: 0.000300\n",
      "2020-01-02 11:47:32,942 Epoch   9 Step:    28500 Batch Loss:     2.085013 Tokens per Sec:    19427, Lr: 0.000300\n",
      "2020-01-02 11:47:44,718 Epoch   9 Step:    28600 Batch Loss:     1.825272 Tokens per Sec:    18997, Lr: 0.000300\n",
      "2020-01-02 11:47:56,475 Epoch   9 Step:    28700 Batch Loss:     1.787926 Tokens per Sec:    19383, Lr: 0.000300\n",
      "2020-01-02 11:48:08,197 Epoch   9 Step:    28800 Batch Loss:     1.939443 Tokens per Sec:    19337, Lr: 0.000300\n",
      "2020-01-02 11:48:20,165 Epoch   9 Step:    28900 Batch Loss:     1.721479 Tokens per Sec:    19778, Lr: 0.000300\n",
      "2020-01-02 11:48:31,851 Epoch   9 Step:    29000 Batch Loss:     2.206229 Tokens per Sec:    19257, Lr: 0.000300\n",
      "2020-01-02 11:49:01,115 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:49:01,115 Saving new checkpoint.\n",
      "2020-01-02 11:49:01,354 Example #0\n",
      "2020-01-02 11:49:01,355 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:49:01,355 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:49:01,355 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanulekedi , kadi kanuena nuangata muaba . ’\n",
      "2020-01-02 11:49:01,355 Example #1\n",
      "2020-01-02 11:49:01,355 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:49:01,355 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:49:01,355 \tHypothesis: Bu muvua bana babu babenga ne bamushikile , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:49:01,355 Example #2\n",
      "2020-01-02 11:49:01,355 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:49:01,356 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:49:01,356 \tHypothesis: Nunku , Mose uvua ne bua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:49:01,356 Example #3\n",
      "2020-01-02 11:49:01,356 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:49:01,356 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:49:01,356 \tHypothesis: Ndi mujadike ne : badi mu divuluka dia Yehowa dibika dia bafue .\n",
      "2020-01-02 11:49:01,356 Validation result (greedy) at epoch   9, step    29000: bleu:  23.64, loss: 42535.6523, ppl:   5.1954, duration: 29.5050s\n",
      "2020-01-02 11:49:13,129 Epoch   9 Step:    29100 Batch Loss:     1.509785 Tokens per Sec:    19230, Lr: 0.000300\n",
      "2020-01-02 11:49:24,870 Epoch   9 Step:    29200 Batch Loss:     1.737535 Tokens per Sec:    19231, Lr: 0.000300\n",
      "2020-01-02 11:49:36,786 Epoch   9 Step:    29300 Batch Loss:     2.090702 Tokens per Sec:    19762, Lr: 0.000300\n",
      "2020-01-02 11:49:48,605 Epoch   9 Step:    29400 Batch Loss:     1.686306 Tokens per Sec:    19332, Lr: 0.000300\n",
      "2020-01-02 11:50:00,361 Epoch   9 Step:    29500 Batch Loss:     2.157708 Tokens per Sec:    19252, Lr: 0.000300\n",
      "2020-01-02 11:50:10,662 Epoch   9: total training loss 5935.49\n",
      "2020-01-02 11:50:10,662 EPOCH 10\n",
      "2020-01-02 11:50:12,459 Epoch  10 Step:    29600 Batch Loss:     1.528454 Tokens per Sec:    16236, Lr: 0.000300\n",
      "2020-01-02 11:50:24,331 Epoch  10 Step:    29700 Batch Loss:     1.924003 Tokens per Sec:    19430, Lr: 0.000300\n",
      "2020-01-02 11:50:36,034 Epoch  10 Step:    29800 Batch Loss:     1.799883 Tokens per Sec:    19277, Lr: 0.000300\n",
      "2020-01-02 11:50:47,898 Epoch  10 Step:    29900 Batch Loss:     1.461035 Tokens per Sec:    19795, Lr: 0.000300\n",
      "2020-01-02 11:50:59,592 Epoch  10 Step:    30000 Batch Loss:     1.523376 Tokens per Sec:    19194, Lr: 0.000300\n",
      "2020-01-02 11:51:28,861 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:51:28,861 Saving new checkpoint.\n",
      "2020-01-02 11:51:29,134 Example #0\n",
      "2020-01-02 11:51:29,135 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:51:29,135 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:51:29,135 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatu wende ne : ‘ Kanuenji bualu buanyi , kadi kanuena nuangata . ’\n",
      "2020-01-02 11:51:29,135 Example #1\n",
      "2020-01-02 11:51:29,135 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:51:29,135 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:51:29,135 \tHypothesis: Bu muvua bana babu babenga ne bamukina , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 11:51:29,135 Example #2\n",
      "2020-01-02 11:51:29,135 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:51:29,135 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:51:29,135 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 11:51:29,135 Example #3\n",
      "2020-01-02 11:51:29,136 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:51:29,136 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:51:29,136 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa diabisha bafue .\n",
      "2020-01-02 11:51:29,136 Validation result (greedy) at epoch  10, step    30000: bleu:  24.03, loss: 42312.2852, ppl:   5.1506, duration: 29.5435s\n",
      "2020-01-02 11:51:41,030 Epoch  10 Step:    30100 Batch Loss:     2.002472 Tokens per Sec:    19741, Lr: 0.000300\n",
      "2020-01-02 11:51:52,764 Epoch  10 Step:    30200 Batch Loss:     1.763584 Tokens per Sec:    19109, Lr: 0.000300\n",
      "2020-01-02 11:52:04,682 Epoch  10 Step:    30300 Batch Loss:     1.590502 Tokens per Sec:    19707, Lr: 0.000300\n",
      "2020-01-02 11:52:16,661 Epoch  10 Step:    30400 Batch Loss:     1.684549 Tokens per Sec:    19724, Lr: 0.000300\n",
      "2020-01-02 11:52:28,464 Epoch  10 Step:    30500 Batch Loss:     1.807932 Tokens per Sec:    19095, Lr: 0.000300\n",
      "2020-01-02 11:52:40,201 Epoch  10 Step:    30600 Batch Loss:     1.579819 Tokens per Sec:    19154, Lr: 0.000300\n",
      "2020-01-02 11:52:51,889 Epoch  10 Step:    30700 Batch Loss:     1.792493 Tokens per Sec:    19131, Lr: 0.000300\n",
      "2020-01-02 11:53:03,646 Epoch  10 Step:    30800 Batch Loss:     2.048460 Tokens per Sec:    19076, Lr: 0.000300\n",
      "2020-01-02 11:53:15,340 Epoch  10 Step:    30900 Batch Loss:     1.932754 Tokens per Sec:    19072, Lr: 0.000300\n",
      "2020-01-02 11:53:27,088 Epoch  10 Step:    31000 Batch Loss:     1.802704 Tokens per Sec:    19341, Lr: 0.000300\n",
      "2020-01-02 11:53:56,390 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:53:56,391 Saving new checkpoint.\n",
      "2020-01-02 11:53:56,630 Example #0\n",
      "2020-01-02 11:53:56,631 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:53:56,631 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:53:56,631 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanulekedi , kadi kanuena nuangata muaba wanyi . ’\n",
      "2020-01-02 11:53:56,631 Example #1\n",
      "2020-01-02 11:53:56,631 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:53:56,631 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:53:56,631 \tHypothesis: Bu muvua bana babu babenga ne bamukine , bakamuenzeja bua kumbuka mu ditunga divua dimutua ku tshiibi .\n",
      "2020-01-02 11:53:56,631 Example #2\n",
      "2020-01-02 11:53:56,632 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:53:56,632 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:53:56,632 \tHypothesis: Nunku , Yehowa uvua ne bua kumvua njila yende , wakadianjila kuleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji .\n",
      "2020-01-02 11:53:56,632 Example #3\n",
      "2020-01-02 11:53:56,632 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:53:56,632 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:53:56,632 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa diindila dia dibika dia bafue .\n",
      "2020-01-02 11:53:56,632 Validation result (greedy) at epoch  10, step    31000: bleu:  23.96, loss: 41830.4297, ppl:   5.0554, duration: 29.5441s\n",
      "2020-01-02 11:54:08,522 Epoch  10 Step:    31100 Batch Loss:     1.793334 Tokens per Sec:    19654, Lr: 0.000300\n",
      "2020-01-02 11:54:20,234 Epoch  10 Step:    31200 Batch Loss:     1.890899 Tokens per Sec:    19517, Lr: 0.000300\n",
      "2020-01-02 11:54:32,119 Epoch  10 Step:    31300 Batch Loss:     1.951227 Tokens per Sec:    19539, Lr: 0.000300\n",
      "2020-01-02 11:54:43,993 Epoch  10 Step:    31400 Batch Loss:     1.759292 Tokens per Sec:    19464, Lr: 0.000300\n",
      "2020-01-02 11:54:55,826 Epoch  10 Step:    31500 Batch Loss:     1.783415 Tokens per Sec:    19198, Lr: 0.000300\n",
      "2020-01-02 11:55:07,672 Epoch  10 Step:    31600 Batch Loss:     1.603964 Tokens per Sec:    18777, Lr: 0.000300\n",
      "2020-01-02 11:55:19,408 Epoch  10 Step:    31700 Batch Loss:     1.551089 Tokens per Sec:    19389, Lr: 0.000300\n",
      "2020-01-02 11:55:31,196 Epoch  10 Step:    31800 Batch Loss:     2.119261 Tokens per Sec:    19050, Lr: 0.000300\n",
      "2020-01-02 11:55:43,004 Epoch  10 Step:    31900 Batch Loss:     1.778865 Tokens per Sec:    19451, Lr: 0.000300\n",
      "2020-01-02 11:55:54,749 Epoch  10 Step:    32000 Batch Loss:     1.920575 Tokens per Sec:    19248, Lr: 0.000300\n",
      "2020-01-02 11:56:24,045 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:56:24,046 Saving new checkpoint.\n",
      "2020-01-02 11:56:24,307 Example #0\n",
      "2020-01-02 11:56:24,308 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:56:24,308 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:56:24,308 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanulekedi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 11:56:24,308 Example #1\n",
      "2020-01-02 11:56:24,308 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:56:24,308 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:56:24,308 \tHypothesis: Bu muvua bana babu bamukine ne bamukine , bakamuenzeja bua kumbuka mu ditunga divua dileja ne : bavua bamushipe .\n",
      "2020-01-02 11:56:24,308 Example #2\n",
      "2020-01-02 11:56:24,309 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:56:24,309 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:56:24,309 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi witaba mpekatu , udi ukuata tshiji .\n",
      "2020-01-02 11:56:24,309 Example #3\n",
      "2020-01-02 11:56:24,309 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:56:24,309 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:56:24,309 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa dia dibika dia bafue .\n",
      "2020-01-02 11:56:24,309 Validation result (greedy) at epoch  10, step    32000: bleu:  24.14, loss: 41604.1445, ppl:   5.0113, duration: 29.5605s\n",
      "2020-01-02 11:56:36,238 Epoch  10 Step:    32100 Batch Loss:     1.549515 Tokens per Sec:    19395, Lr: 0.000300\n",
      "2020-01-02 11:56:47,966 Epoch  10 Step:    32200 Batch Loss:     1.990538 Tokens per Sec:    19010, Lr: 0.000300\n",
      "2020-01-02 11:56:59,736 Epoch  10 Step:    32300 Batch Loss:     1.808720 Tokens per Sec:    19174, Lr: 0.000300\n",
      "2020-01-02 11:57:11,598 Epoch  10 Step:    32400 Batch Loss:     1.853688 Tokens per Sec:    19799, Lr: 0.000300\n",
      "2020-01-02 11:57:23,332 Epoch  10 Step:    32500 Batch Loss:     1.954371 Tokens per Sec:    19170, Lr: 0.000300\n",
      "2020-01-02 11:57:35,188 Epoch  10 Step:    32600 Batch Loss:     1.676861 Tokens per Sec:    18749, Lr: 0.000300\n",
      "2020-01-02 11:57:46,958 Epoch  10 Step:    32700 Batch Loss:     1.843011 Tokens per Sec:    19452, Lr: 0.000300\n",
      "2020-01-02 11:57:58,685 Epoch  10 Step:    32800 Batch Loss:     1.684641 Tokens per Sec:    19523, Lr: 0.000300\n",
      "2020-01-02 11:58:08,754 Epoch  10: total training loss 5824.58\n",
      "2020-01-02 11:58:08,754 EPOCH 11\n",
      "2020-01-02 11:58:10,885 Epoch  11 Step:    32900 Batch Loss:     1.541743 Tokens per Sec:    16767, Lr: 0.000300\n",
      "2020-01-02 11:58:22,678 Epoch  11 Step:    33000 Batch Loss:     1.963629 Tokens per Sec:    19612, Lr: 0.000300\n",
      "2020-01-02 11:58:51,949 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 11:58:51,949 Saving new checkpoint.\n",
      "2020-01-02 11:58:52,190 Example #0\n",
      "2020-01-02 11:58:52,191 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 11:58:52,191 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 11:58:52,191 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanulekedi malu anyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 11:58:52,191 Example #1\n",
      "2020-01-02 11:58:52,191 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 11:58:52,191 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 11:58:52,191 \tHypothesis: Bu muvua bana babu ne mukawu , bakamuenzeja bua kumbuka mu ditunga divua dimutue ku mutu kua mbulamatadi .\n",
      "2020-01-02 11:58:52,191 Example #2\n",
      "2020-01-02 11:58:52,192 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 11:58:52,192 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 11:58:52,192 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 11:58:52,192 Example #3\n",
      "2020-01-02 11:58:52,192 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 11:58:52,192 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 11:58:52,192 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 11:58:52,192 Validation result (greedy) at epoch  11, step    33000: bleu:  24.10, loss: 41377.3203, ppl:   4.9674, duration: 29.5141s\n",
      "2020-01-02 11:59:04,060 Epoch  11 Step:    33100 Batch Loss:     1.847925 Tokens per Sec:    19468, Lr: 0.000300\n",
      "2020-01-02 11:59:15,814 Epoch  11 Step:    33200 Batch Loss:     1.812759 Tokens per Sec:    19452, Lr: 0.000300\n",
      "2020-01-02 11:59:27,580 Epoch  11 Step:    33300 Batch Loss:     1.706146 Tokens per Sec:    19311, Lr: 0.000300\n",
      "2020-01-02 11:59:39,351 Epoch  11 Step:    33400 Batch Loss:     1.826053 Tokens per Sec:    19444, Lr: 0.000300\n",
      "2020-01-02 11:59:51,147 Epoch  11 Step:    33500 Batch Loss:     1.722953 Tokens per Sec:    19345, Lr: 0.000300\n",
      "2020-01-02 12:00:02,819 Epoch  11 Step:    33600 Batch Loss:     1.704188 Tokens per Sec:    19008, Lr: 0.000300\n",
      "2020-01-02 12:00:14,668 Epoch  11 Step:    33700 Batch Loss:     1.821980 Tokens per Sec:    19806, Lr: 0.000300\n",
      "2020-01-02 12:00:26,494 Epoch  11 Step:    33800 Batch Loss:     1.867930 Tokens per Sec:    19527, Lr: 0.000300\n",
      "2020-01-02 12:00:38,141 Epoch  11 Step:    33900 Batch Loss:     1.729012 Tokens per Sec:    18921, Lr: 0.000300\n",
      "2020-01-02 12:00:50,002 Epoch  11 Step:    34000 Batch Loss:     1.582884 Tokens per Sec:    19872, Lr: 0.000300\n",
      "2020-01-02 12:01:19,276 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:01:19,276 Saving new checkpoint.\n",
      "2020-01-02 12:01:19,538 Example #0\n",
      "2020-01-02 12:01:19,538 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:01:19,538 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:01:19,538 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudisunguidi tshianyi , kadi wewe udi ulua . ’\n",
      "2020-01-02 12:01:19,539 Example #1\n",
      "2020-01-02 12:01:19,549 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:01:19,550 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:01:19,550 \tHypothesis: Bu muvua bana babu ne lukuna ne bamushipe , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:01:19,550 Example #2\n",
      "2020-01-02 12:01:19,550 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:01:19,550 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:01:19,551 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 12:01:19,551 Example #3\n",
      "2020-01-02 12:01:19,551 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:01:19,551 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:01:19,551 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa dikadi diindile dibika dia bafue .\n",
      "2020-01-02 12:01:19,551 Validation result (greedy) at epoch  11, step    34000: bleu:  24.52, loss: 41289.3906, ppl:   4.9505, duration: 29.5492s\n",
      "2020-01-02 12:01:31,508 Epoch  11 Step:    34100 Batch Loss:     1.816790 Tokens per Sec:    19521, Lr: 0.000300\n",
      "2020-01-02 12:01:43,283 Epoch  11 Step:    34200 Batch Loss:     1.937090 Tokens per Sec:    19344, Lr: 0.000300\n",
      "2020-01-02 12:01:55,038 Epoch  11 Step:    34300 Batch Loss:     1.668146 Tokens per Sec:    19196, Lr: 0.000300\n",
      "2020-01-02 12:02:06,714 Epoch  11 Step:    34400 Batch Loss:     1.926898 Tokens per Sec:    19363, Lr: 0.000300\n",
      "2020-01-02 12:02:18,489 Epoch  11 Step:    34500 Batch Loss:     1.680268 Tokens per Sec:    19591, Lr: 0.000300\n",
      "2020-01-02 12:02:30,330 Epoch  11 Step:    34600 Batch Loss:     1.757284 Tokens per Sec:    19384, Lr: 0.000300\n",
      "2020-01-02 12:02:42,227 Epoch  11 Step:    34700 Batch Loss:     1.614441 Tokens per Sec:    18968, Lr: 0.000300\n",
      "2020-01-02 12:02:54,030 Epoch  11 Step:    34800 Batch Loss:     1.461823 Tokens per Sec:    19167, Lr: 0.000300\n",
      "2020-01-02 12:03:05,883 Epoch  11 Step:    34900 Batch Loss:     1.604262 Tokens per Sec:    19277, Lr: 0.000300\n",
      "2020-01-02 12:03:17,634 Epoch  11 Step:    35000 Batch Loss:     1.526108 Tokens per Sec:    19617, Lr: 0.000300\n",
      "2020-01-02 12:03:46,903 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:03:46,903 Saving new checkpoint.\n",
      "2020-01-02 12:03:47,161 Example #0\n",
      "2020-01-02 12:03:47,166 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:03:47,167 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:03:47,167 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanulekedi disua dianyi , kadi nuenze malu . ’\n",
      "2020-01-02 12:03:47,167 Example #1\n",
      "2020-01-02 12:03:47,167 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:03:47,167 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:03:47,167 \tHypothesis: Bu muvua bana babu ne mukawu , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:03:47,167 Example #2\n",
      "2020-01-02 12:03:47,167 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:03:47,167 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:03:47,167 \tHypothesis: Nunku , Mose uvua ne bua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , udi ukuata tshiji .\n",
      "2020-01-02 12:03:47,167 Example #3\n",
      "2020-01-02 12:03:47,167 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:03:47,168 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:03:47,168 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 12:03:47,168 Validation result (greedy) at epoch  11, step    35000: bleu:  24.52, loss: 40840.1680, ppl:   4.8651, duration: 29.5334s\n",
      "2020-01-02 12:03:58,942 Epoch  11 Step:    35100 Batch Loss:     1.596765 Tokens per Sec:    19276, Lr: 0.000300\n",
      "2020-01-02 12:04:10,721 Epoch  11 Step:    35200 Batch Loss:     1.687351 Tokens per Sec:    19420, Lr: 0.000300\n",
      "2020-01-02 12:04:22,574 Epoch  11 Step:    35300 Batch Loss:     1.029878 Tokens per Sec:    19686, Lr: 0.000300\n",
      "2020-01-02 12:04:34,383 Epoch  11 Step:    35400 Batch Loss:     2.002840 Tokens per Sec:    19361, Lr: 0.000300\n",
      "2020-01-02 12:04:46,079 Epoch  11 Step:    35500 Batch Loss:     1.585923 Tokens per Sec:    19317, Lr: 0.000300\n",
      "2020-01-02 12:04:57,878 Epoch  11 Step:    35600 Batch Loss:     1.133354 Tokens per Sec:    19304, Lr: 0.000300\n",
      "2020-01-02 12:05:09,746 Epoch  11 Step:    35700 Batch Loss:     1.523103 Tokens per Sec:    19604, Lr: 0.000300\n",
      "2020-01-02 12:05:21,433 Epoch  11 Step:    35800 Batch Loss:     1.874074 Tokens per Sec:    19431, Lr: 0.000300\n",
      "2020-01-02 12:05:33,297 Epoch  11 Step:    35900 Batch Loss:     1.698042 Tokens per Sec:    19323, Lr: 0.000300\n",
      "2020-01-02 12:05:45,192 Epoch  11 Step:    36000 Batch Loss:     1.775265 Tokens per Sec:    19323, Lr: 0.000300\n",
      "2020-01-02 12:06:14,575 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:06:14,575 Saving new checkpoint.\n",
      "2020-01-02 12:06:14,818 Example #0\n",
      "2020-01-02 12:06:14,819 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:06:14,819 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:06:14,819 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanulekedi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 12:06:14,819 Example #1\n",
      "2020-01-02 12:06:14,819 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:06:14,819 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:06:14,819 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamusaka bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:06:14,819 Example #2\n",
      "2020-01-02 12:06:14,819 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:06:14,819 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:06:14,820 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene ne : nansha muvuaye kayi wanyisha mpekatu , kêna ufika munda to .\n",
      "2020-01-02 12:06:14,820 Example #3\n",
      "2020-01-02 12:06:14,820 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:06:14,820 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:06:14,820 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa dia dibika dia bafue .\n",
      "2020-01-02 12:06:14,820 Validation result (greedy) at epoch  11, step    36000: bleu:  24.56, loss: 40723.8984, ppl:   4.8433, duration: 29.6275s\n",
      "2020-01-02 12:06:26,708 Epoch  11 Step:    36100 Batch Loss:     1.788893 Tokens per Sec:    19528, Lr: 0.000300\n",
      "2020-01-02 12:06:35,452 Epoch  11: total training loss 5697.30\n",
      "2020-01-02 12:06:35,453 EPOCH 12\n",
      "2020-01-02 12:06:38,751 Epoch  12 Step:    36200 Batch Loss:     1.745046 Tokens per Sec:    17656, Lr: 0.000300\n",
      "2020-01-02 12:06:50,537 Epoch  12 Step:    36300 Batch Loss:     1.772953 Tokens per Sec:    19129, Lr: 0.000300\n",
      "2020-01-02 12:07:02,345 Epoch  12 Step:    36400 Batch Loss:     1.740670 Tokens per Sec:    18994, Lr: 0.000300\n",
      "2020-01-02 12:07:14,184 Epoch  12 Step:    36500 Batch Loss:     1.439579 Tokens per Sec:    19567, Lr: 0.000300\n",
      "2020-01-02 12:07:26,021 Epoch  12 Step:    36600 Batch Loss:     1.569310 Tokens per Sec:    19467, Lr: 0.000300\n",
      "2020-01-02 12:07:37,932 Epoch  12 Step:    36700 Batch Loss:     1.791692 Tokens per Sec:    19348, Lr: 0.000300\n",
      "2020-01-02 12:07:49,772 Epoch  12 Step:    36800 Batch Loss:     1.509672 Tokens per Sec:    19128, Lr: 0.000300\n",
      "2020-01-02 12:08:01,638 Epoch  12 Step:    36900 Batch Loss:     1.540881 Tokens per Sec:    19718, Lr: 0.000300\n",
      "2020-01-02 12:08:13,471 Epoch  12 Step:    37000 Batch Loss:     1.686094 Tokens per Sec:    19312, Lr: 0.000300\n",
      "2020-01-02 12:08:42,809 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:08:42,809 Saving new checkpoint.\n",
      "2020-01-02 12:08:43,051 Example #0\n",
      "2020-01-02 12:08:43,051 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:08:43,051 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:08:43,051 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudisunguidi tshianyi , kadi nudi nuangata bu tshimuenekelu tshia bantu . ’\n",
      "2020-01-02 12:08:43,052 Example #1\n",
      "2020-01-02 12:08:43,052 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:08:43,052 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:08:43,052 \tHypothesis: Bu muvua bana babu ne mukawu , bakamusaka bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:08:43,052 Example #2\n",
      "2020-01-02 12:08:43,052 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:08:43,052 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:08:43,052 \tHypothesis: Nunku , Mose uvua ne bua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , kêna ukuata tshiji to .\n",
      "2020-01-02 12:08:43,052 Example #3\n",
      "2020-01-02 12:08:43,052 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:08:43,052 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:08:43,052 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibishibua dia bantu ku lufu .\n",
      "2020-01-02 12:08:43,052 Validation result (greedy) at epoch  12, step    37000: bleu:  24.77, loss: 40494.3125, ppl:   4.8004, duration: 29.5804s\n",
      "2020-01-02 12:08:54,844 Epoch  12 Step:    37100 Batch Loss:     1.313380 Tokens per Sec:    19329, Lr: 0.000300\n",
      "2020-01-02 12:09:06,722 Epoch  12 Step:    37200 Batch Loss:     1.696955 Tokens per Sec:    19314, Lr: 0.000300\n",
      "2020-01-02 12:09:18,557 Epoch  12 Step:    37300 Batch Loss:     1.636680 Tokens per Sec:    19229, Lr: 0.000300\n",
      "2020-01-02 12:09:30,260 Epoch  12 Step:    37400 Batch Loss:     1.318887 Tokens per Sec:    19252, Lr: 0.000300\n",
      "2020-01-02 12:09:42,170 Epoch  12 Step:    37500 Batch Loss:     1.783827 Tokens per Sec:    19389, Lr: 0.000300\n",
      "2020-01-02 12:09:53,984 Epoch  12 Step:    37600 Batch Loss:     1.788200 Tokens per Sec:    19451, Lr: 0.000300\n",
      "2020-01-02 12:10:05,781 Epoch  12 Step:    37700 Batch Loss:     1.470804 Tokens per Sec:    19154, Lr: 0.000300\n",
      "2020-01-02 12:10:17,489 Epoch  12 Step:    37800 Batch Loss:     1.619909 Tokens per Sec:    19575, Lr: 0.000300\n",
      "2020-01-02 12:10:29,357 Epoch  12 Step:    37900 Batch Loss:     1.686952 Tokens per Sec:    19375, Lr: 0.000300\n",
      "2020-01-02 12:10:41,128 Epoch  12 Step:    38000 Batch Loss:     1.614372 Tokens per Sec:    19212, Lr: 0.000300\n",
      "2020-01-02 12:11:10,463 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:11:10,463 Saving new checkpoint.\n",
      "2020-01-02 12:11:10,716 Example #0\n",
      "2020-01-02 12:11:10,716 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:11:10,717 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:11:10,717 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanulekedi disua dianyi , kadi nuenze malu enu . ’\n",
      "2020-01-02 12:11:10,717 Example #1\n",
      "2020-01-02 12:11:10,717 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:11:10,717 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:11:10,717 \tHypothesis: Bu muvua bana babu ne mukawu , bakamusaka bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:11:10,717 Example #2\n",
      "2020-01-02 12:11:10,717 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:11:10,717 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:11:10,717 \tHypothesis: Nunku , Mose uvua ne bua kumvua njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi muanyishe mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 12:11:10,717 Example #3\n",
      "2020-01-02 12:11:10,717 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:11:10,717 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:11:10,717 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshiindila dibika dia bafue .\n",
      "2020-01-02 12:11:10,717 Validation result (greedy) at epoch  12, step    38000: bleu:  24.98, loss: 40365.3438, ppl:   4.7765, duration: 29.5892s\n",
      "2020-01-02 12:11:22,687 Epoch  12 Step:    38100 Batch Loss:     1.956142 Tokens per Sec:    19763, Lr: 0.000300\n",
      "2020-01-02 12:11:34,464 Epoch  12 Step:    38200 Batch Loss:     1.196975 Tokens per Sec:    19111, Lr: 0.000300\n",
      "2020-01-02 12:11:46,245 Epoch  12 Step:    38300 Batch Loss:     1.497480 Tokens per Sec:    19558, Lr: 0.000300\n",
      "2020-01-02 12:11:58,119 Epoch  12 Step:    38400 Batch Loss:     1.659431 Tokens per Sec:    19399, Lr: 0.000300\n",
      "2020-01-02 12:12:09,823 Epoch  12 Step:    38500 Batch Loss:     1.867938 Tokens per Sec:    18952, Lr: 0.000300\n",
      "2020-01-02 12:12:21,635 Epoch  12 Step:    38600 Batch Loss:     1.779552 Tokens per Sec:    19175, Lr: 0.000300\n",
      "2020-01-02 12:12:33,503 Epoch  12 Step:    38700 Batch Loss:     1.890115 Tokens per Sec:    19862, Lr: 0.000300\n",
      "2020-01-02 12:12:45,311 Epoch  12 Step:    38800 Batch Loss:     1.688144 Tokens per Sec:    19272, Lr: 0.000300\n",
      "2020-01-02 12:12:57,198 Epoch  12 Step:    38900 Batch Loss:     1.433387 Tokens per Sec:    19287, Lr: 0.000300\n",
      "2020-01-02 12:13:08,905 Epoch  12 Step:    39000 Batch Loss:     1.847583 Tokens per Sec:    19107, Lr: 0.000300\n",
      "2020-01-02 12:13:38,210 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:13:38,210 Saving new checkpoint.\n",
      "2020-01-02 12:13:38,460 Example #0\n",
      "2020-01-02 12:13:38,460 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:13:38,460 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:13:38,460 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanupu , kanuenji malu anyi , kadi nuenu nuenze bu muudi musue . ’\n",
      "2020-01-02 12:13:38,460 Example #1\n",
      "2020-01-02 12:13:38,461 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:13:38,461 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:13:38,461 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:13:38,461 Example #2\n",
      "2020-01-02 12:13:38,461 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:13:38,461 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:13:38,461 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , udi ubenga bua kumvua njila yende .\n",
      "2020-01-02 12:13:38,461 Example #3\n",
      "2020-01-02 12:13:38,462 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:13:38,462 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:13:38,462 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa badi bindila dibika dia bafue .\n",
      "2020-01-02 12:13:38,462 Validation result (greedy) at epoch  12, step    39000: bleu:  24.51, loss: 40251.4766, ppl:   4.7554, duration: 29.5571s\n",
      "2020-01-02 12:13:50,212 Epoch  12 Step:    39100 Batch Loss:     1.792815 Tokens per Sec:    19229, Lr: 0.000300\n",
      "2020-01-02 12:14:01,991 Epoch  12 Step:    39200 Batch Loss:     1.694070 Tokens per Sec:    19159, Lr: 0.000300\n",
      "2020-01-02 12:14:13,914 Epoch  12 Step:    39300 Batch Loss:     1.877643 Tokens per Sec:    20166, Lr: 0.000300\n",
      "2020-01-02 12:14:25,671 Epoch  12 Step:    39400 Batch Loss:     1.784977 Tokens per Sec:    19372, Lr: 0.000300\n",
      "2020-01-02 12:14:33,487 Epoch  12: total training loss 5603.49\n",
      "2020-01-02 12:14:33,488 EPOCH 13\n",
      "2020-01-02 12:14:37,790 Epoch  13 Step:    39500 Batch Loss:     1.769821 Tokens per Sec:    18005, Lr: 0.000300\n",
      "2020-01-02 12:14:49,557 Epoch  13 Step:    39600 Batch Loss:     1.641377 Tokens per Sec:    19757, Lr: 0.000300\n",
      "2020-01-02 12:15:01,358 Epoch  13 Step:    39700 Batch Loss:     1.218839 Tokens per Sec:    19289, Lr: 0.000300\n",
      "2020-01-02 12:15:13,152 Epoch  13 Step:    39800 Batch Loss:     1.394885 Tokens per Sec:    19460, Lr: 0.000300\n",
      "2020-01-02 12:15:24,877 Epoch  13 Step:    39900 Batch Loss:     1.688078 Tokens per Sec:    19556, Lr: 0.000300\n",
      "2020-01-02 12:15:36,687 Epoch  13 Step:    40000 Batch Loss:     1.528411 Tokens per Sec:    19590, Lr: 0.000300\n",
      "2020-01-02 12:16:06,059 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:16:06,059 Saving new checkpoint.\n",
      "2020-01-02 12:16:06,318 Example #0\n",
      "2020-01-02 12:16:06,318 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:16:06,319 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:16:06,319 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanuenji bualu buanyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:16:06,319 Example #1\n",
      "2020-01-02 12:16:06,319 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:16:06,319 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:16:06,319 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:16:06,319 Example #2\n",
      "2020-01-02 12:16:06,319 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:16:06,319 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:16:06,319 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:16:06,319 Example #3\n",
      "2020-01-02 12:16:06,319 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:16:06,319 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:16:06,319 \tHypothesis: Ndi mujadike ne : badi mu tshivulukilu tshia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 12:16:06,319 Validation result (greedy) at epoch  13, step    40000: bleu:  24.92, loss: 40045.0820, ppl:   4.7176, duration: 29.6322s\n",
      "2020-01-02 12:16:18,209 Epoch  13 Step:    40100 Batch Loss:     1.709421 Tokens per Sec:    19498, Lr: 0.000300\n",
      "2020-01-02 12:16:30,016 Epoch  13 Step:    40200 Batch Loss:     1.375931 Tokens per Sec:    19537, Lr: 0.000300\n",
      "2020-01-02 12:16:41,722 Epoch  13 Step:    40300 Batch Loss:     1.579914 Tokens per Sec:    19263, Lr: 0.000300\n",
      "2020-01-02 12:16:53,635 Epoch  13 Step:    40400 Batch Loss:     1.701456 Tokens per Sec:    19767, Lr: 0.000300\n",
      "2020-01-02 12:17:05,277 Epoch  13 Step:    40500 Batch Loss:     1.445637 Tokens per Sec:    18846, Lr: 0.000300\n",
      "2020-01-02 12:17:16,994 Epoch  13 Step:    40600 Batch Loss:     1.714162 Tokens per Sec:    19040, Lr: 0.000300\n",
      "2020-01-02 12:17:28,702 Epoch  13 Step:    40700 Batch Loss:     1.750705 Tokens per Sec:    19181, Lr: 0.000300\n",
      "2020-01-02 12:17:40,643 Epoch  13 Step:    40800 Batch Loss:     1.761242 Tokens per Sec:    19429, Lr: 0.000300\n",
      "2020-01-02 12:17:52,438 Epoch  13 Step:    40900 Batch Loss:     1.723787 Tokens per Sec:    19285, Lr: 0.000300\n",
      "2020-01-02 12:18:04,094 Epoch  13 Step:    41000 Batch Loss:     1.952360 Tokens per Sec:    18939, Lr: 0.000300\n",
      "2020-01-02 12:18:33,401 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:18:33,402 Saving new checkpoint.\n",
      "2020-01-02 12:18:33,668 Example #0\n",
      "2020-01-02 12:18:33,669 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:18:33,669 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:18:33,669 \tHypothesis: YESOSTO mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanulekedi disua dianyi , kadi nuenu nuenze bu mudidi dienzeka . ’\n",
      "2020-01-02 12:18:33,669 Example #1\n",
      "2020-01-02 12:18:33,669 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:18:33,669 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:18:33,669 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:18:33,669 Example #2\n",
      "2020-01-02 12:18:33,670 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:18:33,670 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:18:33,670 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene njila yende , bualu uvua mujadike ne : nansha muvuaye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 12:18:33,670 Example #3\n",
      "2020-01-02 12:18:33,670 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:18:33,670 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:18:33,670 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa ditshidi dibindile dibika dia bafue .\n",
      "2020-01-02 12:18:33,670 Validation result (greedy) at epoch  13, step    41000: bleu:  25.38, loss: 39896.3906, ppl:   4.6905, duration: 29.5758s\n",
      "2020-01-02 12:18:45,515 Epoch  13 Step:    41100 Batch Loss:     1.445228 Tokens per Sec:    19319, Lr: 0.000300\n",
      "2020-01-02 12:18:57,403 Epoch  13 Step:    41200 Batch Loss:     1.752075 Tokens per Sec:    19246, Lr: 0.000300\n",
      "2020-01-02 12:19:09,214 Epoch  13 Step:    41300 Batch Loss:     1.562130 Tokens per Sec:    19367, Lr: 0.000300\n",
      "2020-01-02 12:19:21,009 Epoch  13 Step:    41400 Batch Loss:     1.451138 Tokens per Sec:    19556, Lr: 0.000300\n",
      "2020-01-02 12:19:32,902 Epoch  13 Step:    41500 Batch Loss:     1.476585 Tokens per Sec:    19707, Lr: 0.000300\n",
      "2020-01-02 12:19:44,678 Epoch  13 Step:    41600 Batch Loss:     1.470467 Tokens per Sec:    19090, Lr: 0.000300\n",
      "2020-01-02 12:19:56,456 Epoch  13 Step:    41700 Batch Loss:     1.713960 Tokens per Sec:    19452, Lr: 0.000300\n",
      "2020-01-02 12:20:08,203 Epoch  13 Step:    41800 Batch Loss:     1.443063 Tokens per Sec:    19627, Lr: 0.000300\n",
      "2020-01-02 12:20:20,027 Epoch  13 Step:    41900 Batch Loss:     1.558962 Tokens per Sec:    19331, Lr: 0.000300\n",
      "2020-01-02 12:20:31,819 Epoch  13 Step:    42000 Batch Loss:     1.889925 Tokens per Sec:    19353, Lr: 0.000300\n",
      "2020-01-02 12:21:01,126 Example #0\n",
      "2020-01-02 12:21:01,126 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:21:01,126 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:21:01,127 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanuenji malu anyi , kadi nuenze bu muenu . ’\n",
      "2020-01-02 12:21:01,127 Example #1\n",
      "2020-01-02 12:21:01,128 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:21:01,128 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:21:01,128 \tHypothesis: Bu muvua bana babu ne mukawu ne bamushipe , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutuma .\n",
      "2020-01-02 12:21:01,128 Example #2\n",
      "2020-01-02 12:21:01,128 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:21:01,128 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:21:01,128 \tHypothesis: Nunku , Yehowa uvua mua kumvua Mose njila yende , bualu uvua uleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:21:01,128 Example #3\n",
      "2020-01-02 12:21:01,128 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:21:01,129 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:21:01,129 \tHypothesis: Ndi mutuishibue ne : badi mu divuluka dia Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 12:21:01,129 Validation result (greedy) at epoch  13, step    42000: bleu:  25.05, loss: 39901.0977, ppl:   4.6913, duration: 29.3098s\n",
      "2020-01-02 12:21:12,904 Epoch  13 Step:    42100 Batch Loss:     1.678204 Tokens per Sec:    19673, Lr: 0.000300\n",
      "2020-01-02 12:21:24,683 Epoch  13 Step:    42200 Batch Loss:     1.954503 Tokens per Sec:    19220, Lr: 0.000300\n",
      "2020-01-02 12:21:36,560 Epoch  13 Step:    42300 Batch Loss:     1.516572 Tokens per Sec:    19529, Lr: 0.000300\n",
      "2020-01-02 12:21:48,299 Epoch  13 Step:    42400 Batch Loss:     1.583227 Tokens per Sec:    19083, Lr: 0.000300\n",
      "2020-01-02 12:22:00,224 Epoch  13 Step:    42500 Batch Loss:     2.070624 Tokens per Sec:    19612, Lr: 0.000300\n",
      "2020-01-02 12:22:12,133 Epoch  13 Step:    42600 Batch Loss:     1.547973 Tokens per Sec:    19626, Lr: 0.000300\n",
      "2020-01-02 12:22:23,975 Epoch  13 Step:    42700 Batch Loss:     1.489839 Tokens per Sec:    19746, Lr: 0.000300\n",
      "2020-01-02 12:22:30,284 Epoch  13: total training loss 5507.98\n",
      "2020-01-02 12:22:30,285 EPOCH 14\n",
      "2020-01-02 12:22:36,046 Epoch  14 Step:    42800 Batch Loss:     1.678953 Tokens per Sec:    17991, Lr: 0.000300\n",
      "2020-01-02 12:22:47,776 Epoch  14 Step:    42900 Batch Loss:     1.646692 Tokens per Sec:    19356, Lr: 0.000300\n",
      "2020-01-02 12:22:59,487 Epoch  14 Step:    43000 Batch Loss:     1.777358 Tokens per Sec:    18952, Lr: 0.000300\n",
      "2020-01-02 12:23:28,862 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:23:28,862 Saving new checkpoint.\n",
      "2020-01-02 12:23:29,110 Example #0\n",
      "2020-01-02 12:23:29,110 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:23:29,110 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:23:29,110 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji malu andi musue , kadi nuenze . ’\n",
      "2020-01-02 12:23:29,110 Example #1\n",
      "2020-01-02 12:23:29,110 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:23:29,110 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:23:29,111 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishibue ku mêsu kuende .\n",
      "2020-01-02 12:23:29,111 Example #2\n",
      "2020-01-02 12:23:29,111 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:23:29,111 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:23:29,111 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe bua mushindu uvuaye kayi wanyisha mpekatu , kadi kêna ukuata tshiji to .\n",
      "2020-01-02 12:23:29,111 Example #3\n",
      "2020-01-02 12:23:29,111 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:23:29,111 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:23:29,111 \tHypothesis: Ndi mushindike ne : badi mu lungenyi lua Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 12:23:29,111 Validation result (greedy) at epoch  14, step    43000: bleu:  26.35, loss: 39630.2930, ppl:   4.6424, duration: 29.6240s\n",
      "2020-01-02 12:23:40,901 Epoch  14 Step:    43100 Batch Loss:     1.683308 Tokens per Sec:    19326, Lr: 0.000300\n",
      "2020-01-02 12:23:52,667 Epoch  14 Step:    43200 Batch Loss:     1.641725 Tokens per Sec:    19594, Lr: 0.000300\n",
      "2020-01-02 12:24:04,578 Epoch  14 Step:    43300 Batch Loss:     1.401560 Tokens per Sec:    19676, Lr: 0.000300\n",
      "2020-01-02 12:24:16,330 Epoch  14 Step:    43400 Batch Loss:     1.552978 Tokens per Sec:    19373, Lr: 0.000300\n",
      "2020-01-02 12:24:28,289 Epoch  14 Step:    43500 Batch Loss:     1.704537 Tokens per Sec:    19603, Lr: 0.000300\n",
      "2020-01-02 12:24:40,002 Epoch  14 Step:    43600 Batch Loss:     1.459488 Tokens per Sec:    19065, Lr: 0.000300\n",
      "2020-01-02 12:24:51,755 Epoch  14 Step:    43700 Batch Loss:     1.835134 Tokens per Sec:    19714, Lr: 0.000300\n",
      "2020-01-02 12:25:03,573 Epoch  14 Step:    43800 Batch Loss:     1.625248 Tokens per Sec:    19439, Lr: 0.000300\n",
      "2020-01-02 12:25:15,295 Epoch  14 Step:    43900 Batch Loss:     1.694308 Tokens per Sec:    19598, Lr: 0.000300\n",
      "2020-01-02 12:25:27,152 Epoch  14 Step:    44000 Batch Loss:     1.765265 Tokens per Sec:    19528, Lr: 0.000300\n",
      "2020-01-02 12:25:56,459 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:25:56,459 Saving new checkpoint.\n",
      "2020-01-02 12:25:56,717 Example #0\n",
      "2020-01-02 12:25:56,718 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:25:56,718 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:25:56,718 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshidi disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:25:56,718 Example #1\n",
      "2020-01-02 12:25:56,718 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:25:56,718 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:25:56,718 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:25:56,718 Example #2\n",
      "2020-01-02 12:25:56,718 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:25:56,718 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:25:56,718 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe bua mushindu uvuaye wenza malu mabi , bualu uvua mupangile tshiji .\n",
      "2020-01-02 12:25:56,718 Example #3\n",
      "2020-01-02 12:25:56,718 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:25:56,718 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:25:56,718 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 12:25:56,719 Validation result (greedy) at epoch  14, step    44000: bleu:  26.00, loss: 39337.2578, ppl:   4.5900, duration: 29.5659s\n",
      "2020-01-02 12:26:08,522 Epoch  14 Step:    44100 Batch Loss:     1.632318 Tokens per Sec:    18989, Lr: 0.000300\n",
      "2020-01-02 12:26:20,278 Epoch  14 Step:    44200 Batch Loss:     1.713286 Tokens per Sec:    18984, Lr: 0.000300\n",
      "2020-01-02 12:26:32,129 Epoch  14 Step:    44300 Batch Loss:     1.735960 Tokens per Sec:    19669, Lr: 0.000300\n",
      "2020-01-02 12:26:44,049 Epoch  14 Step:    44400 Batch Loss:     1.682473 Tokens per Sec:    19617, Lr: 0.000300\n",
      "2020-01-02 12:26:55,866 Epoch  14 Step:    44500 Batch Loss:     1.525358 Tokens per Sec:    19217, Lr: 0.000300\n",
      "2020-01-02 12:27:07,757 Epoch  14 Step:    44600 Batch Loss:     1.673940 Tokens per Sec:    19245, Lr: 0.000300\n",
      "2020-01-02 12:27:19,598 Epoch  14 Step:    44700 Batch Loss:     1.809169 Tokens per Sec:    19727, Lr: 0.000300\n",
      "2020-01-02 12:27:31,415 Epoch  14 Step:    44800 Batch Loss:     1.839206 Tokens per Sec:    19370, Lr: 0.000300\n",
      "2020-01-02 12:27:43,299 Epoch  14 Step:    44900 Batch Loss:     1.734048 Tokens per Sec:    19846, Lr: 0.000300\n",
      "2020-01-02 12:27:55,068 Epoch  14 Step:    45000 Batch Loss:     2.071354 Tokens per Sec:    19225, Lr: 0.000300\n",
      "2020-01-02 12:28:24,386 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:28:24,386 Saving new checkpoint.\n",
      "2020-01-02 12:28:24,632 Example #0\n",
      "2020-01-02 12:28:24,632 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:28:24,632 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:28:24,632 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanuenji malu anyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:28:24,632 Example #1\n",
      "2020-01-02 12:28:24,632 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:28:24,632 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:28:24,632 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:28:24,632 Example #2\n",
      "2020-01-02 12:28:24,633 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:28:24,633 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:28:24,633 \tHypothesis: Nunku , Yehowa uvua mua kumvua njila yende , bualu uvua mua kuikala mumvuije patoke ne : nansha mudiye kayi wanyisha bubi , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:28:24,633 Example #3\n",
      "2020-01-02 12:28:24,633 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:28:24,633 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:28:24,633 \tHypothesis: Ndi mushindike ne : badi mu divuluka dia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 12:28:24,633 Validation result (greedy) at epoch  14, step    45000: bleu:  25.62, loss: 39216.5039, ppl:   4.5685, duration: 29.5654s\n",
      "2020-01-02 12:28:36,644 Epoch  14 Step:    45100 Batch Loss:     1.603470 Tokens per Sec:    19771, Lr: 0.000300\n",
      "2020-01-02 12:28:48,361 Epoch  14 Step:    45200 Batch Loss:     1.410292 Tokens per Sec:    19111, Lr: 0.000300\n",
      "2020-01-02 12:29:00,106 Epoch  14 Step:    45300 Batch Loss:     1.832644 Tokens per Sec:    18816, Lr: 0.000300\n",
      "2020-01-02 12:29:11,905 Epoch  14 Step:    45400 Batch Loss:     1.762724 Tokens per Sec:    19850, Lr: 0.000300\n",
      "2020-01-02 12:29:23,843 Epoch  14 Step:    45500 Batch Loss:     1.615610 Tokens per Sec:    19844, Lr: 0.000300\n",
      "2020-01-02 12:29:35,661 Epoch  14 Step:    45600 Batch Loss:     1.469144 Tokens per Sec:    19364, Lr: 0.000300\n",
      "2020-01-02 12:29:47,457 Epoch  14 Step:    45700 Batch Loss:     1.474736 Tokens per Sec:    19199, Lr: 0.000300\n",
      "2020-01-02 12:29:59,331 Epoch  14 Step:    45800 Batch Loss:     1.600278 Tokens per Sec:    19564, Lr: 0.000300\n",
      "2020-01-02 12:30:11,099 Epoch  14 Step:    45900 Batch Loss:     2.019675 Tokens per Sec:    19325, Lr: 0.000300\n",
      "2020-01-02 12:30:22,927 Epoch  14 Step:    46000 Batch Loss:     1.950228 Tokens per Sec:    18945, Lr: 0.000300\n",
      "2020-01-02 12:30:52,177 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:30:52,177 Saving new checkpoint.\n",
      "2020-01-02 12:30:52,453 Example #0\n",
      "2020-01-02 12:30:52,453 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:30:52,453 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:30:52,453 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:30:52,453 Example #1\n",
      "2020-01-02 12:30:52,453 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:30:52,453 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:30:52,453 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:30:52,453 Example #2\n",
      "2020-01-02 12:30:52,453 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:30:52,453 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:30:52,454 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene njila yende , bualu uvua mumanye bimpe ne : nansha mudiye kayi wanyisha bubi , kena ukuata tshiji to .\n",
      "2020-01-02 12:30:52,454 Example #3\n",
      "2020-01-02 12:30:52,454 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:30:52,454 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:30:52,454 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshiindila dibika dia bafue .\n",
      "2020-01-02 12:30:52,454 Validation result (greedy) at epoch  14, step    46000: bleu:  25.64, loss: 39148.9766, ppl:   4.5566, duration: 29.5264s\n",
      "2020-01-02 12:30:56,804 Epoch  14: total training loss 5423.96\n",
      "2020-01-02 12:30:56,804 EPOCH 15\n",
      "2020-01-02 12:31:04,643 Epoch  15 Step:    46100 Batch Loss:     1.630536 Tokens per Sec:    17885, Lr: 0.000300\n",
      "2020-01-02 12:31:16,328 Epoch  15 Step:    46200 Batch Loss:     1.320388 Tokens per Sec:    19394, Lr: 0.000300\n",
      "2020-01-02 12:31:28,066 Epoch  15 Step:    46300 Batch Loss:     1.634776 Tokens per Sec:    19178, Lr: 0.000300\n",
      "2020-01-02 12:31:39,806 Epoch  15 Step:    46400 Batch Loss:     1.765594 Tokens per Sec:    19304, Lr: 0.000300\n",
      "2020-01-02 12:31:51,659 Epoch  15 Step:    46500 Batch Loss:     1.811475 Tokens per Sec:    19458, Lr: 0.000300\n",
      "2020-01-02 12:32:03,461 Epoch  15 Step:    46600 Batch Loss:     1.491610 Tokens per Sec:    19701, Lr: 0.000300\n",
      "2020-01-02 12:32:15,347 Epoch  15 Step:    46700 Batch Loss:     1.730743 Tokens per Sec:    19636, Lr: 0.000300\n",
      "2020-01-02 12:32:27,198 Epoch  15 Step:    46800 Batch Loss:     1.625949 Tokens per Sec:    19165, Lr: 0.000300\n",
      "2020-01-02 12:32:39,025 Epoch  15 Step:    46900 Batch Loss:     1.510464 Tokens per Sec:    19227, Lr: 0.000300\n",
      "2020-01-02 12:32:50,799 Epoch  15 Step:    47000 Batch Loss:     1.607627 Tokens per Sec:    19700, Lr: 0.000300\n",
      "2020-01-02 12:33:20,102 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:33:20,102 Saving new checkpoint.\n",
      "2020-01-02 12:33:20,368 Example #0\n",
      "2020-01-02 12:33:20,368 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:33:20,368 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:33:20,368 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanulekedi disua dianyi , kadi nuenze bu muenu . ’\n",
      "2020-01-02 12:33:20,368 Example #1\n",
      "2020-01-02 12:33:20,369 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:33:20,369 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:33:20,369 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukina , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:33:20,369 Example #2\n",
      "2020-01-02 12:33:20,369 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:33:20,369 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:33:20,369 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe njila yende , wakaleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:33:20,369 Example #3\n",
      "2020-01-02 12:33:20,369 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:33:20,369 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:33:20,369 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshibindile dibishibua ku lufu .\n",
      "2020-01-02 12:33:20,370 Validation result (greedy) at epoch  15, step    47000: bleu:  25.88, loss: 38903.2422, ppl:   4.5134, duration: 29.5705s\n",
      "2020-01-02 12:33:32,273 Epoch  15 Step:    47100 Batch Loss:     1.675951 Tokens per Sec:    19089, Lr: 0.000300\n",
      "2020-01-02 12:33:44,137 Epoch  15 Step:    47200 Batch Loss:     1.832241 Tokens per Sec:    19528, Lr: 0.000300\n",
      "2020-01-02 12:33:55,904 Epoch  15 Step:    47300 Batch Loss:     2.007561 Tokens per Sec:    19702, Lr: 0.000300\n",
      "2020-01-02 12:34:07,714 Epoch  15 Step:    47400 Batch Loss:     1.730815 Tokens per Sec:    19423, Lr: 0.000300\n",
      "2020-01-02 12:34:19,447 Epoch  15 Step:    47500 Batch Loss:     1.972463 Tokens per Sec:    19522, Lr: 0.000300\n",
      "2020-01-02 12:34:31,440 Epoch  15 Step:    47600 Batch Loss:     1.667098 Tokens per Sec:    19756, Lr: 0.000300\n",
      "2020-01-02 12:34:43,102 Epoch  15 Step:    47700 Batch Loss:     1.738276 Tokens per Sec:    19258, Lr: 0.000300\n",
      "2020-01-02 12:34:54,907 Epoch  15 Step:    47800 Batch Loss:     1.555560 Tokens per Sec:    19374, Lr: 0.000300\n",
      "2020-01-02 12:35:06,714 Epoch  15 Step:    47900 Batch Loss:     1.640788 Tokens per Sec:    19541, Lr: 0.000300\n",
      "2020-01-02 12:35:18,485 Epoch  15 Step:    48000 Batch Loss:     1.514060 Tokens per Sec:    19589, Lr: 0.000300\n",
      "2020-01-02 12:35:47,730 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:35:47,730 Saving new checkpoint.\n",
      "2020-01-02 12:35:47,974 Example #0\n",
      "2020-01-02 12:35:47,975 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:35:47,975 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:35:47,975 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudiangatshidi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 12:35:47,975 Example #1\n",
      "2020-01-02 12:35:47,975 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:35:47,975 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:35:47,975 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:35:47,975 Example #2\n",
      "2020-01-02 12:35:47,975 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:35:47,976 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:35:47,976 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe njila yende , bualu uvua mujadike ne : nansha mudiye kayi wanyisha bubi , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 12:35:47,976 Example #3\n",
      "2020-01-02 12:35:47,976 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:35:47,976 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:35:47,976 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 12:35:47,976 Validation result (greedy) at epoch  15, step    48000: bleu:  25.97, loss: 38740.6328, ppl:   4.4851, duration: 29.4912s\n",
      "2020-01-02 12:35:59,857 Epoch  15 Step:    48100 Batch Loss:     1.902363 Tokens per Sec:    19088, Lr: 0.000300\n",
      "2020-01-02 12:36:11,643 Epoch  15 Step:    48200 Batch Loss:     1.814536 Tokens per Sec:    19209, Lr: 0.000300\n",
      "2020-01-02 12:36:23,411 Epoch  15 Step:    48300 Batch Loss:     1.679005 Tokens per Sec:    18961, Lr: 0.000300\n",
      "2020-01-02 12:36:35,237 Epoch  15 Step:    48400 Batch Loss:     2.114493 Tokens per Sec:    19462, Lr: 0.000300\n",
      "2020-01-02 12:36:47,011 Epoch  15 Step:    48500 Batch Loss:     1.533879 Tokens per Sec:    19397, Lr: 0.000300\n",
      "2020-01-02 12:36:58,808 Epoch  15 Step:    48600 Batch Loss:     1.793704 Tokens per Sec:    19672, Lr: 0.000300\n",
      "2020-01-02 12:37:10,536 Epoch  15 Step:    48700 Batch Loss:     1.702703 Tokens per Sec:    19220, Lr: 0.000300\n",
      "2020-01-02 12:37:22,181 Epoch  15 Step:    48800 Batch Loss:     1.648799 Tokens per Sec:    19213, Lr: 0.000300\n",
      "2020-01-02 12:37:33,936 Epoch  15 Step:    48900 Batch Loss:     1.870398 Tokens per Sec:    19103, Lr: 0.000300\n",
      "2020-01-02 12:37:45,718 Epoch  15 Step:    49000 Batch Loss:     1.578555 Tokens per Sec:    19498, Lr: 0.000300\n",
      "2020-01-02 12:38:14,960 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:38:14,960 Saving new checkpoint.\n",
      "2020-01-02 12:38:15,215 Example #0\n",
      "2020-01-02 12:38:15,215 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:38:15,215 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:38:15,216 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi , kadi malu ebe nealue . ’\n",
      "2020-01-02 12:38:15,216 Example #1\n",
      "2020-01-02 12:38:15,216 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:38:15,216 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:38:15,216 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutuma .\n",
      "2020-01-02 12:38:15,216 Example #2\n",
      "2020-01-02 12:38:15,216 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:38:15,216 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:38:15,216 \tHypothesis: Nunku , Yehowa uvua mua kumvua njila yende , bualu uvua uleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:38:15,216 Example #3\n",
      "2020-01-02 12:38:15,216 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:38:15,216 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:38:15,217 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 12:38:15,217 Validation result (greedy) at epoch  15, step    49000: bleu:  26.20, loss: 38619.5703, ppl:   4.4641, duration: 29.4980s\n",
      "2020-01-02 12:38:27,096 Epoch  15 Step:    49100 Batch Loss:     1.715346 Tokens per Sec:    19669, Lr: 0.000300\n",
      "2020-01-02 12:38:38,853 Epoch  15 Step:    49200 Batch Loss:     1.789429 Tokens per Sec:    19041, Lr: 0.000300\n",
      "2020-01-02 12:38:50,666 Epoch  15 Step:    49300 Batch Loss:     1.814335 Tokens per Sec:    19106, Lr: 0.000300\n",
      "2020-01-02 12:38:54,317 Epoch  15: total training loss 5376.43\n",
      "2020-01-02 12:38:54,317 EPOCH 16\n",
      "2020-01-02 12:39:02,813 Epoch  16 Step:    49400 Batch Loss:     1.598716 Tokens per Sec:    18496, Lr: 0.000300\n",
      "2020-01-02 12:39:14,643 Epoch  16 Step:    49500 Batch Loss:     1.671556 Tokens per Sec:    19673, Lr: 0.000300\n",
      "2020-01-02 12:39:26,354 Epoch  16 Step:    49600 Batch Loss:     1.804206 Tokens per Sec:    19290, Lr: 0.000300\n",
      "2020-01-02 12:39:38,123 Epoch  16 Step:    49700 Batch Loss:     1.614463 Tokens per Sec:    19766, Lr: 0.000300\n",
      "2020-01-02 12:39:49,814 Epoch  16 Step:    49800 Batch Loss:     1.599839 Tokens per Sec:    19304, Lr: 0.000300\n",
      "2020-01-02 12:40:01,580 Epoch  16 Step:    49900 Batch Loss:     1.521913 Tokens per Sec:    19362, Lr: 0.000300\n",
      "2020-01-02 12:40:13,380 Epoch  16 Step:    50000 Batch Loss:     1.788685 Tokens per Sec:    19238, Lr: 0.000300\n",
      "2020-01-02 12:40:42,728 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:40:42,728 Saving new checkpoint.\n",
      "2020-01-02 12:40:42,984 Example #0\n",
      "2020-01-02 12:40:42,984 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:40:42,984 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:40:42,985 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanupu disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 12:40:42,985 Example #1\n",
      "2020-01-02 12:40:42,985 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:40:42,985 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:40:42,985 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:40:42,985 Example #2\n",
      "2020-01-02 12:40:42,985 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:40:42,985 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:40:42,985 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene njila yende , bualu uvua mujadike ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:40:42,985 Example #3\n",
      "2020-01-02 12:40:42,986 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:40:42,986 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:40:42,986 \tHypothesis: Ndi mutuishibue ne : badi mu lungenyi lua Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 12:40:42,986 Validation result (greedy) at epoch  16, step    50000: bleu:  26.36, loss: 38614.6367, ppl:   4.4633, duration: 29.6052s\n",
      "2020-01-02 12:40:54,725 Epoch  16 Step:    50100 Batch Loss:     1.470244 Tokens per Sec:    18847, Lr: 0.000300\n",
      "2020-01-02 12:41:06,548 Epoch  16 Step:    50200 Batch Loss:     1.364568 Tokens per Sec:    19585, Lr: 0.000300\n",
      "2020-01-02 12:41:18,368 Epoch  16 Step:    50300 Batch Loss:     1.673682 Tokens per Sec:    19520, Lr: 0.000300\n",
      "2020-01-02 12:41:30,265 Epoch  16 Step:    50400 Batch Loss:     1.432892 Tokens per Sec:    19871, Lr: 0.000300\n",
      "2020-01-02 12:41:42,166 Epoch  16 Step:    50500 Batch Loss:     1.716535 Tokens per Sec:    19905, Lr: 0.000300\n",
      "2020-01-02 12:41:54,007 Epoch  16 Step:    50600 Batch Loss:     1.611530 Tokens per Sec:    19261, Lr: 0.000300\n",
      "2020-01-02 12:42:05,764 Epoch  16 Step:    50700 Batch Loss:     1.476823 Tokens per Sec:    19429, Lr: 0.000300\n",
      "2020-01-02 12:42:17,384 Epoch  16 Step:    50800 Batch Loss:     1.587503 Tokens per Sec:    19346, Lr: 0.000300\n",
      "2020-01-02 12:42:29,217 Epoch  16 Step:    50900 Batch Loss:     1.710888 Tokens per Sec:    19430, Lr: 0.000300\n",
      "2020-01-02 12:42:41,015 Epoch  16 Step:    51000 Batch Loss:     1.565248 Tokens per Sec:    20078, Lr: 0.000300\n",
      "2020-01-02 12:43:10,292 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:43:10,292 Saving new checkpoint.\n",
      "2020-01-02 12:43:10,536 Example #0\n",
      "2020-01-02 12:43:10,536 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:43:10,536 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:43:10,536 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze malu anudi nuenza . ’\n",
      "2020-01-02 12:43:10,536 Example #1\n",
      "2020-01-02 12:43:10,536 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:43:10,536 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:43:10,536 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutuma .\n",
      "2020-01-02 12:43:10,536 Example #2\n",
      "2020-01-02 12:43:10,536 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:43:10,536 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:43:10,536 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene ne : nansha mudiye kayi wanyisha mpekatu , kêna mua kumvua tshiji to .\n",
      "2020-01-02 12:43:10,536 Example #3\n",
      "2020-01-02 12:43:10,537 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:43:10,537 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:43:10,537 \tHypothesis: Ndi mutuishibue ne : badi mu lungenyi lua Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 12:43:10,537 Validation result (greedy) at epoch  16, step    51000: bleu:  26.03, loss: 38496.9375, ppl:   4.4430, duration: 29.5216s\n",
      "2020-01-02 12:43:22,305 Epoch  16 Step:    51100 Batch Loss:     1.381469 Tokens per Sec:    19385, Lr: 0.000300\n",
      "2020-01-02 12:43:34,051 Epoch  16 Step:    51200 Batch Loss:     1.578743 Tokens per Sec:    19165, Lr: 0.000300\n",
      "2020-01-02 12:43:45,764 Epoch  16 Step:    51300 Batch Loss:     1.741488 Tokens per Sec:    19252, Lr: 0.000300\n",
      "2020-01-02 12:43:57,638 Epoch  16 Step:    51400 Batch Loss:     1.444132 Tokens per Sec:    19489, Lr: 0.000300\n",
      "2020-01-02 12:44:09,465 Epoch  16 Step:    51500 Batch Loss:     1.770250 Tokens per Sec:    19726, Lr: 0.000300\n",
      "2020-01-02 12:44:21,120 Epoch  16 Step:    51600 Batch Loss:     1.724636 Tokens per Sec:    19069, Lr: 0.000300\n",
      "2020-01-02 12:44:32,856 Epoch  16 Step:    51700 Batch Loss:     1.674059 Tokens per Sec:    19434, Lr: 0.000300\n",
      "2020-01-02 12:44:44,630 Epoch  16 Step:    51800 Batch Loss:     1.632507 Tokens per Sec:    19546, Lr: 0.000300\n",
      "2020-01-02 12:44:56,338 Epoch  16 Step:    51900 Batch Loss:     1.783121 Tokens per Sec:    19395, Lr: 0.000300\n",
      "2020-01-02 12:45:08,026 Epoch  16 Step:    52000 Batch Loss:     1.298552 Tokens per Sec:    19327, Lr: 0.000300\n",
      "2020-01-02 12:45:37,265 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:45:37,265 Saving new checkpoint.\n",
      "2020-01-02 12:45:37,532 Example #0\n",
      "2020-01-02 12:45:37,533 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:45:37,533 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:45:37,533 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:45:37,533 Example #1\n",
      "2020-01-02 12:45:37,533 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:45:37,533 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:45:37,533 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:45:37,533 Example #2\n",
      "2020-01-02 12:45:37,533 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:45:37,533 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:45:37,533 \tHypothesis: Nunku , Yehowa uvua mua kumvua njila yende , bualu uvua muleje patoke ne : nansha muvuaye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:45:37,533 Example #3\n",
      "2020-01-02 12:45:37,533 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:45:37,533 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:45:37,534 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa ne badi bindila dibishibua dia bafue .\n",
      "2020-01-02 12:45:37,534 Validation result (greedy) at epoch  16, step    52000: bleu:  26.70, loss: 38251.8320, ppl:   4.4010, duration: 29.5071s\n",
      "2020-01-02 12:45:49,342 Epoch  16 Step:    52100 Batch Loss:     1.695063 Tokens per Sec:    19447, Lr: 0.000300\n",
      "2020-01-02 12:46:01,185 Epoch  16 Step:    52200 Batch Loss:     1.704251 Tokens per Sec:    19400, Lr: 0.000300\n",
      "2020-01-02 12:46:12,827 Epoch  16 Step:    52300 Batch Loss:     1.974264 Tokens per Sec:    19399, Lr: 0.000300\n",
      "2020-01-02 12:46:24,591 Epoch  16 Step:    52400 Batch Loss:     1.680222 Tokens per Sec:    19094, Lr: 0.000300\n",
      "2020-01-02 12:46:36,352 Epoch  16 Step:    52500 Batch Loss:     1.405110 Tokens per Sec:    19249, Lr: 0.000300\n",
      "2020-01-02 12:46:48,100 Epoch  16 Step:    52600 Batch Loss:     1.655506 Tokens per Sec:    19271, Lr: 0.000300\n",
      "2020-01-02 12:46:50,726 Epoch  16: total training loss 5304.89\n",
      "2020-01-02 12:46:50,726 EPOCH 17\n",
      "2020-01-02 12:47:00,257 Epoch  17 Step:    52700 Batch Loss:     1.605187 Tokens per Sec:    18606, Lr: 0.000300\n",
      "2020-01-02 12:47:12,141 Epoch  17 Step:    52800 Batch Loss:     1.549491 Tokens per Sec:    19753, Lr: 0.000300\n",
      "2020-01-02 12:47:24,014 Epoch  17 Step:    52900 Batch Loss:     1.730940 Tokens per Sec:    19741, Lr: 0.000300\n",
      "2020-01-02 12:47:35,823 Epoch  17 Step:    53000 Batch Loss:     1.543989 Tokens per Sec:    19504, Lr: 0.000300\n",
      "2020-01-02 12:48:05,079 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:48:05,079 Saving new checkpoint.\n",
      "2020-01-02 12:48:05,332 Example #0\n",
      "2020-01-02 12:48:05,333 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:48:05,333 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:48:05,333 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:48:05,333 Example #1\n",
      "2020-01-02 12:48:05,333 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:48:05,333 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:48:05,333 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutuma ku mikolo .\n",
      "2020-01-02 12:48:05,333 Example #2\n",
      "2020-01-02 12:48:05,333 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:48:05,333 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:48:05,334 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 12:48:05,334 Example #3\n",
      "2020-01-02 12:48:05,334 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:48:05,334 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:48:05,334 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshibindile dibishibua dia bafue .\n",
      "2020-01-02 12:48:05,334 Validation result (greedy) at epoch  17, step    53000: bleu:  26.51, loss: 38213.9375, ppl:   4.3945, duration: 29.5106s\n",
      "2020-01-02 12:48:17,186 Epoch  17 Step:    53100 Batch Loss:     1.202553 Tokens per Sec:    19447, Lr: 0.000300\n",
      "2020-01-02 12:48:29,077 Epoch  17 Step:    53200 Batch Loss:     1.548221 Tokens per Sec:    19769, Lr: 0.000300\n",
      "2020-01-02 12:48:40,857 Epoch  17 Step:    53300 Batch Loss:     1.727692 Tokens per Sec:    19408, Lr: 0.000300\n",
      "2020-01-02 12:48:52,667 Epoch  17 Step:    53400 Batch Loss:     1.738120 Tokens per Sec:    19776, Lr: 0.000300\n",
      "2020-01-02 12:49:04,378 Epoch  17 Step:    53500 Batch Loss:     1.493866 Tokens per Sec:    19110, Lr: 0.000300\n",
      "2020-01-02 12:49:16,086 Epoch  17 Step:    53600 Batch Loss:     1.533823 Tokens per Sec:    19187, Lr: 0.000300\n",
      "2020-01-02 12:49:27,910 Epoch  17 Step:    53700 Batch Loss:     1.523823 Tokens per Sec:    19012, Lr: 0.000300\n",
      "2020-01-02 12:49:39,747 Epoch  17 Step:    53800 Batch Loss:     1.682660 Tokens per Sec:    19660, Lr: 0.000300\n",
      "2020-01-02 12:49:51,569 Epoch  17 Step:    53900 Batch Loss:     1.666013 Tokens per Sec:    19403, Lr: 0.000300\n",
      "2020-01-02 12:50:03,431 Epoch  17 Step:    54000 Batch Loss:     1.761369 Tokens per Sec:    19894, Lr: 0.000300\n",
      "2020-01-02 12:50:32,690 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:50:32,690 Saving new checkpoint.\n",
      "2020-01-02 12:50:32,954 Example #0\n",
      "2020-01-02 12:50:32,954 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:50:32,954 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:50:32,955 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanuikadi disua dianyi , kadi nuenze bu bantu . ’\n",
      "2020-01-02 12:50:32,955 Example #1\n",
      "2020-01-02 12:50:32,955 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:50:32,955 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:50:32,955 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 12:50:32,955 Example #2\n",
      "2020-01-02 12:50:32,955 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:50:32,955 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:50:32,955 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe menemene njila yende , bualu uvua mujadike ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 12:50:32,955 Example #3\n",
      "2020-01-02 12:50:32,955 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:50:32,955 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:50:32,956 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshitshidi tshiindila dibika dia bafue .\n",
      "2020-01-02 12:50:32,956 Validation result (greedy) at epoch  17, step    54000: bleu:  26.46, loss: 38168.2539, ppl:   4.3867, duration: 29.5238s\n",
      "2020-01-02 12:50:44,731 Epoch  17 Step:    54100 Batch Loss:     1.636799 Tokens per Sec:    19032, Lr: 0.000300\n",
      "2020-01-02 12:50:56,557 Epoch  17 Step:    54200 Batch Loss:     1.799842 Tokens per Sec:    19728, Lr: 0.000300\n",
      "2020-01-02 12:51:08,478 Epoch  17 Step:    54300 Batch Loss:     1.462663 Tokens per Sec:    19589, Lr: 0.000300\n",
      "2020-01-02 12:51:20,191 Epoch  17 Step:    54400 Batch Loss:     1.551659 Tokens per Sec:    19232, Lr: 0.000300\n",
      "2020-01-02 12:51:32,103 Epoch  17 Step:    54500 Batch Loss:     1.893824 Tokens per Sec:    19783, Lr: 0.000300\n",
      "2020-01-02 12:51:43,945 Epoch  17 Step:    54600 Batch Loss:     1.346222 Tokens per Sec:    19671, Lr: 0.000300\n",
      "2020-01-02 12:51:55,712 Epoch  17 Step:    54700 Batch Loss:     1.806250 Tokens per Sec:    19474, Lr: 0.000300\n",
      "2020-01-02 12:52:07,449 Epoch  17 Step:    54800 Batch Loss:     1.570705 Tokens per Sec:    19411, Lr: 0.000300\n",
      "2020-01-02 12:52:19,165 Epoch  17 Step:    54900 Batch Loss:     1.358625 Tokens per Sec:    18986, Lr: 0.000300\n",
      "2020-01-02 12:52:31,071 Epoch  17 Step:    55000 Batch Loss:     1.889356 Tokens per Sec:    19458, Lr: 0.000300\n",
      "2020-01-02 12:53:00,319 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:53:00,319 Saving new checkpoint.\n",
      "2020-01-02 12:53:00,579 Example #0\n",
      "2020-01-02 12:53:00,579 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:53:00,579 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:53:00,579 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:53:00,579 Example #1\n",
      "2020-01-02 12:53:00,579 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:53:00,579 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:53:00,579 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishibue .\n",
      "2020-01-02 12:53:00,579 Example #2\n",
      "2020-01-02 12:53:00,579 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:53:00,579 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:53:00,580 \tHypothesis: Nunku , Yehowa uvua mua kumvua bimpe njila yende , bualu uvua mujadike bimpe ne : nansha muvuaye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 12:53:00,580 Example #3\n",
      "2020-01-02 12:53:00,580 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:53:00,580 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:53:00,580 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa ne ditekemena dia dibika dia bafue .\n",
      "2020-01-02 12:53:00,580 Validation result (greedy) at epoch  17, step    55000: bleu:  26.58, loss: 38101.6328, ppl:   4.3754, duration: 29.5089s\n",
      "2020-01-02 12:53:12,263 Epoch  17 Step:    55100 Batch Loss:     1.733870 Tokens per Sec:    18787, Lr: 0.000300\n",
      "2020-01-02 12:53:24,107 Epoch  17 Step:    55200 Batch Loss:     1.405350 Tokens per Sec:    19483, Lr: 0.000300\n",
      "2020-01-02 12:53:35,850 Epoch  17 Step:    55300 Batch Loss:     1.817880 Tokens per Sec:    19314, Lr: 0.000300\n",
      "2020-01-02 12:53:47,634 Epoch  17 Step:    55400 Batch Loss:     1.847015 Tokens per Sec:    19179, Lr: 0.000300\n",
      "2020-01-02 12:53:59,476 Epoch  17 Step:    55500 Batch Loss:     1.115499 Tokens per Sec:    19274, Lr: 0.000300\n",
      "2020-01-02 12:54:11,125 Epoch  17 Step:    55600 Batch Loss:     1.494285 Tokens per Sec:    19201, Lr: 0.000300\n",
      "2020-01-02 12:54:23,042 Epoch  17 Step:    55700 Batch Loss:     1.690531 Tokens per Sec:    19593, Lr: 0.000300\n",
      "2020-01-02 12:54:34,771 Epoch  17 Step:    55800 Batch Loss:     1.521234 Tokens per Sec:    18774, Lr: 0.000300\n",
      "2020-01-02 12:54:46,531 Epoch  17 Step:    55900 Batch Loss:     1.863235 Tokens per Sec:    19760, Lr: 0.000300\n",
      "2020-01-02 12:54:47,239 Epoch  17: total training loss 5231.77\n",
      "2020-01-02 12:54:47,239 EPOCH 18\n",
      "2020-01-02 12:54:58,782 Epoch  18 Step:    56000 Batch Loss:     1.708580 Tokens per Sec:    19063, Lr: 0.000300\n",
      "2020-01-02 12:55:28,052 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 12:55:28,052 Saving new checkpoint.\n",
      "2020-01-02 12:55:28,303 Example #0\n",
      "2020-01-02 12:55:28,304 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:55:28,304 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:55:28,304 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanupu disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:55:28,304 Example #1\n",
      "2020-01-02 12:55:28,304 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:55:28,304 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:55:28,304 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuangatshila mu mikenji .\n",
      "2020-01-02 12:55:28,304 Example #2\n",
      "2020-01-02 12:55:28,304 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:55:28,304 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:55:28,305 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : Mose uvua ne bua kumvua njila yende , kadi uvua ne bua kuikala muitabe bua bubi buvuaye kayi muanyishe , kadi ki mmumanye mua kufiika munda to .\n",
      "2020-01-02 12:55:28,305 Example #3\n",
      "2020-01-02 12:55:28,305 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:55:28,305 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:55:28,305 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 12:55:28,305 Validation result (greedy) at epoch  18, step    56000: bleu:  26.73, loss: 37680.6055, ppl:   4.3047, duration: 29.5231s\n",
      "2020-01-02 12:55:40,166 Epoch  18 Step:    56100 Batch Loss:     1.659315 Tokens per Sec:    19355, Lr: 0.000300\n",
      "2020-01-02 12:55:51,828 Epoch  18 Step:    56200 Batch Loss:     1.347902 Tokens per Sec:    19285, Lr: 0.000300\n",
      "2020-01-02 12:56:03,615 Epoch  18 Step:    56300 Batch Loss:     1.494424 Tokens per Sec:    19216, Lr: 0.000300\n",
      "2020-01-02 12:56:15,454 Epoch  18 Step:    56400 Batch Loss:     1.209448 Tokens per Sec:    20025, Lr: 0.000300\n",
      "2020-01-02 12:56:27,263 Epoch  18 Step:    56500 Batch Loss:     1.852089 Tokens per Sec:    19036, Lr: 0.000300\n",
      "2020-01-02 12:56:39,089 Epoch  18 Step:    56600 Batch Loss:     1.612660 Tokens per Sec:    19266, Lr: 0.000300\n",
      "2020-01-02 12:56:50,826 Epoch  18 Step:    56700 Batch Loss:     1.643156 Tokens per Sec:    19045, Lr: 0.000300\n",
      "2020-01-02 12:57:02,620 Epoch  18 Step:    56800 Batch Loss:     1.562269 Tokens per Sec:    19427, Lr: 0.000300\n",
      "2020-01-02 12:57:14,445 Epoch  18 Step:    56900 Batch Loss:     1.534591 Tokens per Sec:    19406, Lr: 0.000300\n",
      "2020-01-02 12:57:26,334 Epoch  18 Step:    57000 Batch Loss:     1.511395 Tokens per Sec:    19774, Lr: 0.000300\n",
      "2020-01-02 12:57:55,687 Example #0\n",
      "2020-01-02 12:57:55,687 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 12:57:55,687 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 12:57:55,687 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 12:57:55,687 Example #1\n",
      "2020-01-02 12:57:55,688 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 12:57:55,688 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 12:57:55,688 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 12:57:55,688 Example #2\n",
      "2020-01-02 12:57:55,688 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 12:57:55,688 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 12:57:55,688 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 12:57:55,688 Example #3\n",
      "2020-01-02 12:57:55,688 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 12:57:55,688 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 12:57:55,688 \tHypothesis: Ndi mushindike ne : badi mu lungenyi lua Yehowa bua kuindila dibishibua dia bafue .\n",
      "2020-01-02 12:57:55,688 Validation result (greedy) at epoch  18, step    57000: bleu:  26.95, loss: 37818.7812, ppl:   4.3278, duration: 29.3543s\n",
      "2020-01-02 12:58:07,331 Epoch  18 Step:    57100 Batch Loss:     1.716987 Tokens per Sec:    18741, Lr: 0.000300\n",
      "2020-01-02 12:58:18,994 Epoch  18 Step:    57200 Batch Loss:     1.624733 Tokens per Sec:    19207, Lr: 0.000300\n",
      "2020-01-02 12:58:30,779 Epoch  18 Step:    57300 Batch Loss:     1.619381 Tokens per Sec:    19348, Lr: 0.000300\n",
      "2020-01-02 12:58:42,513 Epoch  18 Step:    57400 Batch Loss:     1.470610 Tokens per Sec:    19131, Lr: 0.000300\n",
      "2020-01-02 12:58:54,379 Epoch  18 Step:    57500 Batch Loss:     1.755252 Tokens per Sec:    19431, Lr: 0.000300\n",
      "2020-01-02 12:59:06,316 Epoch  18 Step:    57600 Batch Loss:     1.702703 Tokens per Sec:    19571, Lr: 0.000300\n",
      "2020-01-02 12:59:18,231 Epoch  18 Step:    57700 Batch Loss:     1.189780 Tokens per Sec:    19804, Lr: 0.000300\n",
      "2020-01-02 12:59:30,122 Epoch  18 Step:    57800 Batch Loss:     1.589503 Tokens per Sec:    19393, Lr: 0.000300\n",
      "2020-01-02 12:59:42,064 Epoch  18 Step:    57900 Batch Loss:     1.454413 Tokens per Sec:    19598, Lr: 0.000300\n",
      "2020-01-02 12:59:53,785 Epoch  18 Step:    58000 Batch Loss:     1.859682 Tokens per Sec:    18993, Lr: 0.000300\n",
      "2020-01-02 13:00:23,144 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:00:23,144 Saving new checkpoint.\n",
      "2020-01-02 13:00:23,389 Example #0\n",
      "2020-01-02 13:00:23,390 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:00:23,390 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:00:23,390 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu muudi musue . ’\n",
      "2020-01-02 13:00:23,390 Example #1\n",
      "2020-01-02 13:00:23,390 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:00:23,390 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:00:23,391 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 13:00:23,391 Example #2\n",
      "2020-01-02 13:00:23,391 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:00:23,391 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:00:23,391 \tHypothesis: Nunku , Mose uvua mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 13:00:23,391 Example #3\n",
      "2020-01-02 13:00:23,391 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:00:23,391 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:00:23,391 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa badi bindila dibika dia bafue .\n",
      "2020-01-02 13:00:23,391 Validation result (greedy) at epoch  18, step    58000: bleu:  26.54, loss: 37648.6484, ppl:   4.2993, duration: 29.6059s\n",
      "2020-01-02 13:00:35,402 Epoch  18 Step:    58100 Batch Loss:     1.819024 Tokens per Sec:    19271, Lr: 0.000300\n",
      "2020-01-02 13:00:47,209 Epoch  18 Step:    58200 Batch Loss:     1.475009 Tokens per Sec:    19179, Lr: 0.000300\n",
      "2020-01-02 13:00:58,873 Epoch  18 Step:    58300 Batch Loss:     1.356800 Tokens per Sec:    18845, Lr: 0.000300\n",
      "2020-01-02 13:01:10,736 Epoch  18 Step:    58400 Batch Loss:     1.519096 Tokens per Sec:    19379, Lr: 0.000300\n",
      "2020-01-02 13:01:22,598 Epoch  18 Step:    58500 Batch Loss:     1.453383 Tokens per Sec:    19481, Lr: 0.000300\n",
      "2020-01-02 13:01:34,467 Epoch  18 Step:    58600 Batch Loss:     1.628056 Tokens per Sec:    19713, Lr: 0.000300\n",
      "2020-01-02 13:01:46,339 Epoch  18 Step:    58700 Batch Loss:     1.758116 Tokens per Sec:    19418, Lr: 0.000300\n",
      "2020-01-02 13:01:58,197 Epoch  18 Step:    58800 Batch Loss:     1.372709 Tokens per Sec:    19415, Lr: 0.000300\n",
      "2020-01-02 13:02:09,981 Epoch  18 Step:    58900 Batch Loss:     1.752121 Tokens per Sec:    19505, Lr: 0.000300\n",
      "2020-01-02 13:02:21,832 Epoch  18 Step:    59000 Batch Loss:     1.698119 Tokens per Sec:    19482, Lr: 0.000300\n",
      "2020-01-02 13:02:51,102 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:02:51,102 Saving new checkpoint.\n",
      "2020-01-02 13:02:51,425 Example #0\n",
      "2020-01-02 13:02:51,425 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:02:51,425 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:02:51,425 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanudiangatshi , kadi nuenze bu mudi disua dianyi . ’\n",
      "2020-01-02 13:02:51,425 Example #1\n",
      "2020-01-02 13:02:51,425 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:02:51,425 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:02:51,425 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamusaka bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 13:02:51,425 Example #2\n",
      "2020-01-02 13:02:51,426 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:02:51,426 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:02:51,426 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:02:51,428 Example #3\n",
      "2020-01-02 13:02:51,428 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:02:51,428 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:02:51,428 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshiindila dibika dia bafue .\n",
      "2020-01-02 13:02:51,428 Validation result (greedy) at epoch  18, step    59000: bleu:  26.86, loss: 37401.4375, ppl:   4.2584, duration: 29.5963s\n",
      "2020-01-02 13:03:03,302 Epoch  18 Step:    59100 Batch Loss:     1.762527 Tokens per Sec:    19403, Lr: 0.000300\n",
      "2020-01-02 13:03:14,369 Epoch  18: total training loss 5192.62\n",
      "2020-01-02 13:03:14,369 EPOCH 19\n",
      "2020-01-02 13:03:15,583 Epoch  19 Step:    59200 Batch Loss:     1.543702 Tokens per Sec:    13986, Lr: 0.000300\n",
      "2020-01-02 13:03:27,425 Epoch  19 Step:    59300 Batch Loss:     1.543680 Tokens per Sec:    19721, Lr: 0.000300\n",
      "2020-01-02 13:03:39,489 Epoch  19 Step:    59400 Batch Loss:     1.841722 Tokens per Sec:    19069, Lr: 0.000300\n",
      "2020-01-02 13:03:51,435 Epoch  19 Step:    59500 Batch Loss:     1.698679 Tokens per Sec:    19535, Lr: 0.000300\n",
      "2020-01-02 13:04:03,404 Epoch  19 Step:    59600 Batch Loss:     1.505423 Tokens per Sec:    18899, Lr: 0.000300\n",
      "2020-01-02 13:04:15,389 Epoch  19 Step:    59700 Batch Loss:     1.518404 Tokens per Sec:    19275, Lr: 0.000300\n",
      "2020-01-02 13:04:27,478 Epoch  19 Step:    59800 Batch Loss:     1.740842 Tokens per Sec:    19392, Lr: 0.000300\n",
      "2020-01-02 13:04:39,414 Epoch  19 Step:    59900 Batch Loss:     1.678614 Tokens per Sec:    19121, Lr: 0.000300\n",
      "2020-01-02 13:04:51,246 Epoch  19 Step:    60000 Batch Loss:     1.544918 Tokens per Sec:    19020, Lr: 0.000300\n",
      "2020-01-02 13:05:20,578 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:05:20,578 Saving new checkpoint.\n",
      "2020-01-02 13:05:20,901 Example #0\n",
      "2020-01-02 13:05:20,901 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:05:20,901 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:05:20,901 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusanki bua disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 13:05:20,901 Example #1\n",
      "2020-01-02 13:05:20,902 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:05:20,902 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:05:20,902 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutumikila .\n",
      "2020-01-02 13:05:20,902 Example #2\n",
      "2020-01-02 13:05:20,902 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:05:20,902 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:05:20,902 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena mua kupangila tshiji to .\n",
      "2020-01-02 13:05:20,902 Example #3\n",
      "2020-01-02 13:05:20,902 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:05:20,902 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:05:20,902 \tHypothesis: Ndi mutuishibue ne : Yehowa mmuvuluke dibika dia bafue .\n",
      "2020-01-02 13:05:20,902 Validation result (greedy) at epoch  19, step    60000: bleu:  27.16, loss: 37386.1719, ppl:   4.2558, duration: 29.6561s\n",
      "2020-01-02 13:05:32,710 Epoch  19 Step:    60100 Batch Loss:     1.611858 Tokens per Sec:    18969, Lr: 0.000300\n",
      "2020-01-02 13:05:44,348 Epoch  19 Step:    60200 Batch Loss:     1.952329 Tokens per Sec:    19065, Lr: 0.000300\n",
      "2020-01-02 13:05:56,294 Epoch  19 Step:    60300 Batch Loss:     1.802263 Tokens per Sec:    19595, Lr: 0.000300\n",
      "2020-01-02 13:06:08,131 Epoch  19 Step:    60400 Batch Loss:     1.608970 Tokens per Sec:    19365, Lr: 0.000300\n",
      "2020-01-02 13:06:19,861 Epoch  19 Step:    60500 Batch Loss:     1.458424 Tokens per Sec:    19327, Lr: 0.000300\n",
      "2020-01-02 13:06:31,699 Epoch  19 Step:    60600 Batch Loss:     2.154947 Tokens per Sec:    19360, Lr: 0.000300\n",
      "2020-01-02 13:06:43,555 Epoch  19 Step:    60700 Batch Loss:     1.345103 Tokens per Sec:    19306, Lr: 0.000300\n",
      "2020-01-02 13:06:55,344 Epoch  19 Step:    60800 Batch Loss:     1.325154 Tokens per Sec:    19361, Lr: 0.000300\n",
      "2020-01-02 13:07:07,112 Epoch  19 Step:    60900 Batch Loss:     1.686378 Tokens per Sec:    19553, Lr: 0.000300\n",
      "2020-01-02 13:07:18,726 Epoch  19 Step:    61000 Batch Loss:     1.342599 Tokens per Sec:    19026, Lr: 0.000300\n",
      "2020-01-02 13:07:47,945 Example #0\n",
      "2020-01-02 13:07:47,945 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:07:47,945 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:07:47,945 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu muwakuenza . ’\n",
      "2020-01-02 13:07:47,946 Example #1\n",
      "2020-01-02 13:07:47,946 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:07:47,946 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:07:47,946 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutuma .\n",
      "2020-01-02 13:07:47,946 Example #2\n",
      "2020-01-02 13:07:47,946 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:07:47,946 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:07:47,946 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji to .\n",
      "2020-01-02 13:07:47,946 Example #3\n",
      "2020-01-02 13:07:47,946 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:07:47,946 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:07:47,947 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:07:47,947 Validation result (greedy) at epoch  19, step    61000: bleu:  26.75, loss: 37529.0938, ppl:   4.2795, duration: 29.2206s\n",
      "2020-01-02 13:07:59,665 Epoch  19 Step:    61100 Batch Loss:     1.767878 Tokens per Sec:    19395, Lr: 0.000300\n",
      "2020-01-02 13:08:11,555 Epoch  19 Step:    61200 Batch Loss:     1.703417 Tokens per Sec:    19587, Lr: 0.000300\n",
      "2020-01-02 13:08:23,229 Epoch  19 Step:    61300 Batch Loss:     1.314945 Tokens per Sec:    19215, Lr: 0.000300\n",
      "2020-01-02 13:08:35,096 Epoch  19 Step:    61400 Batch Loss:     1.426723 Tokens per Sec:    19025, Lr: 0.000300\n",
      "2020-01-02 13:08:46,927 Epoch  19 Step:    61500 Batch Loss:     1.533614 Tokens per Sec:    19403, Lr: 0.000300\n",
      "2020-01-02 13:08:58,687 Epoch  19 Step:    61600 Batch Loss:     1.472436 Tokens per Sec:    19129, Lr: 0.000300\n",
      "2020-01-02 13:09:10,558 Epoch  19 Step:    61700 Batch Loss:     1.541951 Tokens per Sec:    19155, Lr: 0.000300\n",
      "2020-01-02 13:09:22,196 Epoch  19 Step:    61800 Batch Loss:     1.289973 Tokens per Sec:    19085, Lr: 0.000300\n",
      "2020-01-02 13:09:34,113 Epoch  19 Step:    61900 Batch Loss:     1.535743 Tokens per Sec:    19360, Lr: 0.000300\n",
      "2020-01-02 13:09:45,887 Epoch  19 Step:    62000 Batch Loss:     1.419370 Tokens per Sec:    19269, Lr: 0.000300\n",
      "2020-01-02 13:10:15,199 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:10:15,199 Saving new checkpoint.\n",
      "2020-01-02 13:10:15,524 Example #0\n",
      "2020-01-02 13:10:15,524 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:10:15,524 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:10:15,524 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bu mudi disua dianyi , kadi wikale bu mundi musue . ’\n",
      "2020-01-02 13:10:15,524 Example #1\n",
      "2020-01-02 13:10:15,524 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:10:15,524 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:10:15,524 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 13:10:15,525 Example #2\n",
      "2020-01-02 13:10:15,525 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:10:15,525 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:10:15,525 \tHypothesis: Nunku , Yehowa uvua ne bua kumvua bimpe njila yende , kadi uvua uleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:10:15,525 Example #3\n",
      "2020-01-02 13:10:15,525 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:10:15,525 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:10:15,525 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 13:10:15,525 Validation result (greedy) at epoch  19, step    62000: bleu:  27.29, loss: 37354.7188, ppl:   4.2507, duration: 29.6375s\n",
      "2020-01-02 13:10:27,307 Epoch  19 Step:    62100 Batch Loss:     1.599464 Tokens per Sec:    19476, Lr: 0.000300\n",
      "2020-01-02 13:10:39,142 Epoch  19 Step:    62200 Batch Loss:     1.621922 Tokens per Sec:    19407, Lr: 0.000300\n",
      "2020-01-02 13:10:50,919 Epoch  19 Step:    62300 Batch Loss:     1.706436 Tokens per Sec:    19428, Lr: 0.000300\n",
      "2020-01-02 13:11:02,844 Epoch  19 Step:    62400 Batch Loss:     1.902482 Tokens per Sec:    19399, Lr: 0.000300\n",
      "2020-01-02 13:11:13,220 Epoch  19: total training loss 5155.97\n",
      "2020-01-02 13:11:13,221 EPOCH 20\n",
      "2020-01-02 13:11:14,998 Epoch  20 Step:    62500 Batch Loss:     1.497223 Tokens per Sec:    14757, Lr: 0.000300\n",
      "2020-01-02 13:11:26,954 Epoch  20 Step:    62600 Batch Loss:     1.353938 Tokens per Sec:    19582, Lr: 0.000300\n",
      "2020-01-02 13:11:38,776 Epoch  20 Step:    62700 Batch Loss:     1.577020 Tokens per Sec:    19586, Lr: 0.000300\n",
      "2020-01-02 13:11:50,538 Epoch  20 Step:    62800 Batch Loss:     1.520546 Tokens per Sec:    19041, Lr: 0.000300\n",
      "2020-01-02 13:12:02,446 Epoch  20 Step:    62900 Batch Loss:     1.368358 Tokens per Sec:    19599, Lr: 0.000300\n",
      "2020-01-02 13:12:14,291 Epoch  20 Step:    63000 Batch Loss:     1.346717 Tokens per Sec:    19734, Lr: 0.000300\n",
      "2020-01-02 13:12:43,693 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:12:43,693 Saving new checkpoint.\n",
      "2020-01-02 13:12:43,983 Example #0\n",
      "2020-01-02 13:12:43,983 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:12:43,983 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:12:43,984 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu mudidi dienzeka . ’\n",
      "2020-01-02 13:12:43,984 Example #1\n",
      "2020-01-02 13:12:43,984 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:12:43,984 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:12:43,984 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila ku mikenji ya mbulamatadi .\n",
      "2020-01-02 13:12:43,984 Example #2\n",
      "2020-01-02 13:12:43,984 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:12:43,984 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:12:43,984 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena mua kumvua tshiji tshiende to .\n",
      "2020-01-02 13:12:43,984 Example #3\n",
      "2020-01-02 13:12:43,985 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:12:43,985 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:12:43,985 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa mbindile dibishibua dia bafue .\n",
      "2020-01-02 13:12:43,985 Validation result (greedy) at epoch  20, step    63000: bleu:  27.26, loss: 37281.2266, ppl:   4.2386, duration: 29.6932s\n",
      "2020-01-02 13:12:55,808 Epoch  20 Step:    63100 Batch Loss:     1.564461 Tokens per Sec:    18972, Lr: 0.000300\n",
      "2020-01-02 13:13:07,551 Epoch  20 Step:    63200 Batch Loss:     1.508813 Tokens per Sec:    19255, Lr: 0.000300\n",
      "2020-01-02 13:13:19,411 Epoch  20 Step:    63300 Batch Loss:     1.724506 Tokens per Sec:    19613, Lr: 0.000300\n",
      "2020-01-02 13:13:31,246 Epoch  20 Step:    63400 Batch Loss:     1.531814 Tokens per Sec:    19375, Lr: 0.000300\n",
      "2020-01-02 13:13:43,153 Epoch  20 Step:    63500 Batch Loss:     1.540648 Tokens per Sec:    19758, Lr: 0.000300\n",
      "2020-01-02 13:13:55,055 Epoch  20 Step:    63600 Batch Loss:     1.639489 Tokens per Sec:    19235, Lr: 0.000300\n",
      "2020-01-02 13:14:06,989 Epoch  20 Step:    63700 Batch Loss:     1.800246 Tokens per Sec:    19733, Lr: 0.000300\n",
      "2020-01-02 13:14:18,805 Epoch  20 Step:    63800 Batch Loss:     1.401865 Tokens per Sec:    19264, Lr: 0.000300\n",
      "2020-01-02 13:14:30,758 Epoch  20 Step:    63900 Batch Loss:     1.534774 Tokens per Sec:    19491, Lr: 0.000300\n",
      "2020-01-02 13:14:42,509 Epoch  20 Step:    64000 Batch Loss:     1.115256 Tokens per Sec:    19246, Lr: 0.000300\n",
      "2020-01-02 13:15:11,868 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:15:11,868 Saving new checkpoint.\n",
      "2020-01-02 13:15:12,154 Example #0\n",
      "2020-01-02 13:15:12,155 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:15:12,155 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:15:12,155 \tHypothesis: YEZU mmutushile tshilejilu tshimpe bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 13:15:12,155 Example #1\n",
      "2020-01-02 13:15:12,155 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:15:12,155 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:15:12,155 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudiye .\n",
      "2020-01-02 13:15:12,155 Example #2\n",
      "2020-01-02 13:15:12,155 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:15:12,155 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:15:12,155 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ne tshiji to .\n",
      "2020-01-02 13:15:12,155 Example #3\n",
      "2020-01-02 13:15:12,155 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:15:12,155 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:15:12,155 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:15:12,155 Validation result (greedy) at epoch  20, step    64000: bleu:  27.58, loss: 37131.8672, ppl:   4.2141, duration: 29.6466s\n",
      "2020-01-02 13:15:24,028 Epoch  20 Step:    64100 Batch Loss:     1.551497 Tokens per Sec:    19083, Lr: 0.000300\n",
      "2020-01-02 13:15:35,857 Epoch  20 Step:    64200 Batch Loss:     1.527945 Tokens per Sec:    18859, Lr: 0.000300\n",
      "2020-01-02 13:15:47,746 Epoch  20 Step:    64300 Batch Loss:     1.637133 Tokens per Sec:    19703, Lr: 0.000300\n",
      "2020-01-02 13:15:59,492 Epoch  20 Step:    64400 Batch Loss:     1.739353 Tokens per Sec:    19187, Lr: 0.000300\n",
      "2020-01-02 13:16:11,379 Epoch  20 Step:    64500 Batch Loss:     1.463230 Tokens per Sec:    18728, Lr: 0.000300\n",
      "2020-01-02 13:16:23,254 Epoch  20 Step:    64600 Batch Loss:     1.673693 Tokens per Sec:    19504, Lr: 0.000300\n",
      "2020-01-02 13:16:35,196 Epoch  20 Step:    64700 Batch Loss:     1.496586 Tokens per Sec:    18894, Lr: 0.000300\n",
      "2020-01-02 13:16:47,062 Epoch  20 Step:    64800 Batch Loss:     1.642408 Tokens per Sec:    19397, Lr: 0.000300\n",
      "2020-01-02 13:16:58,926 Epoch  20 Step:    64900 Batch Loss:     1.480984 Tokens per Sec:    19226, Lr: 0.000300\n",
      "2020-01-02 13:17:10,793 Epoch  20 Step:    65000 Batch Loss:     1.688468 Tokens per Sec:    19419, Lr: 0.000300\n",
      "2020-01-02 13:17:40,219 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:17:40,219 Saving new checkpoint.\n",
      "2020-01-02 13:17:40,530 Example #0\n",
      "2020-01-02 13:17:40,530 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:17:40,530 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:17:40,530 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusangishi disua dianyi , kadi wenze . ’\n",
      "2020-01-02 13:17:40,530 Example #1\n",
      "2020-01-02 13:17:40,530 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:17:40,530 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:17:40,531 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufume kudiye .\n",
      "2020-01-02 13:17:40,531 Example #2\n",
      "2020-01-02 13:17:40,531 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:17:40,531 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:17:40,531 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:17:40,531 Example #3\n",
      "2020-01-02 13:17:40,531 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:17:40,531 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:17:40,531 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:17:40,531 Validation result (greedy) at epoch  20, step    65000: bleu:  27.33, loss: 36952.8789, ppl:   4.1850, duration: 29.7385s\n",
      "2020-01-02 13:17:52,371 Epoch  20 Step:    65100 Batch Loss:     1.560413 Tokens per Sec:    19070, Lr: 0.000300\n",
      "2020-01-02 13:18:04,343 Epoch  20 Step:    65200 Batch Loss:     1.800903 Tokens per Sec:    19488, Lr: 0.000300\n",
      "2020-01-02 13:18:16,183 Epoch  20 Step:    65300 Batch Loss:     1.578141 Tokens per Sec:    19353, Lr: 0.000300\n",
      "2020-01-02 13:18:28,049 Epoch  20 Step:    65400 Batch Loss:     1.730287 Tokens per Sec:    19013, Lr: 0.000300\n",
      "2020-01-02 13:18:39,906 Epoch  20 Step:    65500 Batch Loss:     1.656110 Tokens per Sec:    19313, Lr: 0.000300\n",
      "2020-01-02 13:18:51,688 Epoch  20 Step:    65600 Batch Loss:     1.601561 Tokens per Sec:    19412, Lr: 0.000300\n",
      "2020-01-02 13:19:03,440 Epoch  20 Step:    65700 Batch Loss:     1.653548 Tokens per Sec:    19041, Lr: 0.000300\n",
      "2020-01-02 13:19:12,214 Epoch  20: total training loss 5090.56\n",
      "2020-01-02 13:19:12,214 EPOCH 21\n",
      "2020-01-02 13:19:15,673 Epoch  21 Step:    65800 Batch Loss:     1.526909 Tokens per Sec:    17915, Lr: 0.000300\n",
      "2020-01-02 13:19:27,540 Epoch  21 Step:    65900 Batch Loss:     1.510631 Tokens per Sec:    19657, Lr: 0.000300\n",
      "2020-01-02 13:19:39,429 Epoch  21 Step:    66000 Batch Loss:     1.289038 Tokens per Sec:    18990, Lr: 0.000300\n",
      "2020-01-02 13:20:08,813 Example #0\n",
      "2020-01-02 13:20:08,813 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:20:08,813 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:20:08,813 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:20:08,813 Example #1\n",
      "2020-01-02 13:20:08,813 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:20:08,813 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:20:08,813 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukina , bakamuenzeja bua kumbuka mu buloba buvua bufume kudi mbulamatadi .\n",
      "2020-01-02 13:20:08,813 Example #2\n",
      "2020-01-02 13:20:08,814 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:20:08,814 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:20:08,814 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 13:20:08,814 Example #3\n",
      "2020-01-02 13:20:08,814 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:20:08,814 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:20:08,814 \tHypothesis: Ndi mutuishibue ne : Yehowa mmuvuluke dibika dia bafue .\n",
      "2020-01-02 13:20:08,814 Validation result (greedy) at epoch  21, step    66000: bleu:  27.15, loss: 37028.4141, ppl:   4.1973, duration: 29.3843s\n",
      "2020-01-02 13:20:20,814 Epoch  21 Step:    66100 Batch Loss:     1.594489 Tokens per Sec:    19555, Lr: 0.000300\n",
      "2020-01-02 13:20:32,650 Epoch  21 Step:    66200 Batch Loss:     1.633169 Tokens per Sec:    19317, Lr: 0.000300\n",
      "2020-01-02 13:20:44,648 Epoch  21 Step:    66300 Batch Loss:     1.717528 Tokens per Sec:    19224, Lr: 0.000300\n",
      "2020-01-02 13:20:56,397 Epoch  21 Step:    66400 Batch Loss:     1.602124 Tokens per Sec:    18778, Lr: 0.000300\n",
      "2020-01-02 13:21:08,251 Epoch  21 Step:    66500 Batch Loss:     1.513870 Tokens per Sec:    19253, Lr: 0.000300\n",
      "2020-01-02 13:21:19,981 Epoch  21 Step:    66600 Batch Loss:     1.471517 Tokens per Sec:    19055, Lr: 0.000300\n",
      "2020-01-02 13:21:31,827 Epoch  21 Step:    66700 Batch Loss:     1.400528 Tokens per Sec:    19477, Lr: 0.000300\n",
      "2020-01-02 13:21:43,720 Epoch  21 Step:    66800 Batch Loss:     1.361728 Tokens per Sec:    19481, Lr: 0.000300\n",
      "2020-01-02 13:21:55,609 Epoch  21 Step:    66900 Batch Loss:     1.563208 Tokens per Sec:    18877, Lr: 0.000300\n",
      "2020-01-02 13:22:07,445 Epoch  21 Step:    67000 Batch Loss:     1.543682 Tokens per Sec:    19503, Lr: 0.000300\n",
      "2020-01-02 13:22:36,822 Example #0\n",
      "2020-01-02 13:22:36,822 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:22:36,822 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:22:36,822 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengelayi disua dianyi , kadi nuenze bu mudi disua dianyi . ’\n",
      "2020-01-02 13:22:36,822 Example #1\n",
      "2020-01-02 13:22:36,822 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:22:36,822 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:22:36,822 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufume kudiye .\n",
      "2020-01-02 13:22:36,822 Example #2\n",
      "2020-01-02 13:22:36,822 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:22:36,823 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:22:36,823 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:22:36,823 Example #3\n",
      "2020-01-02 13:22:36,823 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:22:36,823 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:22:36,823 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshivuluka dibika dia bafue .\n",
      "2020-01-02 13:22:36,823 Validation result (greedy) at epoch  21, step    67000: bleu:  27.10, loss: 36970.8789, ppl:   4.1879, duration: 29.3780s\n",
      "2020-01-02 13:22:48,690 Epoch  21 Step:    67100 Batch Loss:     1.498986 Tokens per Sec:    19432, Lr: 0.000300\n",
      "2020-01-02 13:23:00,440 Epoch  21 Step:    67200 Batch Loss:     1.871021 Tokens per Sec:    19342, Lr: 0.000300\n",
      "2020-01-02 13:23:12,334 Epoch  21 Step:    67300 Batch Loss:     1.663087 Tokens per Sec:    19396, Lr: 0.000300\n",
      "2020-01-02 13:23:24,200 Epoch  21 Step:    67400 Batch Loss:     1.420169 Tokens per Sec:    19139, Lr: 0.000300\n",
      "2020-01-02 13:23:36,174 Epoch  21 Step:    67500 Batch Loss:     1.296027 Tokens per Sec:    19481, Lr: 0.000300\n",
      "2020-01-02 13:23:47,931 Epoch  21 Step:    67600 Batch Loss:     1.479290 Tokens per Sec:    19506, Lr: 0.000300\n",
      "2020-01-02 13:23:59,716 Epoch  21 Step:    67700 Batch Loss:     1.610924 Tokens per Sec:    18687, Lr: 0.000300\n",
      "2020-01-02 13:24:11,573 Epoch  21 Step:    67800 Batch Loss:     1.562082 Tokens per Sec:    19404, Lr: 0.000300\n",
      "2020-01-02 13:24:23,335 Epoch  21 Step:    67900 Batch Loss:     1.584745 Tokens per Sec:    18884, Lr: 0.000300\n",
      "2020-01-02 13:24:35,152 Epoch  21 Step:    68000 Batch Loss:     1.209223 Tokens per Sec:    19012, Lr: 0.000300\n",
      "2020-01-02 13:25:04,575 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:25:04,575 Saving new checkpoint.\n",
      "2020-01-02 13:25:04,842 Example #0\n",
      "2020-01-02 13:25:04,843 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:25:04,843 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:25:04,843 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusambilayi disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 13:25:04,843 Example #1\n",
      "2020-01-02 13:25:04,843 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:25:04,843 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:25:04,843 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumina ku mikenji ya mbulamatadi .\n",
      "2020-01-02 13:25:04,843 Example #2\n",
      "2020-01-02 13:25:04,843 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:25:04,843 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:25:04,843 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 13:25:04,843 Example #3\n",
      "2020-01-02 13:25:04,844 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:25:04,844 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:25:04,844 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshiindila dibika dia bafue .\n",
      "2020-01-02 13:25:04,844 Validation result (greedy) at epoch  21, step    68000: bleu:  27.14, loss: 36757.1016, ppl:   4.1534, duration: 29.6917s\n",
      "2020-01-02 13:25:16,656 Epoch  21 Step:    68100 Batch Loss:     1.467605 Tokens per Sec:    19086, Lr: 0.000300\n",
      "2020-01-02 13:25:28,689 Epoch  21 Step:    68200 Batch Loss:     1.743668 Tokens per Sec:    19020, Lr: 0.000300\n",
      "2020-01-02 13:25:40,527 Epoch  21 Step:    68300 Batch Loss:     1.488257 Tokens per Sec:    19034, Lr: 0.000300\n",
      "2020-01-02 13:25:52,401 Epoch  21 Step:    68400 Batch Loss:     1.355988 Tokens per Sec:    19239, Lr: 0.000300\n",
      "2020-01-02 13:26:04,367 Epoch  21 Step:    68500 Batch Loss:     1.577096 Tokens per Sec:    19206, Lr: 0.000300\n",
      "2020-01-02 13:26:16,138 Epoch  21 Step:    68600 Batch Loss:     1.479814 Tokens per Sec:    19432, Lr: 0.000300\n",
      "2020-01-02 13:26:27,959 Epoch  21 Step:    68700 Batch Loss:     1.508405 Tokens per Sec:    18900, Lr: 0.000300\n",
      "2020-01-02 13:26:39,839 Epoch  21 Step:    68800 Batch Loss:     1.736632 Tokens per Sec:    19333, Lr: 0.000300\n",
      "2020-01-02 13:26:51,644 Epoch  21 Step:    68900 Batch Loss:     1.496779 Tokens per Sec:    19089, Lr: 0.000300\n",
      "2020-01-02 13:27:03,520 Epoch  21 Step:    69000 Batch Loss:     1.493901 Tokens per Sec:    19421, Lr: 0.000300\n",
      "2020-01-02 13:27:32,930 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:27:32,930 Saving new checkpoint.\n",
      "2020-01-02 13:27:33,221 Example #0\n",
      "2020-01-02 13:27:33,221 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:27:33,221 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:27:33,221 \tHypothesis: YEZU mmutushile tshilejilu tshia bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanusangishi disua dianyi , kadi nuenu nuenze bu mudi disua dianyi . ’\n",
      "2020-01-02 13:27:33,221 Example #1\n",
      "2020-01-02 13:27:33,221 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:27:33,221 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:27:33,221 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 13:27:33,221 Example #2\n",
      "2020-01-02 13:27:33,221 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:27:33,221 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:27:33,222 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:27:33,222 Example #3\n",
      "2020-01-02 13:27:33,222 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:27:33,222 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:27:33,222 \tHypothesis: Ndi mujadike ne : badi mu tshivulukilu tshia Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 13:27:33,222 Validation result (greedy) at epoch  21, step    69000: bleu:  27.89, loss: 36698.9258, ppl:   4.1440, duration: 29.7019s\n",
      "2020-01-02 13:27:42,035 Epoch  21: total training loss 5080.00\n",
      "2020-01-02 13:27:42,036 EPOCH 22\n",
      "2020-01-02 13:27:45,473 Epoch  22 Step:    69100 Batch Loss:     1.620187 Tokens per Sec:    16489, Lr: 0.000300\n",
      "2020-01-02 13:27:57,388 Epoch  22 Step:    69200 Batch Loss:     1.398827 Tokens per Sec:    19356, Lr: 0.000300\n",
      "2020-01-02 13:28:09,174 Epoch  22 Step:    69300 Batch Loss:     1.537304 Tokens per Sec:    19401, Lr: 0.000300\n",
      "2020-01-02 13:28:20,995 Epoch  22 Step:    69400 Batch Loss:     1.480311 Tokens per Sec:    19344, Lr: 0.000300\n",
      "2020-01-02 13:28:32,763 Epoch  22 Step:    69500 Batch Loss:     1.238741 Tokens per Sec:    19050, Lr: 0.000300\n",
      "2020-01-02 13:28:44,645 Epoch  22 Step:    69600 Batch Loss:     1.401859 Tokens per Sec:    19666, Lr: 0.000300\n",
      "2020-01-02 13:28:56,488 Epoch  22 Step:    69700 Batch Loss:     1.935161 Tokens per Sec:    19581, Lr: 0.000300\n",
      "2020-01-02 13:29:08,269 Epoch  22 Step:    69800 Batch Loss:     1.675744 Tokens per Sec:    19069, Lr: 0.000300\n",
      "2020-01-02 13:29:20,058 Epoch  22 Step:    69900 Batch Loss:     1.575214 Tokens per Sec:    19899, Lr: 0.000300\n",
      "2020-01-02 13:29:31,840 Epoch  22 Step:    70000 Batch Loss:     1.494822 Tokens per Sec:    19323, Lr: 0.000300\n",
      "2020-01-02 13:30:01,106 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:30:01,106 Saving new checkpoint.\n",
      "2020-01-02 13:30:01,373 Example #0\n",
      "2020-01-02 13:30:01,374 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:30:01,374 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:30:01,374 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:30:01,374 Example #1\n",
      "2020-01-02 13:30:01,374 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:30:01,374 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:30:01,374 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumina ku dibenga kumushipa .\n",
      "2020-01-02 13:30:01,374 Example #2\n",
      "2020-01-02 13:30:01,374 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:30:01,374 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:30:01,374 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 13:30:01,374 Example #3\n",
      "2020-01-02 13:30:01,374 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:30:01,374 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:30:01,374 \tHypothesis: Ndi mushindike ne : Yehowa mmuvuluke dibika dia bafue .\n",
      "2020-01-02 13:30:01,375 Validation result (greedy) at epoch  22, step    70000: bleu:  27.75, loss: 36668.2109, ppl:   4.1391, duration: 29.5343s\n",
      "2020-01-02 13:30:13,114 Epoch  22 Step:    70100 Batch Loss:     1.562760 Tokens per Sec:    19389, Lr: 0.000300\n",
      "2020-01-02 13:30:24,803 Epoch  22 Step:    70200 Batch Loss:     1.675725 Tokens per Sec:    19409, Lr: 0.000300\n",
      "2020-01-02 13:30:36,716 Epoch  22 Step:    70300 Batch Loss:     1.518468 Tokens per Sec:    19625, Lr: 0.000300\n",
      "2020-01-02 13:30:48,443 Epoch  22 Step:    70400 Batch Loss:     1.671407 Tokens per Sec:    19572, Lr: 0.000300\n",
      "2020-01-02 13:31:00,105 Epoch  22 Step:    70500 Batch Loss:     1.367976 Tokens per Sec:    19608, Lr: 0.000300\n",
      "2020-01-02 13:31:11,848 Epoch  22 Step:    70600 Batch Loss:     1.713209 Tokens per Sec:    19584, Lr: 0.000300\n",
      "2020-01-02 13:31:23,545 Epoch  22 Step:    70700 Batch Loss:     1.621870 Tokens per Sec:    19551, Lr: 0.000300\n",
      "2020-01-02 13:31:35,269 Epoch  22 Step:    70800 Batch Loss:     1.429596 Tokens per Sec:    19521, Lr: 0.000300\n",
      "2020-01-02 13:31:46,910 Epoch  22 Step:    70900 Batch Loss:     1.144146 Tokens per Sec:    19324, Lr: 0.000300\n",
      "2020-01-02 13:31:58,646 Epoch  22 Step:    71000 Batch Loss:     1.552992 Tokens per Sec:    19836, Lr: 0.000300\n",
      "2020-01-02 13:32:27,819 Example #0\n",
      "2020-01-02 13:32:27,819 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:32:27,820 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:32:27,820 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji malu andi musue , kadi nuenze . ’\n",
      "2020-01-02 13:32:27,820 Example #1\n",
      "2020-01-02 13:32:27,820 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:32:27,820 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:32:27,820 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudiye .\n",
      "2020-01-02 13:32:27,820 Example #2\n",
      "2020-01-02 13:32:27,820 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:32:27,820 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:32:27,820 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 13:32:27,820 Example #3\n",
      "2020-01-02 13:32:27,820 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:32:27,820 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:32:27,820 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa badi bindila dibishibua .\n",
      "2020-01-02 13:32:27,821 Validation result (greedy) at epoch  22, step    71000: bleu:  27.43, loss: 36720.4766, ppl:   4.1475, duration: 29.1744s\n",
      "2020-01-02 13:32:39,463 Epoch  22 Step:    71100 Batch Loss:     1.290975 Tokens per Sec:    19167, Lr: 0.000300\n",
      "2020-01-02 13:32:51,115 Epoch  22 Step:    71200 Batch Loss:     1.208915 Tokens per Sec:    19608, Lr: 0.000300\n",
      "2020-01-02 13:33:02,883 Epoch  22 Step:    71300 Batch Loss:     1.705336 Tokens per Sec:    19786, Lr: 0.000300\n",
      "2020-01-02 13:33:14,497 Epoch  22 Step:    71400 Batch Loss:     1.824931 Tokens per Sec:    19697, Lr: 0.000300\n",
      "2020-01-02 13:33:26,171 Epoch  22 Step:    71500 Batch Loss:     1.882621 Tokens per Sec:    19353, Lr: 0.000300\n",
      "2020-01-02 13:33:37,892 Epoch  22 Step:    71600 Batch Loss:     1.660692 Tokens per Sec:    19892, Lr: 0.000300\n",
      "2020-01-02 13:33:49,548 Epoch  22 Step:    71700 Batch Loss:     1.600187 Tokens per Sec:    19866, Lr: 0.000300\n",
      "2020-01-02 13:34:01,269 Epoch  22 Step:    71800 Batch Loss:     1.441140 Tokens per Sec:    19192, Lr: 0.000300\n",
      "2020-01-02 13:34:12,814 Epoch  22 Step:    71900 Batch Loss:     1.493759 Tokens per Sec:    19525, Lr: 0.000300\n",
      "2020-01-02 13:34:24,535 Epoch  22 Step:    72000 Batch Loss:     1.499098 Tokens per Sec:    19860, Lr: 0.000300\n",
      "2020-01-02 13:34:53,662 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:34:53,662 Saving new checkpoint.\n",
      "2020-01-02 13:34:53,897 Example #0\n",
      "2020-01-02 13:34:53,897 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:34:53,898 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:34:53,898 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanusankishayi disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 13:34:53,898 Example #1\n",
      "2020-01-02 13:34:53,898 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:34:53,898 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:34:53,898 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushipa .\n",
      "2020-01-02 13:34:53,898 Example #2\n",
      "2020-01-02 13:34:53,898 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:34:53,898 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:34:53,898 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:34:53,898 Example #3\n",
      "2020-01-02 13:34:53,898 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:34:53,898 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:34:53,898 \tHypothesis: Ndi mujadike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:34:53,898 Validation result (greedy) at epoch  22, step    72000: bleu:  27.61, loss: 36541.9883, ppl:   4.1189, duration: 29.3631s\n",
      "2020-01-02 13:35:05,522 Epoch  22 Step:    72100 Batch Loss:     1.331421 Tokens per Sec:    19044, Lr: 0.000300\n",
      "2020-01-02 13:35:17,126 Epoch  22 Step:    72200 Batch Loss:     1.272124 Tokens per Sec:    19734, Lr: 0.000300\n",
      "2020-01-02 13:35:28,933 Epoch  22 Step:    72300 Batch Loss:     1.620844 Tokens per Sec:    19813, Lr: 0.000300\n",
      "2020-01-02 13:35:36,158 Epoch  22: total training loss 5018.41\n",
      "2020-01-02 13:35:36,158 EPOCH 23\n",
      "2020-01-02 13:35:40,982 Epoch  23 Step:    72400 Batch Loss:     1.533818 Tokens per Sec:    18014, Lr: 0.000300\n",
      "2020-01-02 13:35:52,702 Epoch  23 Step:    72500 Batch Loss:     1.569990 Tokens per Sec:    19555, Lr: 0.000300\n",
      "2020-01-02 13:36:04,350 Epoch  23 Step:    72600 Batch Loss:     1.418628 Tokens per Sec:    19674, Lr: 0.000300\n",
      "2020-01-02 13:36:15,907 Epoch  23 Step:    72700 Batch Loss:     1.591354 Tokens per Sec:    19429, Lr: 0.000300\n",
      "2020-01-02 13:36:27,635 Epoch  23 Step:    72800 Batch Loss:     1.549009 Tokens per Sec:    19635, Lr: 0.000300\n",
      "2020-01-02 13:36:39,370 Epoch  23 Step:    72900 Batch Loss:     2.442791 Tokens per Sec:    19141, Lr: 0.000300\n",
      "2020-01-02 13:36:51,040 Epoch  23 Step:    73000 Batch Loss:     1.421678 Tokens per Sec:    19750, Lr: 0.000300\n",
      "2020-01-02 13:37:20,188 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:37:20,188 Saving new checkpoint.\n",
      "2020-01-02 13:37:20,433 Example #0\n",
      "2020-01-02 13:37:20,433 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:37:20,433 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:37:20,433 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 13:37:20,433 Example #1\n",
      "2020-01-02 13:37:20,433 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:37:20,433 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:37:20,433 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine ku mikenji ya Nzambi .\n",
      "2020-01-02 13:37:20,433 Example #2\n",
      "2020-01-02 13:37:20,433 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:37:20,434 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:37:20,434 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu .\n",
      "2020-01-02 13:37:20,434 Example #3\n",
      "2020-01-02 13:37:20,434 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:37:20,434 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:37:20,434 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa tshidi tshiindile dibika dia bafue .\n",
      "2020-01-02 13:37:20,434 Validation result (greedy) at epoch  23, step    73000: bleu:  27.50, loss: 36489.2539, ppl:   4.1105, duration: 29.3939s\n",
      "2020-01-02 13:37:32,288 Epoch  23 Step:    73100 Batch Loss:     1.708114 Tokens per Sec:    19978, Lr: 0.000300\n",
      "2020-01-02 13:37:44,049 Epoch  23 Step:    73200 Batch Loss:     1.571562 Tokens per Sec:    20160, Lr: 0.000300\n",
      "2020-01-02 13:37:55,790 Epoch  23 Step:    73300 Batch Loss:     1.373129 Tokens per Sec:    19728, Lr: 0.000300\n",
      "2020-01-02 13:38:07,503 Epoch  23 Step:    73400 Batch Loss:     1.550979 Tokens per Sec:    19596, Lr: 0.000300\n",
      "2020-01-02 13:38:19,066 Epoch  23 Step:    73500 Batch Loss:     1.579326 Tokens per Sec:    19410, Lr: 0.000300\n",
      "2020-01-02 13:38:30,653 Epoch  23 Step:    73600 Batch Loss:     1.611576 Tokens per Sec:    19505, Lr: 0.000300\n",
      "2020-01-02 13:38:42,338 Epoch  23 Step:    73700 Batch Loss:     1.659977 Tokens per Sec:    19706, Lr: 0.000300\n",
      "2020-01-02 13:38:54,005 Epoch  23 Step:    73800 Batch Loss:     1.361121 Tokens per Sec:    19545, Lr: 0.000300\n",
      "2020-01-02 13:39:05,831 Epoch  23 Step:    73900 Batch Loss:     1.636284 Tokens per Sec:    19959, Lr: 0.000300\n",
      "2020-01-02 13:39:17,415 Epoch  23 Step:    74000 Batch Loss:     1.646727 Tokens per Sec:    19609, Lr: 0.000300\n",
      "2020-01-02 13:39:46,601 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:39:46,601 Saving new checkpoint.\n",
      "2020-01-02 13:39:46,841 Example #0\n",
      "2020-01-02 13:39:46,841 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:39:46,841 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:39:46,841 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengelayi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:39:46,841 Example #1\n",
      "2020-01-02 13:39:46,841 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:39:46,842 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:39:46,842 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 13:39:46,842 Example #2\n",
      "2020-01-02 13:39:46,842 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:39:46,842 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:39:46,842 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:39:46,842 Example #3\n",
      "2020-01-02 13:39:46,842 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:39:46,842 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:39:46,842 \tHypothesis: Ndi mujadike ne : badi mu tshivulukilu tshia Yehowa badi bindila dibika dia bafue .\n",
      "2020-01-02 13:39:46,842 Validation result (greedy) at epoch  23, step    74000: bleu:  27.69, loss: 36391.7500, ppl:   4.0950, duration: 29.4269s\n",
      "2020-01-02 13:39:58,547 Epoch  23 Step:    74100 Batch Loss:     1.442713 Tokens per Sec:    19846, Lr: 0.000300\n",
      "2020-01-02 13:40:10,117 Epoch  23 Step:    74200 Batch Loss:     1.461806 Tokens per Sec:    19175, Lr: 0.000300\n",
      "2020-01-02 13:40:21,731 Epoch  23 Step:    74300 Batch Loss:     1.408800 Tokens per Sec:    19541, Lr: 0.000300\n",
      "2020-01-02 13:40:33,418 Epoch  23 Step:    74400 Batch Loss:     1.309273 Tokens per Sec:    19911, Lr: 0.000300\n",
      "2020-01-02 13:40:45,149 Epoch  23 Step:    74500 Batch Loss:     1.552029 Tokens per Sec:    19615, Lr: 0.000300\n",
      "2020-01-02 13:40:56,867 Epoch  23 Step:    74600 Batch Loss:     1.632233 Tokens per Sec:    19517, Lr: 0.000300\n",
      "2020-01-02 13:41:08,667 Epoch  23 Step:    74700 Batch Loss:     1.708084 Tokens per Sec:    19737, Lr: 0.000300\n",
      "2020-01-02 13:41:20,138 Epoch  23 Step:    74800 Batch Loss:     1.645302 Tokens per Sec:    19037, Lr: 0.000300\n",
      "2020-01-02 13:41:31,839 Epoch  23 Step:    74900 Batch Loss:     1.664963 Tokens per Sec:    19529, Lr: 0.000300\n",
      "2020-01-02 13:41:43,540 Epoch  23 Step:    75000 Batch Loss:     1.536532 Tokens per Sec:    20030, Lr: 0.000300\n",
      "2020-01-02 13:42:12,675 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:42:12,676 Saving new checkpoint.\n",
      "2020-01-02 13:42:12,932 Example #0\n",
      "2020-01-02 13:42:12,932 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:42:12,932 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:42:12,932 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusangishi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:42:12,932 Example #1\n",
      "2020-01-02 13:42:12,933 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:42:12,933 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:42:12,933 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila .\n",
      "2020-01-02 13:42:12,933 Example #2\n",
      "2020-01-02 13:42:12,933 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:42:12,933 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:42:12,933 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji to .\n",
      "2020-01-02 13:42:12,933 Example #3\n",
      "2020-01-02 13:42:12,933 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:42:12,933 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:42:12,934 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:42:12,934 Validation result (greedy) at epoch  23, step    75000: bleu:  27.54, loss: 36205.1484, ppl:   4.0655, duration: 29.3939s\n",
      "2020-01-02 13:42:24,669 Epoch  23 Step:    75100 Batch Loss:     1.571798 Tokens per Sec:    19476, Lr: 0.000300\n",
      "2020-01-02 13:42:36,307 Epoch  23 Step:    75200 Batch Loss:     1.752954 Tokens per Sec:    19635, Lr: 0.000300\n",
      "2020-01-02 13:42:47,877 Epoch  23 Step:    75300 Batch Loss:     1.347679 Tokens per Sec:    19524, Lr: 0.000300\n",
      "2020-01-02 13:42:59,531 Epoch  23 Step:    75400 Batch Loss:     1.047841 Tokens per Sec:    19480, Lr: 0.000300\n",
      "2020-01-02 13:43:11,124 Epoch  23 Step:    75500 Batch Loss:     1.600799 Tokens per Sec:    19914, Lr: 0.000300\n",
      "2020-01-02 13:43:22,844 Epoch  23 Step:    75600 Batch Loss:     1.523753 Tokens per Sec:    19543, Lr: 0.000300\n",
      "2020-01-02 13:43:28,341 Epoch  23: total training loss 4979.32\n",
      "2020-01-02 13:43:28,341 EPOCH 24\n",
      "2020-01-02 13:43:34,920 Epoch  24 Step:    75700 Batch Loss:     1.625068 Tokens per Sec:    18604, Lr: 0.000300\n",
      "2020-01-02 13:43:46,674 Epoch  24 Step:    75800 Batch Loss:     1.325946 Tokens per Sec:    19856, Lr: 0.000300\n",
      "2020-01-02 13:43:58,099 Epoch  24 Step:    75900 Batch Loss:     1.488673 Tokens per Sec:    19607, Lr: 0.000300\n",
      "2020-01-02 13:44:09,696 Epoch  24 Step:    76000 Batch Loss:     1.704807 Tokens per Sec:    19341, Lr: 0.000300\n",
      "2020-01-02 13:44:38,793 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:44:38,793 Saving new checkpoint.\n",
      "2020-01-02 13:44:39,029 Example #0\n",
      "2020-01-02 13:44:39,029 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:44:39,029 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:44:39,029 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi Nzambi bu mudi disua dianyi , kadi nuenze . ’\n",
      "2020-01-02 13:44:39,029 Example #1\n",
      "2020-01-02 13:44:39,029 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:44:39,030 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:44:39,030 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumbuka ku bupika .\n",
      "2020-01-02 13:44:39,030 Example #2\n",
      "2020-01-02 13:44:39,030 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:44:39,030 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:44:39,030 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ne tshiji to .\n",
      "2020-01-02 13:44:39,030 Example #3\n",
      "2020-01-02 13:44:39,030 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:44:39,030 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:44:39,030 \tHypothesis: Ndi mushindike ne : badi mu lungenyi lua Yehowa badi bindile dibishibua dia bafue .\n",
      "2020-01-02 13:44:39,030 Validation result (greedy) at epoch  24, step    76000: bleu:  28.07, loss: 36112.9336, ppl:   4.0510, duration: 29.3346s\n",
      "2020-01-02 13:44:50,603 Epoch  24 Step:    76100 Batch Loss:     1.528413 Tokens per Sec:    19115, Lr: 0.000300\n",
      "2020-01-02 13:45:02,346 Epoch  24 Step:    76200 Batch Loss:     1.537135 Tokens per Sec:    19817, Lr: 0.000300\n",
      "2020-01-02 13:45:13,984 Epoch  24 Step:    76300 Batch Loss:     1.416251 Tokens per Sec:    19701, Lr: 0.000300\n",
      "2020-01-02 13:45:25,552 Epoch  24 Step:    76400 Batch Loss:     1.273938 Tokens per Sec:    19558, Lr: 0.000300\n",
      "2020-01-02 13:45:37,170 Epoch  24 Step:    76500 Batch Loss:     1.694085 Tokens per Sec:    19792, Lr: 0.000300\n",
      "2020-01-02 13:45:48,764 Epoch  24 Step:    76600 Batch Loss:     1.524164 Tokens per Sec:    19830, Lr: 0.000300\n",
      "2020-01-02 13:46:00,621 Epoch  24 Step:    76700 Batch Loss:     1.387865 Tokens per Sec:    19810, Lr: 0.000300\n",
      "2020-01-02 13:46:12,326 Epoch  24 Step:    76800 Batch Loss:     1.467781 Tokens per Sec:    19867, Lr: 0.000300\n",
      "2020-01-02 13:46:23,837 Epoch  24 Step:    76900 Batch Loss:     1.146249 Tokens per Sec:    19100, Lr: 0.000300\n",
      "2020-01-02 13:46:35,503 Epoch  24 Step:    77000 Batch Loss:     1.754458 Tokens per Sec:    19597, Lr: 0.000300\n",
      "2020-01-02 13:47:04,632 Example #0\n",
      "2020-01-02 13:47:04,632 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:47:04,632 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:47:04,632 \tHypothesis: YEZU mmutushile tshilejilu bua bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:47:04,632 Example #1\n",
      "2020-01-02 13:47:04,632 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:47:04,632 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:47:04,632 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudi mbulamatadi .\n",
      "2020-01-02 13:47:04,632 Example #2\n",
      "2020-01-02 13:47:04,632 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:47:04,632 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:47:04,632 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 13:47:04,632 Example #3\n",
      "2020-01-02 13:47:04,633 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:47:04,633 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:47:04,633 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa ne : badi bindile dibika dia bafue .\n",
      "2020-01-02 13:47:04,633 Validation result (greedy) at epoch  24, step    77000: bleu:  28.27, loss: 36160.3828, ppl:   4.0585, duration: 29.1295s\n",
      "2020-01-02 13:47:16,159 Epoch  24 Step:    77100 Batch Loss:     1.630073 Tokens per Sec:    19641, Lr: 0.000300\n",
      "2020-01-02 13:47:27,758 Epoch  24 Step:    77200 Batch Loss:     1.261245 Tokens per Sec:    19612, Lr: 0.000300\n",
      "2020-01-02 13:47:39,362 Epoch  24 Step:    77300 Batch Loss:     1.546076 Tokens per Sec:    19813, Lr: 0.000300\n",
      "2020-01-02 13:47:50,848 Epoch  24 Step:    77400 Batch Loss:     1.642152 Tokens per Sec:    19668, Lr: 0.000300\n",
      "2020-01-02 13:48:02,558 Epoch  24 Step:    77500 Batch Loss:     1.729860 Tokens per Sec:    19693, Lr: 0.000300\n",
      "2020-01-02 13:48:14,139 Epoch  24 Step:    77600 Batch Loss:     1.413718 Tokens per Sec:    19811, Lr: 0.000300\n",
      "2020-01-02 13:48:25,739 Epoch  24 Step:    77700 Batch Loss:     1.559894 Tokens per Sec:    19771, Lr: 0.000300\n",
      "2020-01-02 13:48:37,500 Epoch  24 Step:    77800 Batch Loss:     1.547454 Tokens per Sec:    20092, Lr: 0.000300\n",
      "2020-01-02 13:48:49,087 Epoch  24 Step:    77900 Batch Loss:     1.322765 Tokens per Sec:    19582, Lr: 0.000300\n",
      "2020-01-02 13:49:00,705 Epoch  24 Step:    78000 Batch Loss:     1.324094 Tokens per Sec:    19680, Lr: 0.000300\n",
      "2020-01-02 13:49:29,826 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:49:29,826 Saving new checkpoint.\n",
      "2020-01-02 13:49:30,085 Example #0\n",
      "2020-01-02 13:49:30,085 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:49:30,085 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:49:30,085 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:49:30,085 Example #1\n",
      "2020-01-02 13:49:30,085 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:49:30,085 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:49:30,086 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumutuadila .\n",
      "2020-01-02 13:49:30,086 Example #2\n",
      "2020-01-02 13:49:30,086 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:49:30,086 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:49:30,086 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:49:30,086 Example #3\n",
      "2020-01-02 13:49:30,086 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:49:30,086 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:49:30,086 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 13:49:30,086 Validation result (greedy) at epoch  24, step    78000: bleu:  27.95, loss: 36083.9688, ppl:   4.0465, duration: 29.3804s\n",
      "2020-01-02 13:49:41,765 Epoch  24 Step:    78100 Batch Loss:     1.416294 Tokens per Sec:    19658, Lr: 0.000300\n",
      "2020-01-02 13:49:53,406 Epoch  24 Step:    78200 Batch Loss:     1.538504 Tokens per Sec:    19806, Lr: 0.000300\n",
      "2020-01-02 13:50:05,054 Epoch  24 Step:    78300 Batch Loss:     1.546410 Tokens per Sec:    19335, Lr: 0.000300\n",
      "2020-01-02 13:50:16,570 Epoch  24 Step:    78400 Batch Loss:     1.482824 Tokens per Sec:    19481, Lr: 0.000300\n",
      "2020-01-02 13:50:28,438 Epoch  24 Step:    78500 Batch Loss:     1.533702 Tokens per Sec:    20085, Lr: 0.000300\n",
      "2020-01-02 13:50:40,040 Epoch  24 Step:    78600 Batch Loss:     1.486297 Tokens per Sec:    19407, Lr: 0.000300\n",
      "2020-01-02 13:50:51,799 Epoch  24 Step:    78700 Batch Loss:     1.407116 Tokens per Sec:    20033, Lr: 0.000300\n",
      "2020-01-02 13:51:03,359 Epoch  24 Step:    78800 Batch Loss:     1.450908 Tokens per Sec:    19207, Lr: 0.000300\n",
      "2020-01-02 13:51:15,121 Epoch  24 Step:    78900 Batch Loss:     1.638385 Tokens per Sec:    19548, Lr: 0.000300\n",
      "2020-01-02 13:51:19,556 Epoch  24: total training loss 4962.84\n",
      "2020-01-02 13:51:19,557 EPOCH 25\n",
      "2020-01-02 13:51:27,208 Epoch  25 Step:    79000 Batch Loss:     1.636938 Tokens per Sec:    18913, Lr: 0.000300\n",
      "2020-01-02 13:51:56,332 Example #0\n",
      "2020-01-02 13:51:56,332 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:51:56,332 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:51:56,332 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:51:56,332 Example #1\n",
      "2020-01-02 13:51:56,332 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:51:56,332 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:51:56,333 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishe .\n",
      "2020-01-02 13:51:56,333 Example #2\n",
      "2020-01-02 13:51:56,333 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:51:56,333 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:51:56,333 \tHypothesis: Nunku bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:51:56,333 Example #3\n",
      "2020-01-02 13:51:56,333 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:51:56,333 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:51:56,333 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:51:56,333 Validation result (greedy) at epoch  25, step    79000: bleu:  27.49, loss: 36122.8555, ppl:   4.0526, duration: 29.1253s\n",
      "2020-01-02 13:52:07,867 Epoch  25 Step:    79100 Batch Loss:     1.525944 Tokens per Sec:    19439, Lr: 0.000300\n",
      "2020-01-02 13:52:19,404 Epoch  25 Step:    79200 Batch Loss:     1.513818 Tokens per Sec:    19304, Lr: 0.000300\n",
      "2020-01-02 13:52:31,065 Epoch  25 Step:    79300 Batch Loss:     1.579118 Tokens per Sec:    19611, Lr: 0.000300\n",
      "2020-01-02 13:52:42,748 Epoch  25 Step:    79400 Batch Loss:     1.591551 Tokens per Sec:    20249, Lr: 0.000300\n",
      "2020-01-02 13:52:54,453 Epoch  25 Step:    79500 Batch Loss:     1.458378 Tokens per Sec:    19895, Lr: 0.000300\n",
      "2020-01-02 13:53:06,019 Epoch  25 Step:    79600 Batch Loss:     1.322201 Tokens per Sec:    19445, Lr: 0.000300\n",
      "2020-01-02 13:53:17,675 Epoch  25 Step:    79700 Batch Loss:     1.720013 Tokens per Sec:    19964, Lr: 0.000300\n",
      "2020-01-02 13:53:29,214 Epoch  25 Step:    79800 Batch Loss:     1.589324 Tokens per Sec:    19004, Lr: 0.000300\n",
      "2020-01-02 13:53:40,824 Epoch  25 Step:    79900 Batch Loss:     1.406261 Tokens per Sec:    19777, Lr: 0.000300\n",
      "2020-01-02 13:53:52,566 Epoch  25 Step:    80000 Batch Loss:     1.089121 Tokens per Sec:    19797, Lr: 0.000300\n",
      "2020-01-02 13:54:21,676 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:54:21,676 Saving new checkpoint.\n",
      "2020-01-02 13:54:21,913 Example #0\n",
      "2020-01-02 13:54:21,913 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:54:21,913 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:54:21,913 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 13:54:21,913 Example #1\n",
      "2020-01-02 13:54:21,914 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:54:21,914 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:54:21,914 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukina , bakamusaka bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 13:54:21,914 Example #2\n",
      "2020-01-02 13:54:21,914 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:54:21,914 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:54:21,914 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha bubi , kena ne tshiji to .\n",
      "2020-01-02 13:54:21,914 Example #3\n",
      "2020-01-02 13:54:21,914 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:54:21,914 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:54:21,914 \tHypothesis: Ndi mutuishibue ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:54:21,914 Validation result (greedy) at epoch  25, step    80000: bleu:  28.09, loss: 36005.5820, ppl:   4.0342, duration: 29.3480s\n",
      "2020-01-02 13:54:33,578 Epoch  25 Step:    80100 Batch Loss:     1.676302 Tokens per Sec:    19498, Lr: 0.000300\n",
      "2020-01-02 13:54:45,258 Epoch  25 Step:    80200 Batch Loss:     1.557446 Tokens per Sec:    19541, Lr: 0.000300\n",
      "2020-01-02 13:54:56,986 Epoch  25 Step:    80300 Batch Loss:     1.288793 Tokens per Sec:    19960, Lr: 0.000300\n",
      "2020-01-02 13:55:08,620 Epoch  25 Step:    80400 Batch Loss:     1.327953 Tokens per Sec:    19710, Lr: 0.000300\n",
      "2020-01-02 13:55:20,242 Epoch  25 Step:    80500 Batch Loss:     1.558537 Tokens per Sec:    19404, Lr: 0.000300\n",
      "2020-01-02 13:55:31,790 Epoch  25 Step:    80600 Batch Loss:     1.654422 Tokens per Sec:    18958, Lr: 0.000300\n",
      "2020-01-02 13:55:43,497 Epoch  25 Step:    80700 Batch Loss:     1.391479 Tokens per Sec:    19761, Lr: 0.000300\n",
      "2020-01-02 13:55:55,108 Epoch  25 Step:    80800 Batch Loss:     1.513500 Tokens per Sec:    19768, Lr: 0.000300\n",
      "2020-01-02 13:56:06,800 Epoch  25 Step:    80900 Batch Loss:     1.556967 Tokens per Sec:    19720, Lr: 0.000300\n",
      "2020-01-02 13:56:18,478 Epoch  25 Step:    81000 Batch Loss:     1.652247 Tokens per Sec:    19987, Lr: 0.000300\n",
      "2020-01-02 13:56:47,654 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:56:47,655 Saving new checkpoint.\n",
      "2020-01-02 13:56:47,908 Example #0\n",
      "2020-01-02 13:56:47,908 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:56:47,908 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:56:47,908 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusankishayi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:56:47,908 Example #1\n",
      "2020-01-02 13:56:47,908 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:56:47,908 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:56:47,908 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamusaka bua kumbuka mu buloba buvua bumuanyishila ku mikenji .\n",
      "2020-01-02 13:56:47,909 Example #2\n",
      "2020-01-02 13:56:47,909 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:56:47,909 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:56:47,909 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:56:47,909 Example #3\n",
      "2020-01-02 13:56:47,909 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:56:47,909 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:56:47,909 \tHypothesis: Ndi mushindike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 13:56:47,909 Validation result (greedy) at epoch  25, step    81000: bleu:  28.08, loss: 35966.7891, ppl:   4.0281, duration: 29.4309s\n",
      "2020-01-02 13:56:59,601 Epoch  25 Step:    81100 Batch Loss:     1.707249 Tokens per Sec:    19569, Lr: 0.000300\n",
      "2020-01-02 13:57:11,392 Epoch  25 Step:    81200 Batch Loss:     1.462096 Tokens per Sec:    20356, Lr: 0.000300\n",
      "2020-01-02 13:57:23,027 Epoch  25 Step:    81300 Batch Loss:     1.671633 Tokens per Sec:    19579, Lr: 0.000300\n",
      "2020-01-02 13:57:34,709 Epoch  25 Step:    81400 Batch Loss:     1.230557 Tokens per Sec:    19197, Lr: 0.000300\n",
      "2020-01-02 13:57:46,352 Epoch  25 Step:    81500 Batch Loss:     1.183532 Tokens per Sec:    19791, Lr: 0.000300\n",
      "2020-01-02 13:57:58,014 Epoch  25 Step:    81600 Batch Loss:     1.386534 Tokens per Sec:    19475, Lr: 0.000300\n",
      "2020-01-02 13:58:09,673 Epoch  25 Step:    81700 Batch Loss:     1.579830 Tokens per Sec:    19603, Lr: 0.000300\n",
      "2020-01-02 13:58:21,309 Epoch  25 Step:    81800 Batch Loss:     1.456339 Tokens per Sec:    19643, Lr: 0.000300\n",
      "2020-01-02 13:58:32,913 Epoch  25 Step:    81900 Batch Loss:     1.493522 Tokens per Sec:    19490, Lr: 0.000300\n",
      "2020-01-02 13:58:44,550 Epoch  25 Step:    82000 Batch Loss:     1.707104 Tokens per Sec:    19613, Lr: 0.000300\n",
      "2020-01-02 13:59:13,672 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 13:59:13,673 Saving new checkpoint.\n",
      "2020-01-02 13:59:13,909 Example #0\n",
      "2020-01-02 13:59:13,910 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 13:59:13,910 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 13:59:13,910 \tHypothesis: YEZU mmutushile tshilejilu tshimpe kudi bena Kristo bonso pa kusambila Tatuende ne : ‘ Kanusankishayi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 13:59:13,910 Example #1\n",
      "2020-01-02 13:59:13,910 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 13:59:13,910 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 13:59:13,910 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 13:59:13,910 Example #2\n",
      "2020-01-02 13:59:13,910 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 13:59:13,910 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 13:59:13,910 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 13:59:13,910 Example #3\n",
      "2020-01-02 13:59:13,911 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 13:59:13,911 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 13:59:13,911 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 13:59:13,911 Validation result (greedy) at epoch  25, step    82000: bleu:  28.20, loss: 35764.5000, ppl:   3.9967, duration: 29.3602s\n",
      "2020-01-02 13:59:25,553 Epoch  25 Step:    82100 Batch Loss:     1.893657 Tokens per Sec:    19665, Lr: 0.000300\n",
      "2020-01-02 13:59:37,263 Epoch  25 Step:    82200 Batch Loss:     1.750831 Tokens per Sec:    19421, Lr: 0.000300\n",
      "2020-01-02 13:59:40,420 Epoch  25: total training loss 4924.25\n",
      "2020-01-02 13:59:40,420 EPOCH 26\n",
      "2020-01-02 13:59:49,294 Epoch  26 Step:    82300 Batch Loss:     1.572908 Tokens per Sec:    19347, Lr: 0.000300\n",
      "2020-01-02 14:00:00,932 Epoch  26 Step:    82400 Batch Loss:     1.570689 Tokens per Sec:    19624, Lr: 0.000300\n",
      "2020-01-02 14:00:12,397 Epoch  26 Step:    82500 Batch Loss:     1.787584 Tokens per Sec:    19454, Lr: 0.000300\n",
      "2020-01-02 14:00:24,179 Epoch  26 Step:    82600 Batch Loss:     1.251156 Tokens per Sec:    19969, Lr: 0.000300\n",
      "2020-01-02 14:00:35,935 Epoch  26 Step:    82700 Batch Loss:     1.695398 Tokens per Sec:    19636, Lr: 0.000300\n",
      "2020-01-02 14:00:47,587 Epoch  26 Step:    82800 Batch Loss:     1.385056 Tokens per Sec:    19224, Lr: 0.000300\n",
      "2020-01-02 14:00:59,110 Epoch  26 Step:    82900 Batch Loss:     1.491561 Tokens per Sec:    19447, Lr: 0.000300\n",
      "2020-01-02 14:01:10,821 Epoch  26 Step:    83000 Batch Loss:     1.585039 Tokens per Sec:    19615, Lr: 0.000300\n",
      "2020-01-02 14:01:40,056 Example #0\n",
      "2020-01-02 14:01:40,057 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:01:40,057 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:01:40,057 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:01:40,057 Example #1\n",
      "2020-01-02 14:01:40,057 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:01:40,057 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:01:40,057 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamusaka bua kumbuka mu buloba buvua bufumine kudi mbulamatadi .\n",
      "2020-01-02 14:01:40,057 Example #2\n",
      "2020-01-02 14:01:40,058 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:01:40,058 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:01:40,058 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:01:40,058 Example #3\n",
      "2020-01-02 14:01:40,058 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:01:40,058 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:01:40,058 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 14:01:40,058 Validation result (greedy) at epoch  26, step    83000: bleu:  28.06, loss: 35774.5859, ppl:   3.9983, duration: 29.2373s\n",
      "2020-01-02 14:01:51,759 Epoch  26 Step:    83100 Batch Loss:     1.615775 Tokens per Sec:    19828, Lr: 0.000300\n",
      "2020-01-02 14:02:03,406 Epoch  26 Step:    83200 Batch Loss:     1.558454 Tokens per Sec:    19647, Lr: 0.000300\n",
      "2020-01-02 14:02:15,078 Epoch  26 Step:    83300 Batch Loss:     1.457946 Tokens per Sec:    19660, Lr: 0.000300\n",
      "2020-01-02 14:02:26,670 Epoch  26 Step:    83400 Batch Loss:     1.599417 Tokens per Sec:    19506, Lr: 0.000300\n",
      "2020-01-02 14:02:38,345 Epoch  26 Step:    83500 Batch Loss:     1.500199 Tokens per Sec:    19312, Lr: 0.000300\n",
      "2020-01-02 14:02:49,861 Epoch  26 Step:    83600 Batch Loss:     1.696544 Tokens per Sec:    19119, Lr: 0.000300\n",
      "2020-01-02 14:03:01,590 Epoch  26 Step:    83700 Batch Loss:     1.552692 Tokens per Sec:    20009, Lr: 0.000300\n",
      "2020-01-02 14:03:13,264 Epoch  26 Step:    83800 Batch Loss:     1.425630 Tokens per Sec:    19719, Lr: 0.000300\n",
      "2020-01-02 14:03:24,881 Epoch  26 Step:    83900 Batch Loss:     1.429521 Tokens per Sec:    19264, Lr: 0.000300\n",
      "2020-01-02 14:03:36,494 Epoch  26 Step:    84000 Batch Loss:     1.601341 Tokens per Sec:    19580, Lr: 0.000300\n",
      "2020-01-02 14:04:05,571 Example #0\n",
      "2020-01-02 14:04:05,571 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:04:05,571 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:04:05,572 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:04:05,572 Example #1\n",
      "2020-01-02 14:04:05,572 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:04:05,572 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:04:05,572 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 14:04:05,572 Example #2\n",
      "2020-01-02 14:04:05,572 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:04:05,572 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:04:05,572 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , kêna ne tshiji to .\n",
      "2020-01-02 14:04:05,572 Example #3\n",
      "2020-01-02 14:04:05,572 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:04:05,572 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:04:05,572 \tHypothesis: Ndi mujadike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 14:04:05,572 Validation result (greedy) at epoch  26, step    84000: bleu:  28.32, loss: 35871.2891, ppl:   4.0133, duration: 29.0781s\n",
      "2020-01-02 14:04:17,201 Epoch  26 Step:    84100 Batch Loss:     1.670275 Tokens per Sec:    19948, Lr: 0.000300\n",
      "2020-01-02 14:04:28,871 Epoch  26 Step:    84200 Batch Loss:     1.568247 Tokens per Sec:    19675, Lr: 0.000300\n",
      "2020-01-02 14:04:40,407 Epoch  26 Step:    84300 Batch Loss:     1.234181 Tokens per Sec:    19581, Lr: 0.000300\n",
      "2020-01-02 14:04:52,132 Epoch  26 Step:    84400 Batch Loss:     1.588489 Tokens per Sec:    19699, Lr: 0.000300\n",
      "2020-01-02 14:05:03,709 Epoch  26 Step:    84500 Batch Loss:     1.500394 Tokens per Sec:    19820, Lr: 0.000300\n",
      "2020-01-02 14:05:15,338 Epoch  26 Step:    84600 Batch Loss:     1.461184 Tokens per Sec:    20121, Lr: 0.000300\n",
      "2020-01-02 14:05:27,065 Epoch  26 Step:    84700 Batch Loss:     1.572989 Tokens per Sec:    19874, Lr: 0.000300\n",
      "2020-01-02 14:05:38,658 Epoch  26 Step:    84800 Batch Loss:     1.202561 Tokens per Sec:    19243, Lr: 0.000300\n",
      "2020-01-02 14:05:50,315 Epoch  26 Step:    84900 Batch Loss:     1.453535 Tokens per Sec:    19694, Lr: 0.000300\n",
      "2020-01-02 14:06:01,951 Epoch  26 Step:    85000 Batch Loss:     1.527431 Tokens per Sec:    19307, Lr: 0.000300\n",
      "2020-01-02 14:06:31,106 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:06:31,106 Saving new checkpoint.\n",
      "2020-01-02 14:06:31,368 Example #0\n",
      "2020-01-02 14:06:31,369 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:06:31,369 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:06:31,369 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:06:31,369 Example #1\n",
      "2020-01-02 14:06:31,369 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:06:31,369 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:06:31,369 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 14:06:31,369 Example #2\n",
      "2020-01-02 14:06:31,369 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:06:31,369 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:06:31,370 \tHypothesis: Nunku bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:06:31,370 Example #3\n",
      "2020-01-02 14:06:31,370 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:06:31,370 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:06:31,370 \tHypothesis: Ndi mujadike ne : Yehowa udi uvuluka dibika dia bafue .\n",
      "2020-01-02 14:06:31,370 Validation result (greedy) at epoch  26, step    85000: bleu:  28.42, loss: 35723.0156, ppl:   3.9903, duration: 29.4186s\n",
      "2020-01-02 14:06:43,117 Epoch  26 Step:    85100 Batch Loss:     1.594670 Tokens per Sec:    20005, Lr: 0.000300\n",
      "2020-01-02 14:06:54,734 Epoch  26 Step:    85200 Batch Loss:     1.288119 Tokens per Sec:    19190, Lr: 0.000300\n",
      "2020-01-02 14:07:06,403 Epoch  26 Step:    85300 Batch Loss:     1.505761 Tokens per Sec:    19445, Lr: 0.000300\n",
      "2020-01-02 14:07:18,064 Epoch  26 Step:    85400 Batch Loss:     1.492151 Tokens per Sec:    19792, Lr: 0.000300\n",
      "2020-01-02 14:07:29,601 Epoch  26 Step:    85500 Batch Loss:     1.415364 Tokens per Sec:    19058, Lr: 0.000300\n",
      "2020-01-02 14:07:32,227 Epoch  26: total training loss 4907.53\n",
      "2020-01-02 14:07:32,227 EPOCH 27\n",
      "2020-01-02 14:07:41,648 Epoch  27 Step:    85600 Batch Loss:     1.501612 Tokens per Sec:    19003, Lr: 0.000300\n",
      "2020-01-02 14:07:53,258 Epoch  27 Step:    85700 Batch Loss:     1.607520 Tokens per Sec:    19885, Lr: 0.000300\n",
      "2020-01-02 14:08:05,019 Epoch  27 Step:    85800 Batch Loss:     1.588084 Tokens per Sec:    19954, Lr: 0.000300\n",
      "2020-01-02 14:08:16,565 Epoch  27 Step:    85900 Batch Loss:     1.483538 Tokens per Sec:    19542, Lr: 0.000300\n",
      "2020-01-02 14:08:28,286 Epoch  27 Step:    86000 Batch Loss:     1.205328 Tokens per Sec:    19734, Lr: 0.000300\n",
      "2020-01-02 14:08:57,394 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:08:57,394 Saving new checkpoint.\n",
      "2020-01-02 14:08:57,666 Example #0\n",
      "2020-01-02 14:08:57,666 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:08:57,666 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:08:57,666 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenu nuenze bu bantu . ’\n",
      "2020-01-02 14:08:57,666 Example #1\n",
      "2020-01-02 14:08:57,666 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:08:57,666 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:08:57,666 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudiye .\n",
      "2020-01-02 14:08:57,666 Example #2\n",
      "2020-01-02 14:08:57,666 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:08:57,667 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:08:57,667 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:08:57,667 Example #3\n",
      "2020-01-02 14:08:57,667 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:08:57,667 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:08:57,667 \tHypothesis: Ndi mujadike ne : Yehowa mmuindile dibika dia bafue .\n",
      "2020-01-02 14:08:57,667 Validation result (greedy) at epoch  27, step    86000: bleu:  28.27, loss: 35697.0430, ppl:   3.9863, duration: 29.3805s\n",
      "2020-01-02 14:09:09,234 Epoch  27 Step:    86100 Batch Loss:     1.240942 Tokens per Sec:    19505, Lr: 0.000300\n",
      "2020-01-02 14:09:20,719 Epoch  27 Step:    86200 Batch Loss:     1.418978 Tokens per Sec:    19344, Lr: 0.000300\n",
      "2020-01-02 14:09:32,369 Epoch  27 Step:    86300 Batch Loss:     1.557904 Tokens per Sec:    19567, Lr: 0.000300\n",
      "2020-01-02 14:09:44,014 Epoch  27 Step:    86400 Batch Loss:     1.574031 Tokens per Sec:    19928, Lr: 0.000300\n",
      "2020-01-02 14:09:55,631 Epoch  27 Step:    86500 Batch Loss:     1.541623 Tokens per Sec:    19491, Lr: 0.000300\n",
      "2020-01-02 14:10:07,342 Epoch  27 Step:    86600 Batch Loss:     1.432385 Tokens per Sec:    19151, Lr: 0.000300\n",
      "2020-01-02 14:10:19,064 Epoch  27 Step:    86700 Batch Loss:     1.431653 Tokens per Sec:    19980, Lr: 0.000300\n",
      "2020-01-02 14:10:30,657 Epoch  27 Step:    86800 Batch Loss:     1.474977 Tokens per Sec:    19407, Lr: 0.000300\n",
      "2020-01-02 14:10:42,344 Epoch  27 Step:    86900 Batch Loss:     1.700543 Tokens per Sec:    19876, Lr: 0.000300\n",
      "2020-01-02 14:10:53,872 Epoch  27 Step:    87000 Batch Loss:     1.548765 Tokens per Sec:    19520, Lr: 0.000300\n",
      "2020-01-02 14:11:23,018 Example #0\n",
      "2020-01-02 14:11:23,018 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:11:23,019 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:11:23,019 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:11:23,019 Example #1\n",
      "2020-01-02 14:11:23,019 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:11:23,019 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:11:23,019 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishibue .\n",
      "2020-01-02 14:11:23,019 Example #2\n",
      "2020-01-02 14:11:23,019 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:11:23,019 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:11:23,019 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:11:23,019 Example #3\n",
      "2020-01-02 14:11:23,019 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:11:23,019 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:11:23,020 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa badi bindila dibika dia bafue .\n",
      "2020-01-02 14:11:23,020 Validation result (greedy) at epoch  27, step    87000: bleu:  28.19, loss: 35712.0664, ppl:   3.9886, duration: 29.1473s\n",
      "2020-01-02 14:11:34,749 Epoch  27 Step:    87100 Batch Loss:     1.170818 Tokens per Sec:    19756, Lr: 0.000300\n",
      "2020-01-02 14:11:46,433 Epoch  27 Step:    87200 Batch Loss:     1.467067 Tokens per Sec:    19752, Lr: 0.000300\n",
      "2020-01-02 14:11:58,165 Epoch  27 Step:    87300 Batch Loss:     1.692358 Tokens per Sec:    19368, Lr: 0.000300\n",
      "2020-01-02 14:12:09,743 Epoch  27 Step:    87400 Batch Loss:     1.633161 Tokens per Sec:    19580, Lr: 0.000300\n",
      "2020-01-02 14:12:21,261 Epoch  27 Step:    87500 Batch Loss:     1.442457 Tokens per Sec:    19486, Lr: 0.000300\n",
      "2020-01-02 14:12:32,991 Epoch  27 Step:    87600 Batch Loss:     1.253593 Tokens per Sec:    19745, Lr: 0.000300\n",
      "2020-01-02 14:12:44,591 Epoch  27 Step:    87700 Batch Loss:     1.450635 Tokens per Sec:    19621, Lr: 0.000300\n",
      "2020-01-02 14:12:56,377 Epoch  27 Step:    87800 Batch Loss:     1.663737 Tokens per Sec:    19739, Lr: 0.000300\n",
      "2020-01-02 14:13:07,918 Epoch  27 Step:    87900 Batch Loss:     1.456448 Tokens per Sec:    19420, Lr: 0.000300\n",
      "2020-01-02 14:13:19,548 Epoch  27 Step:    88000 Batch Loss:     1.646799 Tokens per Sec:    19858, Lr: 0.000300\n",
      "2020-01-02 14:13:48,642 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:13:48,642 Saving new checkpoint.\n",
      "2020-01-02 14:13:48,876 Example #0\n",
      "2020-01-02 14:13:48,876 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:13:48,876 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:13:48,877 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze muenu muonso . ’\n",
      "2020-01-02 14:13:48,877 Example #1\n",
      "2020-01-02 14:13:48,877 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:13:48,877 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:13:48,877 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudiye .\n",
      "2020-01-02 14:13:48,877 Example #2\n",
      "2020-01-02 14:13:48,877 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:13:48,877 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:13:48,877 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:13:48,877 Example #3\n",
      "2020-01-02 14:13:48,877 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:13:48,877 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:13:48,877 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa tshidi tshibindile dibika dia bafue .\n",
      "2020-01-02 14:13:48,878 Validation result (greedy) at epoch  27, step    88000: bleu:  28.46, loss: 35649.1758, ppl:   3.9789, duration: 29.3288s\n",
      "2020-01-02 14:14:00,587 Epoch  27 Step:    88100 Batch Loss:     1.311411 Tokens per Sec:    19325, Lr: 0.000300\n",
      "2020-01-02 14:14:12,199 Epoch  27 Step:    88200 Batch Loss:     1.624313 Tokens per Sec:    19484, Lr: 0.000300\n",
      "2020-01-02 14:14:23,869 Epoch  27 Step:    88300 Batch Loss:     1.689752 Tokens per Sec:    19858, Lr: 0.000300\n",
      "2020-01-02 14:14:35,549 Epoch  27 Step:    88400 Batch Loss:     1.535869 Tokens per Sec:    19607, Lr: 0.000300\n",
      "2020-01-02 14:14:47,247 Epoch  27 Step:    88500 Batch Loss:     1.539980 Tokens per Sec:    20164, Lr: 0.000300\n",
      "2020-01-02 14:14:58,960 Epoch  27 Step:    88600 Batch Loss:     1.450191 Tokens per Sec:    19699, Lr: 0.000300\n",
      "2020-01-02 14:15:10,555 Epoch  27 Step:    88700 Batch Loss:     1.564841 Tokens per Sec:    19876, Lr: 0.000300\n",
      "2020-01-02 14:15:22,279 Epoch  27 Step:    88800 Batch Loss:     1.236791 Tokens per Sec:    20019, Lr: 0.000300\n",
      "2020-01-02 14:15:23,172 Epoch  27: total training loss 4854.36\n",
      "2020-01-02 14:15:23,172 EPOCH 28\n",
      "2020-01-02 14:15:34,260 Epoch  28 Step:    88900 Batch Loss:     1.582006 Tokens per Sec:    18708, Lr: 0.000300\n",
      "2020-01-02 14:15:45,835 Epoch  28 Step:    89000 Batch Loss:     1.386179 Tokens per Sec:    19807, Lr: 0.000300\n",
      "2020-01-02 14:16:14,939 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:16:14,940 Saving new checkpoint.\n",
      "2020-01-02 14:16:15,176 Example #0\n",
      "2020-01-02 14:16:15,177 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:16:15,177 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:16:15,177 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu munudi nuenza . ’\n",
      "2020-01-02 14:16:15,177 Example #1\n",
      "2020-01-02 14:16:15,177 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:16:15,177 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:16:15,177 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila ku mikenji .\n",
      "2020-01-02 14:16:15,177 Example #2\n",
      "2020-01-02 14:16:15,177 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:16:15,177 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:16:15,178 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 14:16:15,178 Example #3\n",
      "2020-01-02 14:16:15,178 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:16:15,178 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:16:15,178 \tHypothesis: Ndi mutuishibue ne : Yehowa mmuvuluke dibika dia bafue .\n",
      "2020-01-02 14:16:15,178 Validation result (greedy) at epoch  28, step    89000: bleu:  28.32, loss: 35510.9336, ppl:   3.9576, duration: 29.3431s\n",
      "2020-01-02 14:16:26,783 Epoch  28 Step:    89100 Batch Loss:     1.900801 Tokens per Sec:    19877, Lr: 0.000300\n",
      "2020-01-02 14:16:38,462 Epoch  28 Step:    89200 Batch Loss:     1.477897 Tokens per Sec:    19593, Lr: 0.000300\n",
      "2020-01-02 14:16:50,099 Epoch  28 Step:    89300 Batch Loss:     1.439405 Tokens per Sec:    19681, Lr: 0.000300\n",
      "2020-01-02 14:17:01,902 Epoch  28 Step:    89400 Batch Loss:     1.497220 Tokens per Sec:    20073, Lr: 0.000300\n",
      "2020-01-02 14:17:13,487 Epoch  28 Step:    89500 Batch Loss:     1.210858 Tokens per Sec:    19188, Lr: 0.000300\n",
      "2020-01-02 14:17:25,230 Epoch  28 Step:    89600 Batch Loss:     1.610329 Tokens per Sec:    19743, Lr: 0.000300\n",
      "2020-01-02 14:17:36,853 Epoch  28 Step:    89700 Batch Loss:     0.999224 Tokens per Sec:    19532, Lr: 0.000300\n",
      "2020-01-02 14:17:48,393 Epoch  28 Step:    89800 Batch Loss:     1.570440 Tokens per Sec:    19509, Lr: 0.000300\n",
      "2020-01-02 14:18:00,084 Epoch  28 Step:    89900 Batch Loss:     1.228214 Tokens per Sec:    19560, Lr: 0.000300\n",
      "2020-01-02 14:18:11,838 Epoch  28 Step:    90000 Batch Loss:     1.266409 Tokens per Sec:    19896, Lr: 0.000300\n",
      "2020-01-02 14:18:40,960 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:18:40,961 Saving new checkpoint.\n",
      "2020-01-02 14:18:41,195 Example #0\n",
      "2020-01-02 14:18:41,195 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:18:41,195 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:18:41,195 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusengeledi bua disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:18:41,195 Example #1\n",
      "2020-01-02 14:18:41,195 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:18:41,195 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:18:41,195 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua mikenji mifundishe ku mikanda .\n",
      "2020-01-02 14:18:41,196 Example #2\n",
      "2020-01-02 14:18:41,196 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:18:41,196 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:18:41,196 \tHypothesis: Nunku bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ukuata tshiji lubilu to .\n",
      "2020-01-02 14:18:41,196 Example #3\n",
      "2020-01-02 14:18:41,196 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:18:41,196 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:18:41,196 \tHypothesis: Ndi mujadike ne : badi mu tshivulukilu tshia Yehowa badi bindile dibishibua .\n",
      "2020-01-02 14:18:41,196 Validation result (greedy) at epoch  28, step    90000: bleu:  29.01, loss: 35427.6914, ppl:   3.9449, duration: 29.3583s\n",
      "2020-01-02 14:18:52,840 Epoch  28 Step:    90100 Batch Loss:     1.745631 Tokens per Sec:    19743, Lr: 0.000300\n",
      "2020-01-02 14:19:04,401 Epoch  28 Step:    90200 Batch Loss:     1.578848 Tokens per Sec:    19267, Lr: 0.000300\n",
      "2020-01-02 14:19:15,924 Epoch  28 Step:    90300 Batch Loss:     1.485073 Tokens per Sec:    19599, Lr: 0.000300\n",
      "2020-01-02 14:19:27,657 Epoch  28 Step:    90400 Batch Loss:     1.577268 Tokens per Sec:    19718, Lr: 0.000300\n",
      "2020-01-02 14:19:39,358 Epoch  28 Step:    90500 Batch Loss:     1.603005 Tokens per Sec:    19834, Lr: 0.000300\n",
      "2020-01-02 14:19:50,892 Epoch  28 Step:    90600 Batch Loss:     1.526069 Tokens per Sec:    19460, Lr: 0.000300\n",
      "2020-01-02 14:20:02,653 Epoch  28 Step:    90700 Batch Loss:     1.334376 Tokens per Sec:    19973, Lr: 0.000300\n",
      "2020-01-02 14:20:14,177 Epoch  28 Step:    90800 Batch Loss:     1.529369 Tokens per Sec:    19532, Lr: 0.000300\n",
      "2020-01-02 14:20:25,758 Epoch  28 Step:    90900 Batch Loss:     1.467695 Tokens per Sec:    19728, Lr: 0.000300\n",
      "2020-01-02 14:20:37,459 Epoch  28 Step:    91000 Batch Loss:     1.639791 Tokens per Sec:    19950, Lr: 0.000300\n",
      "2020-01-02 14:21:06,596 Example #0\n",
      "2020-01-02 14:21:06,596 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:21:06,596 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:21:06,596 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusankianganyi bu mudi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:21:06,596 Example #1\n",
      "2020-01-02 14:21:06,596 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:21:06,596 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:21:06,596 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 14:21:06,596 Example #2\n",
      "2020-01-02 14:21:06,597 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:21:06,597 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:21:06,597 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ne tshiji to .\n",
      "2020-01-02 14:21:06,597 Example #3\n",
      "2020-01-02 14:21:06,597 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:21:06,597 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:21:06,597 \tHypothesis: Ndi mushindike ne : Yehowa mmuindile dibika dia bafue .\n",
      "2020-01-02 14:21:06,597 Validation result (greedy) at epoch  28, step    91000: bleu:  28.77, loss: 35440.0703, ppl:   3.9468, duration: 29.1381s\n",
      "2020-01-02 14:21:18,237 Epoch  28 Step:    91100 Batch Loss:     1.298150 Tokens per Sec:    19635, Lr: 0.000300\n",
      "2020-01-02 14:21:29,911 Epoch  28 Step:    91200 Batch Loss:     1.647556 Tokens per Sec:    19666, Lr: 0.000300\n",
      "2020-01-02 14:21:41,552 Epoch  28 Step:    91300 Batch Loss:     1.423276 Tokens per Sec:    20069, Lr: 0.000300\n",
      "2020-01-02 14:21:53,227 Epoch  28 Step:    91400 Batch Loss:     1.596666 Tokens per Sec:    19581, Lr: 0.000300\n",
      "2020-01-02 14:22:04,909 Epoch  28 Step:    91500 Batch Loss:     1.400947 Tokens per Sec:    20193, Lr: 0.000300\n",
      "2020-01-02 14:22:16,611 Epoch  28 Step:    91600 Batch Loss:     1.374709 Tokens per Sec:    19580, Lr: 0.000300\n",
      "2020-01-02 14:22:28,377 Epoch  28 Step:    91700 Batch Loss:     1.536177 Tokens per Sec:    19640, Lr: 0.000300\n",
      "2020-01-02 14:22:39,770 Epoch  28 Step:    91800 Batch Loss:     1.431406 Tokens per Sec:    19414, Lr: 0.000300\n",
      "2020-01-02 14:22:51,406 Epoch  28 Step:    91900 Batch Loss:     1.625530 Tokens per Sec:    19543, Lr: 0.000300\n",
      "2020-01-02 14:23:03,119 Epoch  28 Step:    92000 Batch Loss:     1.562843 Tokens per Sec:    19584, Lr: 0.000300\n",
      "2020-01-02 14:23:32,221 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:23:32,221 Saving new checkpoint.\n",
      "2020-01-02 14:23:32,476 Example #0\n",
      "2020-01-02 14:23:32,476 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:23:32,476 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:23:32,477 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusangishi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:23:32,477 Example #1\n",
      "2020-01-02 14:23:32,477 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:23:32,477 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:23:32,477 \tHypothesis: Bu muvua bana babu bikale ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudiye .\n",
      "2020-01-02 14:23:32,477 Example #2\n",
      "2020-01-02 14:23:32,477 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:23:32,477 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:23:32,477 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:23:32,477 Example #3\n",
      "2020-01-02 14:23:32,477 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:23:32,477 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:23:32,477 \tHypothesis: Ndi mutuishibue ne : Yehowa mmuvuluke dibika dia bafue .\n",
      "2020-01-02 14:23:32,477 Validation result (greedy) at epoch  28, step    92000: bleu:  28.63, loss: 35333.9375, ppl:   3.9306, duration: 29.3585s\n",
      "2020-01-02 14:23:43,348 Epoch  28: total training loss 4832.50\n",
      "2020-01-02 14:23:43,349 EPOCH 29\n",
      "2020-01-02 14:23:44,411 Epoch  29 Step:    92100 Batch Loss:     1.634050 Tokens per Sec:    14106, Lr: 0.000300\n",
      "2020-01-02 14:23:55,994 Epoch  29 Step:    92200 Batch Loss:     1.174509 Tokens per Sec:    19643, Lr: 0.000300\n",
      "2020-01-02 14:24:07,726 Epoch  29 Step:    92300 Batch Loss:     1.538106 Tokens per Sec:    20257, Lr: 0.000300\n",
      "2020-01-02 14:24:19,350 Epoch  29 Step:    92400 Batch Loss:     1.071787 Tokens per Sec:    19671, Lr: 0.000300\n",
      "2020-01-02 14:24:31,050 Epoch  29 Step:    92500 Batch Loss:     1.384531 Tokens per Sec:    19910, Lr: 0.000300\n",
      "2020-01-02 14:24:42,448 Epoch  29 Step:    92600 Batch Loss:     1.614421 Tokens per Sec:    19208, Lr: 0.000300\n",
      "2020-01-02 14:24:54,208 Epoch  29 Step:    92700 Batch Loss:     1.645797 Tokens per Sec:    19712, Lr: 0.000300\n",
      "2020-01-02 14:25:05,856 Epoch  29 Step:    92800 Batch Loss:     1.568648 Tokens per Sec:    19494, Lr: 0.000300\n",
      "2020-01-02 14:25:17,459 Epoch  29 Step:    92900 Batch Loss:     1.571568 Tokens per Sec:    19796, Lr: 0.000300\n",
      "2020-01-02 14:25:29,022 Epoch  29 Step:    93000 Batch Loss:     1.611725 Tokens per Sec:    19234, Lr: 0.000300\n",
      "2020-01-02 14:25:58,216 Example #0\n",
      "2020-01-02 14:25:58,217 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:25:58,217 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:25:58,217 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanusangi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:25:58,217 Example #1\n",
      "2020-01-02 14:25:58,217 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:25:58,217 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:25:58,217 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bufumine kudiye .\n",
      "2020-01-02 14:25:58,217 Example #2\n",
      "2020-01-02 14:25:58,217 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:25:58,218 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:25:58,218 \tHypothesis: Nunku , bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kena ne tshiji to .\n",
      "2020-01-02 14:25:58,218 Example #3\n",
      "2020-01-02 14:25:58,218 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:25:58,218 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:25:58,218 \tHypothesis: Ndi mutuishibue ne : Yehowa mmubindile dibika dia bafue .\n",
      "2020-01-02 14:25:58,218 Validation result (greedy) at epoch  29, step    93000: bleu:  28.12, loss: 35402.2227, ppl:   3.9410, duration: 29.1958s\n",
      "2020-01-02 14:26:09,881 Epoch  29 Step:    93100 Batch Loss:     1.524670 Tokens per Sec:    19591, Lr: 0.000300\n",
      "2020-01-02 14:26:21,549 Epoch  29 Step:    93200 Batch Loss:     1.580577 Tokens per Sec:    19274, Lr: 0.000300\n",
      "2020-01-02 14:26:33,237 Epoch  29 Step:    93300 Batch Loss:     1.115845 Tokens per Sec:    19685, Lr: 0.000300\n",
      "2020-01-02 14:26:44,930 Epoch  29 Step:    93400 Batch Loss:     1.507583 Tokens per Sec:    19955, Lr: 0.000300\n",
      "2020-01-02 14:26:56,575 Epoch  29 Step:    93500 Batch Loss:     1.477799 Tokens per Sec:    19493, Lr: 0.000300\n",
      "2020-01-02 14:27:08,291 Epoch  29 Step:    93600 Batch Loss:     1.629925 Tokens per Sec:    19945, Lr: 0.000300\n",
      "2020-01-02 14:27:20,051 Epoch  29 Step:    93700 Batch Loss:     1.652617 Tokens per Sec:    20109, Lr: 0.000300\n",
      "2020-01-02 14:27:31,795 Epoch  29 Step:    93800 Batch Loss:     1.518943 Tokens per Sec:    19581, Lr: 0.000300\n",
      "2020-01-02 14:27:43,467 Epoch  29 Step:    93900 Batch Loss:     1.261880 Tokens per Sec:    19786, Lr: 0.000300\n",
      "2020-01-02 14:27:55,123 Epoch  29 Step:    94000 Batch Loss:     1.506257 Tokens per Sec:    19481, Lr: 0.000300\n",
      "2020-01-02 14:28:24,210 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:28:24,210 Saving new checkpoint.\n",
      "2020-01-02 14:28:24,469 Example #0\n",
      "2020-01-02 14:28:24,469 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:28:24,469 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:28:24,469 \tHypothesis: YEZU mmutushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanudiangatshi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:28:24,469 Example #1\n",
      "2020-01-02 14:28:24,470 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:28:24,470 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:28:24,470 \tHypothesis: Bu muvua bana babu ne lukunukunu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 14:28:24,470 Example #2\n",
      "2020-01-02 14:28:24,470 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:28:24,470 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:28:24,470 \tHypothesis: Nunku bua Mose kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:28:24,470 Example #3\n",
      "2020-01-02 14:28:24,470 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:28:24,470 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:28:24,470 \tHypothesis: Ndi mutuishibue ne : badi mu tshivulukilu tshia Yehowa badi bindile dibika dia bafue .\n",
      "2020-01-02 14:28:24,470 Validation result (greedy) at epoch  29, step    94000: bleu:  28.77, loss: 35315.7305, ppl:   3.9278, duration: 29.3468s\n",
      "2020-01-02 14:28:36,240 Epoch  29 Step:    94100 Batch Loss:     1.738268 Tokens per Sec:    19678, Lr: 0.000300\n",
      "2020-01-02 14:28:47,698 Epoch  29 Step:    94200 Batch Loss:     1.382370 Tokens per Sec:    19562, Lr: 0.000300\n",
      "2020-01-02 14:28:59,335 Epoch  29 Step:    94300 Batch Loss:     1.488135 Tokens per Sec:    19111, Lr: 0.000300\n",
      "2020-01-02 14:29:10,942 Epoch  29 Step:    94400 Batch Loss:     1.601539 Tokens per Sec:    20001, Lr: 0.000300\n",
      "2020-01-02 14:29:22,489 Epoch  29 Step:    94500 Batch Loss:     1.455554 Tokens per Sec:    19892, Lr: 0.000300\n",
      "2020-01-02 14:29:34,182 Epoch  29 Step:    94600 Batch Loss:     1.633091 Tokens per Sec:    19447, Lr: 0.000300\n",
      "2020-01-02 14:29:45,705 Epoch  29 Step:    94700 Batch Loss:     1.640550 Tokens per Sec:    19164, Lr: 0.000300\n",
      "2020-01-02 14:29:57,442 Epoch  29 Step:    94800 Batch Loss:     1.163836 Tokens per Sec:    20130, Lr: 0.000300\n",
      "2020-01-02 14:30:09,096 Epoch  29 Step:    94900 Batch Loss:     1.603785 Tokens per Sec:    19590, Lr: 0.000300\n",
      "2020-01-02 14:30:20,743 Epoch  29 Step:    95000 Batch Loss:     1.442795 Tokens per Sec:    19437, Lr: 0.000300\n",
      "2020-01-02 14:30:49,918 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:30:49,918 Saving new checkpoint.\n",
      "2020-01-02 14:30:50,173 Example #0\n",
      "2020-01-02 14:30:50,174 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:30:50,174 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:30:50,174 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:30:50,174 Example #1\n",
      "2020-01-02 14:30:50,174 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:30:50,174 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:30:50,174 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 14:30:50,174 Example #2\n",
      "2020-01-02 14:30:50,174 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:30:50,174 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:30:50,174 \tHypothesis: Nunku , bu muvua Mose mua kumvua njila yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:30:50,174 Example #3\n",
      "2020-01-02 14:30:50,174 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:30:50,174 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:30:50,174 \tHypothesis: Ndi mutuishibue ne : Yehowa mmuindile dibika dia bafue .\n",
      "2020-01-02 14:30:50,174 Validation result (greedy) at epoch  29, step    95000: bleu:  28.76, loss: 35239.1523, ppl:   3.9162, duration: 29.4306s\n",
      "2020-01-02 14:31:01,849 Epoch  29 Step:    95100 Batch Loss:     1.620403 Tokens per Sec:    19358, Lr: 0.000300\n",
      "2020-01-02 14:31:13,412 Epoch  29 Step:    95200 Batch Loss:     1.443670 Tokens per Sec:    19353, Lr: 0.000300\n",
      "2020-01-02 14:31:25,044 Epoch  29 Step:    95300 Batch Loss:     1.517223 Tokens per Sec:    19851, Lr: 0.000300\n",
      "2020-01-02 14:31:34,839 Epoch  29: total training loss 4818.90\n",
      "2020-01-02 14:31:34,839 EPOCH 30\n",
      "2020-01-02 14:31:37,054 Epoch  30 Step:    95400 Batch Loss:     1.647085 Tokens per Sec:    17048, Lr: 0.000300\n",
      "2020-01-02 14:31:48,651 Epoch  30 Step:    95500 Batch Loss:     1.558226 Tokens per Sec:    19262, Lr: 0.000300\n",
      "2020-01-02 14:32:00,285 Epoch  30 Step:    95600 Batch Loss:     1.484925 Tokens per Sec:    19720, Lr: 0.000300\n",
      "2020-01-02 14:32:11,981 Epoch  30 Step:    95700 Batch Loss:     1.671994 Tokens per Sec:    19732, Lr: 0.000300\n",
      "2020-01-02 14:32:23,488 Epoch  30 Step:    95800 Batch Loss:     1.244525 Tokens per Sec:    19559, Lr: 0.000300\n",
      "2020-01-02 14:32:35,245 Epoch  30 Step:    95900 Batch Loss:     1.649011 Tokens per Sec:    19779, Lr: 0.000300\n",
      "2020-01-02 14:32:47,005 Epoch  30 Step:    96000 Batch Loss:     1.445109 Tokens per Sec:    19795, Lr: 0.000300\n",
      "2020-01-02 14:33:16,133 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:33:16,133 Saving new checkpoint.\n",
      "2020-01-02 14:33:16,368 Example #0\n",
      "2020-01-02 14:33:16,369 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:33:16,369 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:33:16,369 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshia kusambila Tatuende ne : ‘ Kanuenji disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:33:16,369 Example #1\n",
      "2020-01-02 14:33:16,369 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:33:16,369 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:33:16,369 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumuanyishila kudi mbulamatadi .\n",
      "2020-01-02 14:33:16,369 Example #2\n",
      "2020-01-02 14:33:16,369 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:33:16,369 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:33:16,369 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:33:16,369 Example #3\n",
      "2020-01-02 14:33:16,370 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:33:16,370 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:33:16,370 \tHypothesis: Ndi mutuishibue ne : Yehowa mmuvuluke dibika dia bafue .\n",
      "2020-01-02 14:33:16,370 Validation result (greedy) at epoch  30, step    96000: bleu:  29.03, loss: 35210.6680, ppl:   3.9119, duration: 29.3646s\n",
      "2020-01-02 14:33:27,960 Epoch  30 Step:    96100 Batch Loss:     1.372465 Tokens per Sec:    19888, Lr: 0.000300\n",
      "2020-01-02 14:33:39,566 Epoch  30 Step:    96200 Batch Loss:     1.231085 Tokens per Sec:    19470, Lr: 0.000300\n",
      "2020-01-02 14:33:51,107 Epoch  30 Step:    96300 Batch Loss:     1.360615 Tokens per Sec:    19326, Lr: 0.000300\n",
      "2020-01-02 14:34:02,768 Epoch  30 Step:    96400 Batch Loss:     1.576857 Tokens per Sec:    19569, Lr: 0.000300\n",
      "2020-01-02 14:34:14,442 Epoch  30 Step:    96500 Batch Loss:     1.538884 Tokens per Sec:    19782, Lr: 0.000300\n",
      "2020-01-02 14:34:26,123 Epoch  30 Step:    96600 Batch Loss:     1.364654 Tokens per Sec:    19861, Lr: 0.000300\n",
      "2020-01-02 14:34:37,730 Epoch  30 Step:    96700 Batch Loss:     0.906132 Tokens per Sec:    19571, Lr: 0.000300\n",
      "2020-01-02 14:34:49,462 Epoch  30 Step:    96800 Batch Loss:     1.625338 Tokens per Sec:    19785, Lr: 0.000300\n",
      "2020-01-02 14:35:00,985 Epoch  30 Step:    96900 Batch Loss:     1.680116 Tokens per Sec:    19160, Lr: 0.000300\n",
      "2020-01-02 14:35:12,561 Epoch  30 Step:    97000 Batch Loss:     1.397109 Tokens per Sec:    19915, Lr: 0.000300\n",
      "2020-01-02 14:35:41,716 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:35:41,716 Saving new checkpoint.\n",
      "2020-01-02 14:35:41,977 Example #0\n",
      "2020-01-02 14:35:41,977 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:35:41,977 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:35:41,977 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusanki bu mudi disua dianyi , kadi nuenze . ’\n",
      "2020-01-02 14:35:41,977 Example #1\n",
      "2020-01-02 14:35:41,978 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:35:41,978 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:35:41,978 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumusaka bua kumushiya ku bupika .\n",
      "2020-01-02 14:35:41,978 Example #2\n",
      "2020-01-02 14:35:41,978 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:35:41,978 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:35:41,978 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:35:41,978 Example #3\n",
      "2020-01-02 14:35:41,978 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:35:41,978 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:35:41,978 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 14:35:41,978 Validation result (greedy) at epoch  30, step    97000: bleu:  28.99, loss: 35144.5547, ppl:   3.9019, duration: 29.4166s\n",
      "2020-01-02 14:35:53,671 Epoch  30 Step:    97100 Batch Loss:     1.416647 Tokens per Sec:    19993, Lr: 0.000300\n",
      "2020-01-02 14:36:05,432 Epoch  30 Step:    97200 Batch Loss:     1.224349 Tokens per Sec:    19682, Lr: 0.000300\n",
      "2020-01-02 14:36:17,045 Epoch  30 Step:    97300 Batch Loss:     1.540123 Tokens per Sec:    19536, Lr: 0.000300\n",
      "2020-01-02 14:36:28,786 Epoch  30 Step:    97400 Batch Loss:     1.306432 Tokens per Sec:    19432, Lr: 0.000300\n",
      "2020-01-02 14:36:40,375 Epoch  30 Step:    97500 Batch Loss:     1.185667 Tokens per Sec:    19448, Lr: 0.000300\n",
      "2020-01-02 14:36:52,008 Epoch  30 Step:    97600 Batch Loss:     1.630430 Tokens per Sec:    19718, Lr: 0.000300\n",
      "2020-01-02 14:37:03,660 Epoch  30 Step:    97700 Batch Loss:     1.593200 Tokens per Sec:    19551, Lr: 0.000300\n",
      "2020-01-02 14:37:15,313 Epoch  30 Step:    97800 Batch Loss:     1.218249 Tokens per Sec:    19696, Lr: 0.000300\n",
      "2020-01-02 14:37:27,027 Epoch  30 Step:    97900 Batch Loss:     1.496379 Tokens per Sec:    19699, Lr: 0.000300\n",
      "2020-01-02 14:37:38,704 Epoch  30 Step:    98000 Batch Loss:     1.378946 Tokens per Sec:    19282, Lr: 0.000300\n",
      "2020-01-02 14:38:07,903 Hooray! New best validation result [ppl]!\n",
      "2020-01-02 14:38:07,903 Saving new checkpoint.\n",
      "2020-01-02 14:38:08,145 Example #0\n",
      "2020-01-02 14:38:08,146 \tSource:     JESUS set a pattern for all Christians by praying to his Father : “ Let , not my will , but yours take place . ”\n",
      "2020-01-02 14:38:08,146 \tReference:  YEZU wakashila bena Kristo bonso tshilejilu tshia kulonda pakalombaye Tatuende ne : ‘ Bualu buenjibue bu muudi musue , kabuenjibu bu mundi musue . ’\n",
      "2020-01-02 14:38:08,146 \tHypothesis: YEZU mmushile bena Kristo bonso tshilejilu tshimpe pa kusambila Tatuende ne : ‘ Kanusankishidi disua dianyi , kadi nuenze bu mundi musue . ’\n",
      "2020-01-02 14:38:08,146 Example #1\n",
      "2020-01-02 14:38:08,146 \tSource:     Because his brothers were jealous and hated him , they forced him to leave the land that legally belonged to him .\n",
      "2020-01-02 14:38:08,146 \tReference:  Bu muvua bana babu bamumvuile mukawu ne bamukine , bakamuenzeja bua kumbuka mu buloba buvua bumpianyi buende .\n",
      "2020-01-02 14:38:08,146 \tHypothesis: Bu muvua bana babu ne mukawu ne bamukine , bakamusaka bua kumbuka mu buloba buvua bumusaka bua kumushiya .\n",
      "2020-01-02 14:38:08,146 Example #2\n",
      "2020-01-02 14:38:08,147 \tSource:     So that Moses would understand His ways , Jehovah made it clear that although he does not approve of sin , he is slow to anger .\n",
      "2020-01-02 14:38:08,147 \tReference:  Bua kujadika ne : Mose uvua mua kumvua bimpe njila Yende , Yehowa wakaleja patoke ne : nansha mudiye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu .\n",
      "2020-01-02 14:38:08,147 \tHypothesis: Nunku , Yehowa wakaleja patoke ne : nansha muvuaye kayi wanyisha mpekatu , kêna ukuata tshiji lubilu to .\n",
      "2020-01-02 14:38:08,147 Example #3\n",
      "2020-01-02 14:38:08,147 \tSource:     I am confident that they are in Jehovah’s memory awaiting the resurrection .\n",
      "2020-01-02 14:38:08,147 \tReference:  Ndi mutuishibue ne : Yehowa mmubalame mu meji bua kubabisha ku lufu .\n",
      "2020-01-02 14:38:08,147 \tHypothesis: Ndi mushindike ne : badi mu tshivulukilu tshia Yehowa bua kuindila dibika dia bafue .\n",
      "2020-01-02 14:38:08,147 Validation result (greedy) at epoch  30, step    98000: bleu:  28.92, loss: 35133.7812, ppl:   3.9002, duration: 29.4429s\n",
      "2020-01-02 14:38:19,822 Epoch  30 Step:    98100 Batch Loss:     1.474000 Tokens per Sec:    19623, Lr: 0.000300\n",
      "2020-01-02 14:38:31,327 Epoch  30 Step:    98200 Batch Loss:     1.604056 Tokens per Sec:    18929, Lr: 0.000300\n",
      "2020-01-02 14:38:43,030 Epoch  30 Step:    98300 Batch Loss:     1.582840 Tokens per Sec:    19857, Lr: 0.000300\n",
      "2020-01-02 14:38:54,732 Epoch  30 Step:    98400 Batch Loss:     1.132458 Tokens per Sec:    19604, Lr: 0.000300\n",
      "2020-01-02 14:39:06,504 Epoch  30 Step:    98500 Batch Loss:     1.477367 Tokens per Sec:    20078, Lr: 0.000300\n",
      "2020-01-02 14:39:18,111 Epoch  30 Step:    98600 Batch Loss:     1.156565 Tokens per Sec:    19780, Lr: 0.000300\n",
      "2020-01-02 14:39:26,778 Epoch  30: total training loss 4796.03\n",
      "2020-01-02 14:39:26,779 Training ended after  30 epochs.\n",
      "2020-01-02 14:39:26,779 Best validation result (greedy) at step    98000:   3.90 ppl.\n",
      "2020-01-02 14:39:50,951  dev bleu:  29.96 [Beam search decoding with beam size = 5 and alpha = 1.0]\n",
      "2020-01-02 14:39:50,952 Translations saved to: models/enlua_transformer/00098000.hyps.dev\n",
      "2020-01-02 14:40:24,373 test bleu:  42.52 [Beam search decoding with beam size = 5 and alpha = 1.0]\n",
      "2020-01-02 14:40:24,374 Translations saved to: models/enlua_transformer/00098000.hyps.test\n"
     ]
    }
   ],
   "source": [
    "# Train the model\n",
    "# You can press Ctrl-C to stop. And then run the next cell to save your checkpoints! \n",
    "!cd joeynmt; python3 -m joeynmt train configs/transformer_$src$tgt.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 33
    },
    "colab_type": "code",
    "id": "MBoDS09JM807",
    "outputId": "ec06ffd7-cf03-489d-f723-e8b1dfd09daf"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cp: cannot create symbolic link '/content/drive/My Drive/masakhane/en-lua-baseline/models/enlua_transformer/best.ckpt': Operation not supported\n"
     ]
    }
   ],
   "source": [
    "# Copy the created models from the notebook storage to google drive for persistant storage \n",
    "!cp -r joeynmt/models/${src}${tgt}_transformer/* \"$gdrive_path/models/${src}${tgt}_transformer/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AYCOImAVVCFJ"
   },
   "outputs": [],
   "source": [
    "# Copy the created models from the notebook storage to google drive for persistant storage \n",
    "!cp joeynmt/models/${src}${tgt}_transformer/best.ckpt \"$gdrive_path/models/${src}${tgt}_transformer/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 383
    },
    "colab_type": "code",
    "id": "cSoGZWeeUFob",
    "outputId": "1f251652-bf49-41b1-9440-c433523c7af0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00098000.hyps.dev   28000.hyps\t48000.hyps  68000.hyps\t88000.hyps\n",
      "00098000.hyps.test  29000.hyps\t49000.hyps  69000.hyps\t89000.hyps\n",
      "10000.hyps\t    30000.hyps\t50000.hyps  70000.hyps\t90000.hyps\n",
      "1000.hyps\t    3000.hyps\t5000.hyps   7000.hyps\t9000.hyps\n",
      "11000.hyps\t    31000.hyps\t51000.hyps  71000.hyps\t91000.hyps\n",
      "12000.hyps\t    32000.hyps\t52000.hyps  72000.hyps\t92000.hyps\n",
      "13000.hyps\t    33000.hyps\t53000.hyps  73000.hyps\t93000.hyps\n",
      "14000.hyps\t    34000.hyps\t54000.hyps  74000.hyps\t94000.hyps\n",
      "15000.hyps\t    35000.hyps\t55000.hyps  75000.hyps\t95000.hyps\n",
      "16000.hyps\t    36000.hyps\t56000.hyps  76000.hyps\t96000.ckpt\n",
      "17000.hyps\t    37000.hyps\t57000.hyps  77000.hyps\t96000.hyps\n",
      "18000.hyps\t    38000.hyps\t58000.hyps  78000.hyps\t97000.ckpt\n",
      "19000.hyps\t    39000.hyps\t59000.hyps  79000.hyps\t97000.hyps\n",
      "20000.hyps\t    40000.hyps\t60000.hyps  80000.hyps\t98000.ckpt\n",
      "2000.hyps\t    4000.hyps\t6000.hyps   8000.hyps\t98000.hyps\n",
      "21000.hyps\t    41000.hyps\t61000.hyps  81000.hyps\tbest.ckpt\n",
      "22000.hyps\t    42000.hyps\t62000.hyps  82000.hyps\tconfig.yaml\n",
      "23000.hyps\t    43000.hyps\t63000.hyps  83000.hyps\tsrc_vocab.txt\n",
      "24000.hyps\t    44000.hyps\t64000.hyps  84000.hyps\ttensorboard\n",
      "25000.hyps\t    45000.hyps\t65000.hyps  85000.hyps\ttrain.log\n",
      "26000.hyps\t    46000.hyps\t66000.hyps  86000.hyps\ttrg_vocab.txt\n",
      "27000.hyps\t    47000.hyps\t67000.hyps  87000.hyps\tvalidations.txt\n"
     ]
    }
   ],
   "source": [
    "!ls joeynmt/models/${src}${tgt}_transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "n94wlrCjVc17",
    "outputId": "ecdeff40-cc4f-4dc6-ec79-ae591b7eb1f8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Steps: 1000\tLoss: 102899.78906\tPPL: 53.84991\tbleu: 1.12929\tLR: 0.00030000\t*\n",
      "Steps: 2000\tLoss: 84754.05469\tPPL: 26.66253\tbleu: 2.72745\tLR: 0.00030000\t*\n",
      "Steps: 3000\tLoss: 75980.53906\tPPL: 18.97998\tbleu: 5.21574\tLR: 0.00030000\t*\n",
      "Steps: 4000\tLoss: 70149.67969\tPPL: 15.14250\tbleu: 7.78372\tLR: 0.00030000\t*\n",
      "Steps: 5000\tLoss: 66669.04688\tPPL: 13.23242\tbleu: 9.80320\tLR: 0.00030000\t*\n",
      "Steps: 6000\tLoss: 62869.32031\tPPL: 11.42123\tbleu: 11.31751\tLR: 0.00030000\t*\n",
      "Steps: 7000\tLoss: 60245.42188\tPPL: 10.31736\tbleu: 12.77085\tLR: 0.00030000\t*\n",
      "Steps: 8000\tLoss: 58000.27344\tPPL: 9.45793\tbleu: 14.21334\tLR: 0.00030000\t*\n",
      "Steps: 9000\tLoss: 56034.40234\tPPL: 8.76440\tbleu: 15.74813\tLR: 0.00030000\t*\n",
      "Steps: 10000\tLoss: 54509.39453\tPPL: 8.26163\tbleu: 16.10168\tLR: 0.00030000\t*\n",
      "Steps: 11000\tLoss: 53012.06641\tPPL: 7.79605\tbleu: 17.35481\tLR: 0.00030000\t*\n",
      "Steps: 12000\tLoss: 51737.62109\tPPL: 7.42050\tbleu: 18.25946\tLR: 0.00030000\t*\n",
      "Steps: 13000\tLoss: 50633.38281\tPPL: 7.10977\tbleu: 18.31056\tLR: 0.00030000\t*\n",
      "Steps: 14000\tLoss: 49630.21484\tPPL: 6.83877\tbleu: 18.93929\tLR: 0.00030000\t*\n",
      "Steps: 15000\tLoss: 48967.03516\tPPL: 6.66532\tbleu: 19.52296\tLR: 0.00030000\t*\n",
      "Steps: 16000\tLoss: 48093.68359\tPPL: 6.44359\tbleu: 19.83276\tLR: 0.00030000\t*\n",
      "Steps: 17000\tLoss: 47514.90234\tPPL: 6.30072\tbleu: 19.94566\tLR: 0.00030000\t*\n",
      "Steps: 18000\tLoss: 47023.11328\tPPL: 6.18182\tbleu: 20.35781\tLR: 0.00030000\t*\n",
      "Steps: 19000\tLoss: 46357.82031\tPPL: 6.02453\tbleu: 21.08363\tLR: 0.00030000\t*\n",
      "Steps: 20000\tLoss: 45906.89062\tPPL: 5.92021\tbleu: 21.19038\tLR: 0.00030000\t*\n",
      "Steps: 21000\tLoss: 45281.07812\tPPL: 5.77841\tbleu: 21.88843\tLR: 0.00030000\t*\n",
      "Steps: 22000\tLoss: 44715.19531\tPPL: 5.65312\tbleu: 21.79593\tLR: 0.00030000\t*\n",
      "Steps: 23000\tLoss: 45031.08203\tPPL: 5.72272\tbleu: 21.68914\tLR: 0.00030000\t\n",
      "Steps: 24000\tLoss: 44329.76172\tPPL: 5.56934\tbleu: 22.31552\tLR: 0.00030000\t*\n",
      "Steps: 25000\tLoss: 43693.29297\tPPL: 5.43370\tbleu: 22.44743\tLR: 0.00030000\t*\n",
      "Steps: 26000\tLoss: 43213.68359\tPPL: 5.33367\tbleu: 22.51617\tLR: 0.00030000\t*\n",
      "Steps: 27000\tLoss: 42980.87109\tPPL: 5.28579\tbleu: 22.79999\tLR: 0.00030000\t*\n",
      "Steps: 28000\tLoss: 42610.90625\tPPL: 5.21057\tbleu: 23.38897\tLR: 0.00030000\t*\n",
      "Steps: 29000\tLoss: 42535.65234\tPPL: 5.19541\tbleu: 23.63879\tLR: 0.00030000\t*\n",
      "Steps: 30000\tLoss: 42312.28516\tPPL: 5.15064\tbleu: 24.02978\tLR: 0.00030000\t*\n",
      "Steps: 31000\tLoss: 41830.42969\tPPL: 5.05539\tbleu: 23.96090\tLR: 0.00030000\t*\n",
      "Steps: 32000\tLoss: 41604.14453\tPPL: 5.01127\tbleu: 24.14225\tLR: 0.00030000\t*\n",
      "Steps: 33000\tLoss: 41377.32031\tPPL: 4.96743\tbleu: 24.10188\tLR: 0.00030000\t*\n",
      "Steps: 34000\tLoss: 41289.39062\tPPL: 4.95054\tbleu: 24.51540\tLR: 0.00030000\t*\n",
      "Steps: 35000\tLoss: 40840.16797\tPPL: 4.86513\tbleu: 24.52375\tLR: 0.00030000\t*\n",
      "Steps: 36000\tLoss: 40723.89844\tPPL: 4.84327\tbleu: 24.55918\tLR: 0.00030000\t*\n",
      "Steps: 37000\tLoss: 40494.31250\tPPL: 4.80038\tbleu: 24.76842\tLR: 0.00030000\t*\n",
      "Steps: 38000\tLoss: 40365.34375\tPPL: 4.77646\tbleu: 24.97502\tLR: 0.00030000\t*\n",
      "Steps: 39000\tLoss: 40251.47656\tPPL: 4.75544\tbleu: 24.51077\tLR: 0.00030000\t*\n",
      "Steps: 40000\tLoss: 40045.08203\tPPL: 4.71757\tbleu: 24.92373\tLR: 0.00030000\t*\n",
      "Steps: 41000\tLoss: 39896.39062\tPPL: 4.69047\tbleu: 25.37874\tLR: 0.00030000\t*\n",
      "Steps: 42000\tLoss: 39901.09766\tPPL: 4.69133\tbleu: 25.04666\tLR: 0.00030000\t\n",
      "Steps: 43000\tLoss: 39630.29297\tPPL: 4.64237\tbleu: 26.34869\tLR: 0.00030000\t*\n",
      "Steps: 44000\tLoss: 39337.25781\tPPL: 4.58997\tbleu: 25.99776\tLR: 0.00030000\t*\n",
      "Steps: 45000\tLoss: 39216.50391\tPPL: 4.56855\tbleu: 25.61746\tLR: 0.00030000\t*\n",
      "Steps: 46000\tLoss: 39148.97656\tPPL: 4.55661\tbleu: 25.63757\tLR: 0.00030000\t*\n",
      "Steps: 47000\tLoss: 38903.24219\tPPL: 4.51344\tbleu: 25.87739\tLR: 0.00030000\t*\n",
      "Steps: 48000\tLoss: 38740.63281\tPPL: 4.48510\tbleu: 25.96574\tLR: 0.00030000\t*\n",
      "Steps: 49000\tLoss: 38619.57031\tPPL: 4.46411\tbleu: 26.19643\tLR: 0.00030000\t*\n",
      "Steps: 50000\tLoss: 38614.63672\tPPL: 4.46326\tbleu: 26.36254\tLR: 0.00030000\t*\n",
      "Steps: 51000\tLoss: 38496.93750\tPPL: 4.44296\tbleu: 26.03110\tLR: 0.00030000\t*\n",
      "Steps: 52000\tLoss: 38251.83203\tPPL: 4.40097\tbleu: 26.70205\tLR: 0.00030000\t*\n",
      "Steps: 53000\tLoss: 38213.93750\tPPL: 4.39451\tbleu: 26.50597\tLR: 0.00030000\t*\n",
      "Steps: 54000\tLoss: 38168.25391\tPPL: 4.38674\tbleu: 26.46199\tLR: 0.00030000\t*\n",
      "Steps: 55000\tLoss: 38101.63281\tPPL: 4.37544\tbleu: 26.57891\tLR: 0.00030000\t*\n",
      "Steps: 56000\tLoss: 37680.60547\tPPL: 4.30465\tbleu: 26.72689\tLR: 0.00030000\t*\n",
      "Steps: 57000\tLoss: 37818.78125\tPPL: 4.32776\tbleu: 26.94940\tLR: 0.00030000\t\n",
      "Steps: 58000\tLoss: 37648.64844\tPPL: 4.29933\tbleu: 26.54304\tLR: 0.00030000\t*\n",
      "Steps: 59000\tLoss: 37401.43750\tPPL: 4.25835\tbleu: 26.85918\tLR: 0.00030000\t*\n",
      "Steps: 60000\tLoss: 37386.17188\tPPL: 4.25583\tbleu: 27.15915\tLR: 0.00030000\t*\n",
      "Steps: 61000\tLoss: 37529.09375\tPPL: 4.27946\tbleu: 26.74895\tLR: 0.00030000\t\n",
      "Steps: 62000\tLoss: 37354.71875\tPPL: 4.25065\tbleu: 27.29078\tLR: 0.00030000\t*\n",
      "Steps: 63000\tLoss: 37281.22656\tPPL: 4.23857\tbleu: 27.25590\tLR: 0.00030000\t*\n",
      "Steps: 64000\tLoss: 37131.86719\tPPL: 4.21411\tbleu: 27.58058\tLR: 0.00030000\t*\n",
      "Steps: 65000\tLoss: 36952.87891\tPPL: 4.18499\tbleu: 27.33229\tLR: 0.00030000\t*\n",
      "Steps: 66000\tLoss: 37028.41406\tPPL: 4.19726\tbleu: 27.15411\tLR: 0.00030000\t\n",
      "Steps: 67000\tLoss: 36970.87891\tPPL: 4.18791\tbleu: 27.09968\tLR: 0.00030000\t\n",
      "Steps: 68000\tLoss: 36757.10156\tPPL: 4.15337\tbleu: 27.13710\tLR: 0.00030000\t*\n",
      "Steps: 69000\tLoss: 36698.92578\tPPL: 4.14402\tbleu: 27.88980\tLR: 0.00030000\t*\n",
      "Steps: 70000\tLoss: 36668.21094\tPPL: 4.13910\tbleu: 27.75302\tLR: 0.00030000\t*\n",
      "Steps: 71000\tLoss: 36720.47656\tPPL: 4.14749\tbleu: 27.42695\tLR: 0.00030000\t\n",
      "Steps: 72000\tLoss: 36541.98828\tPPL: 4.11891\tbleu: 27.61069\tLR: 0.00030000\t*\n",
      "Steps: 73000\tLoss: 36489.25391\tPPL: 4.11050\tbleu: 27.50095\tLR: 0.00030000\t*\n",
      "Steps: 74000\tLoss: 36391.75000\tPPL: 4.09501\tbleu: 27.68670\tLR: 0.00030000\t*\n",
      "Steps: 75000\tLoss: 36205.14844\tPPL: 4.06551\tbleu: 27.54025\tLR: 0.00030000\t*\n",
      "Steps: 76000\tLoss: 36112.93359\tPPL: 4.05101\tbleu: 28.06918\tLR: 0.00030000\t*\n",
      "Steps: 77000\tLoss: 36160.38281\tPPL: 4.05847\tbleu: 28.27255\tLR: 0.00030000\t\n",
      "Steps: 78000\tLoss: 36083.96875\tPPL: 4.04647\tbleu: 27.94955\tLR: 0.00030000\t*\n",
      "Steps: 79000\tLoss: 36122.85547\tPPL: 4.05257\tbleu: 27.49365\tLR: 0.00030000\t\n",
      "Steps: 80000\tLoss: 36005.58203\tPPL: 4.03420\tbleu: 28.09085\tLR: 0.00030000\t*\n",
      "Steps: 81000\tLoss: 35966.78906\tPPL: 4.02814\tbleu: 28.08345\tLR: 0.00030000\t*\n",
      "Steps: 82000\tLoss: 35764.50000\tPPL: 3.99670\tbleu: 28.20023\tLR: 0.00030000\t*\n",
      "Steps: 83000\tLoss: 35774.58594\tPPL: 3.99826\tbleu: 28.06374\tLR: 0.00030000\t\n",
      "Steps: 84000\tLoss: 35871.28906\tPPL: 4.01327\tbleu: 28.32058\tLR: 0.00030000\t\n",
      "Steps: 85000\tLoss: 35723.01562\tPPL: 3.99028\tbleu: 28.41656\tLR: 0.00030000\t*\n",
      "Steps: 86000\tLoss: 35697.04297\tPPL: 3.98627\tbleu: 28.27429\tLR: 0.00030000\t*\n",
      "Steps: 87000\tLoss: 35712.06641\tPPL: 3.98859\tbleu: 28.19326\tLR: 0.00030000\t\n",
      "Steps: 88000\tLoss: 35649.17578\tPPL: 3.97888\tbleu: 28.45988\tLR: 0.00030000\t*\n",
      "Steps: 89000\tLoss: 35510.93359\tPPL: 3.95763\tbleu: 28.31753\tLR: 0.00030000\t*\n",
      "Steps: 90000\tLoss: 35427.69141\tPPL: 3.94489\tbleu: 29.00604\tLR: 0.00030000\t*\n",
      "Steps: 91000\tLoss: 35440.07031\tPPL: 3.94678\tbleu: 28.77265\tLR: 0.00030000\t\n",
      "Steps: 92000\tLoss: 35333.93750\tPPL: 3.93059\tbleu: 28.62932\tLR: 0.00030000\t*\n",
      "Steps: 93000\tLoss: 35402.22266\tPPL: 3.94100\tbleu: 28.12490\tLR: 0.00030000\t\n",
      "Steps: 94000\tLoss: 35315.73047\tPPL: 3.92782\tbleu: 28.77303\tLR: 0.00030000\t*\n",
      "Steps: 95000\tLoss: 35239.15234\tPPL: 3.91618\tbleu: 28.75989\tLR: 0.00030000\t*\n",
      "Steps: 96000\tLoss: 35210.66797\tPPL: 3.91187\tbleu: 29.02997\tLR: 0.00030000\t*\n",
      "Steps: 97000\tLoss: 35144.55469\tPPL: 3.90186\tbleu: 28.98882\tLR: 0.00030000\t*\n",
      "Steps: 98000\tLoss: 35133.78125\tPPL: 3.90023\tbleu: 28.92395\tLR: 0.00030000\t*\n"
     ]
    }
   ],
   "source": [
    "# Output our validation accuracy\n",
    "! cat \"$gdrive_path/models/${src}${tgt}_transformer/validations.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 66
    },
    "colab_type": "code",
    "id": "66WhRE9lIhoD",
    "outputId": "b24577f6-df42-4ed7-8bcb-17ba1a9aef58"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01-02 14:49:38,856 Hello! This is Joey-NMT.\n",
      "2020-01-02 14:50:04,491  dev bleu:  29.96 [Beam search decoding with beam size = 5 and alpha = 1.0]\n",
      "2020-01-02 14:50:37,607 test bleu:  42.52 [Beam search decoding with beam size = 5 and alpha = 1.0]\n"
     ]
    }
   ],
   "source": [
    "# Test our model\n",
    "! cd joeynmt; python3 -m joeynmt test \"$gdrive_path/models/${src}${tgt}_transformer/config.yaml\""
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "Copie de starter_notebook.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}