{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Igc5itf-xMGj"
   },
   "source": [
    "# Masakhane - Machine Translation for African Languages (Using JoeyNMT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x4fXCKCf36IK"
   },
   "source": [
    "## Note before beginning:\n",
    "### - The idea is that you should be able to make minimal changes to this in order to get SOME result for your own translation corpus. \n",
    "\n",
    "### - The tl;dr: Go to the **\"TODO\"** comments which will tell you what to update to get up and running\n",
    "\n",
    "### - If you actually want to have a clue what you're doing, read the text and peek at the links\n",
    "\n",
    "### - With 100 epochs, it should take around 7 hours to run in Google Colab\n",
    "\n",
    "### - Once you've gotten a result for your language, please attach and email your notebook that generated it to masakhanetranslation@gmail.com\n",
    "\n",
    "### - If you care enough and get a chance, doing a brief background on your language would be amazing. See examples in  [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "l929HimrxS0a"
   },
   "source": [
    "## Retrieve your data & make a parallel corpus\n",
    "\n",
    "If you are wanting to use the JW300 data referenced on the Masakhane website or in our GitHub repo, you can use `opus-tools` to convert the data into a convenient format. `opus_read` from that package provides a convenient tool for reading the native aligned XML files and to convert them to TMX format. The tool can also be used to fetch relevant files from OPUS on the fly and to filter the data as necessary. [Read the documentation](https://pypi.org/project/opustools-pkg/) for more details.\n",
    "\n",
    "Once you have your corpus files in TMX format (an xml structure which will include the sentences in your target language and your source language in a single file), we recommend reading them into a pandas dataframe. Thankfully, Jade wrote a silly `tmx2dataframe` package which converts your tmx file to a pandas dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "oGRmDELn7Az0"
   },
   "outputs": [],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "Cn3tgQLzUxwn"
   },
   "outputs": [],
   "source": [
    "# TODO: Set your source and target languages. Keep in mind, these traditionally use language codes as found here:\n",
    "# These will also become the suffix's of all vocab and corpus files used throughout\n",
    "import os\n",
    "source_language = \"en\"\n",
    "target_language = \"sw\" \n",
    "lc = False  # If True, lowercase the data.\n",
    "seed = 42  # Random seed for shuffling.\n",
    "tag = \"baseline\" # Give a unique name to your folder - this is to ensure you don't rewrite any models you've already submitted\n",
    "\n",
    "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n",
    "os.environ[\"tgt\"] = target_language\n",
    "os.environ[\"tag\"] = tag\n",
    "\n",
    "# This will save it to a folder in our gdrive instead!\n",
    "# !mkdir -p \"/content/drive/My Drive/masakhane/$src-$tgt-$tag\"\n",
    "# os.environ[\"gdrive_path\"] = \"/content/drive/My Drive/masakhane/%s-%s-%s\" % (source_language, target_language, tag)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "kBSgJHEw7Nvx"
   },
   "outputs": [],
   "source": [
    "# !echo $gdrive_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "gA75Fs9ys8Y9"
   },
   "outputs": [],
   "source": [
    "# Install opus-tools\n",
    "# ! pip install opustools-pkg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xq-tDZVks7ZD"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Alignment file /proj/nlpl/data/OPUS/JW300/latest/xml/en-sw.xml.gz not found. The following files are available for downloading:\n",
      "\n",
      "   8 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en-sw.xml.gz\n",
      " 263 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/en.zip\n",
      "  94 MB https://object.pouta.csc.fi/OPUS-JW300/v1/xml/sw.zip\n",
      "\n",
      " 365 MB Total size\n",
      "./JW300_latest_xml_en-sw.xml.gz ... 100% of 8 MB\n",
      "./JW300_latest_xml_en.zip ... 100% of 263 MB\n",
      "./JW300_latest_xml_sw.zip ... 100% of 94 MB\n"
     ]
    }
   ],
   "source": [
    "# Downloading our corpus\n",
    "! opus_read -d JW300 -s $src -t $tgt -wm moses -w jw300.$src jw300.$tgt -q\n",
    "\n",
    "# extract the corpus file\n",
    "! gunzip JW300_latest_xml_$src-$tgt.xml.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n48GDRnP8y2G"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-10-30 04:02:19--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.192.133, 151.101.128.133, 151.101.64.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.192.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 277791 (271K) [text/plain]\n",
      "Saving to: ‘test.en-any.en’\n",
      "\n",
      "test.en-any.en      100%[===================>] 271.28K   298KB/s    in 0.9s    \n",
      "\n",
      "2019-10-30 04:02:21 (298 KB/s) - ‘test.en-any.en’ saved [277791/277791]\n",
      "\n",
      "--2019-10-30 04:02:22--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-sw.en\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.192.133, 151.101.128.133, 151.101.64.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.192.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 206336 (202K) [text/plain]\n",
      "Saving to: ‘test.en-sw.en’\n",
      "\n",
      "test.en-sw.en       100%[===================>] 201.50K   272KB/s    in 0.7s    \n",
      "\n",
      "2019-10-30 04:02:24 (272 KB/s) - ‘test.en-sw.en’ saved [206336/206336]\n",
      "\n",
      "--2019-10-30 04:02:24--  https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-sw.sw\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.192.133, 151.101.128.133, 151.101.64.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.192.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 214836 (210K) [text/plain]\n",
      "Saving to: ‘test.en-sw.sw’\n",
      "\n",
      "test.en-sw.sw       100%[===================>] 209.80K   280KB/s    in 0.7s    \n",
      "\n",
      "2019-10-30 04:02:26 (280 KB/s) - ‘test.en-sw.sw’ saved [214836/214836]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Download the global test set.\n",
    "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-any.en\n",
    "  \n",
    "# And the specific test set for this language pair.\n",
    "os.environ[\"trg\"] = target_language \n",
    "os.environ[\"src\"] = source_language \n",
    "\n",
    "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.en \n",
    "! mv test.en-$trg.en test.en\n",
    "! wget https://raw.githubusercontent.com/juliakreutzer/masakhane/master/jw300_utils/test/test.en-$trg.$trg \n",
    "! mv test.en-$trg.$trg test.$trg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NqDG-CI28y2L"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 3571 global test sentences to filter from the training/dev data.\n"
     ]
    }
   ],
   "source": [
    "# Read the test data to filter from train and dev splits.\n",
    "# Store english portion in set for quick filtering checks.\n",
    "en_test_sents = set()\n",
    "filter_test_sents = \"test.en-any.en\"\n",
    "j = 0\n",
    "with open(filter_test_sents) as f:\n",
    "  for line in f:\n",
    "    en_test_sents.add(line.strip())\n",
    "    j += 1\n",
    "print('Loaded {} global test sentences to filter from the training/dev data.'.format(j))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3CNdwLBCfSIl"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded data and skipped 6478/979526 lines since contained in test set.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source_sentence</th>\n",
       "      <th>target_sentence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>“ Look !</td>\n",
       "      <td>“ Tazama !</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>I Am Making All Things New ”</td>\n",
       "      <td>Mimi Ninafanya Vitu Vyote Kuwa Vipya ”</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The above is a promise from God that has been ...</td>\n",
       "      <td>Iliyopo juu ni ahadi itokayo kwa Mungu ambayo ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     source_sentence  \\\n",
       "0                                           “ Look !   \n",
       "1                       I Am Making All Things New ”   \n",
       "2  The above is a promise from God that has been ...   \n",
       "\n",
       "                                     target_sentence  \n",
       "0                                         “ Tazama !  \n",
       "1             Mimi Ninafanya Vitu Vyote Kuwa Vipya ”  \n",
       "2  Iliyopo juu ni ahadi itokayo kwa Mungu ambayo ...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# TMX file to dataframe\n",
    "source_file = 'jw300.' + source_language\n",
    "target_file = 'jw300.' + target_language\n",
    "\n",
    "source = []\n",
    "target = []\n",
    "skip_lines = []  # Collect the line numbers of the source portion to skip the same lines for the target portion.\n",
    "with open(source_file) as f:\n",
    "    for i, line in enumerate(f):\n",
    "        # Skip sentences that are contained in the test set.\n",
    "        if line.strip() not in en_test_sents:\n",
    "            source.append(line.strip())\n",
    "        else:\n",
    "            skip_lines.append(i)             \n",
    "with open(target_file) as f:\n",
    "    for j, line in enumerate(f):\n",
    "        # Only add to corpus if corresponding source was not skipped.\n",
    "        if j not in skip_lines:\n",
    "            target.append(line.strip())\n",
    "    \n",
    "print('Loaded data and skipped {}/{} lines since contained in test set.'.format(len(skip_lines), i))\n",
    "    \n",
    "df = pd.DataFrame(zip(source, target), columns=['source_sentence', 'target_sentence'])\n",
    "df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YkuK3B4p2AkN"
   },
   "source": [
    "## Pre-processing and export\n",
    "\n",
    "It is generally a good idea to remove duplicate translations and conflicting translations from the corpus. In practice, these public corpora include some number of these that need to be cleaned.\n",
    "\n",
    "In addition we will split our data into dev/test/train and export to the filesystem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "M_2ouEOH1_1q"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ks/.local/lib/python3.6/site-packages/pandas/core/generic.py:6786: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  self._update_inplace(new_data)\n",
      "/home/ks/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:8: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  \n",
      "/home/ks/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:9: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  if __name__ == '__main__':\n",
      "/home/ks/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:12: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  if sys.path[0] == '':\n",
      "/home/ks/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:13: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  del sys.path[0]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "# drop duplicate translations\n",
    "df_pp = df.drop_duplicates()\n",
    "\n",
    "#drop empty lines (alp)\n",
    "df_pp['source_sentence'].replace('', np.nan, inplace=True)\n",
    "df_pp['target_sentence'].replace('', np.nan, inplace=True)\n",
    "df_pp.dropna(subset=['source_sentence'], inplace=True)\n",
    "df_pp.dropna(subset=['target_sentence'], inplace=True)\n",
    "\n",
    "# drop conflicting translations\n",
    "df_pp.drop_duplicates(subset='source_sentence', inplace=True)\n",
    "df_pp.drop_duplicates(subset='target_sentence', inplace=True)\n",
    "\n",
    "# Shuffle the data to remove bias in dev set selection.\n",
    "df_pp = df_pp.sample(frac=1, random_state=seed).reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hxxBOCA-xXhy"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==> train.en <==\n",
      "To make matters worse , I made the big mistake of opening my mouth to speak only when I was sure of my French .\n",
      "Initially , the Waldenses withdrew from Roman Catholicism , the State religion in Central Europe .\n",
      "The advantages of allowing more sunshine into a building have not gone unnoticed among retailers either .\n",
      "Not even the joy from events such as Bangkok’s bicentennial celebration ​ — floral parades , historical exhibits , royal ceremonies , classical dances , and fireworks — ​ can blot out the sadness felt in this city .\n",
      "Century Dictionary\n",
      "The apostle Paul also warned against “ the trickery of men . ”\n",
      "The predicted end of this diabolic system of things will not come before the Kingdom witness has been given in all the earth as a grand finale to the activity of the followers of Jesus Christ , the Witnesses of Jehovah .\n",
      "Women as well as men are given many privileges by Jehovah .\n",
      "As the study progressed , I began to see the need for making big changes in my personality .\n",
      "More than that , he was willing to do whatever his Father asked of him .\n",
      "\n",
      "==> train.sw <==\n",
      "Jambo baya hata zaidi nililofanya lilikuwa kuzungumza lugha hiyo wakati tu nilipokuwa na uhakika kwamba Kifaransa changu ni sahihi .\n",
      "Mwanzoni , wafuasi wa Waldo walijitenga na Kanisa Katoliki , lililokuwa dini rasmi huko Ulaya ya Kati .\n",
      "Mafaa ya kuruhusu nuru zaidi ya jua ndani ya jengo hayajakosa kuonwa na wenye maduka ya rejareja .\n",
      "36 ] Hata ile shangwe itokanayo na sherehe kama ile ya kusherehekea miaka mia mbili ya Bangkok — mwandamano wa watu wenye maua , maonyesho ya kihistoria , sherehe za kifalme , dansi za kitamaduni , na fataki — haiwezi kuondoa huzuni iliyo katika jiji hilo .\n",
      "Century Dictionary\n",
      "Mtume Paulo alionya pia dhidi ya “ hadaa ya watu . ”\n",
      "Mwisho uliotabiriwa wa mfumo huu wa mambo ya kidiabolo ( kiibilisi ) hautakuja mpaka ushahidi wa Ufalme utolewe katika dunia yote kuwa upeo mtukufu kwa utendaji wa wafuasi wa Yesu Kristo , wale Mashahidi wa Yehova .\n",
      "Yehova amewapa wanawake na pia wanaume mapendeleo mengi .\n",
      "Nilipoendelea kujifunza , niliona kwamba nilihitaji kufanya mabadiliko makubwa katika utu wangu .\n",
      "Kwa kuongezea , yeye alikuwa na nia ya kufanya lolote lile ambalo Baba yake alimwomba afanye .\n",
      "==> dev.en <==\n",
      "She became pregnant at the age of 17 .\n",
      "The Bible does not promote credulity .\n",
      "What are some things that cause anxiety , and what can anxiety result in ?\n",
      "Lots of hugs and kisses .\n",
      "How did Jehovah view Job ?\n",
      "While Aaron undoubtedly gathered manna into a jar at that time , the depositing of it before the Testimony had to wait until Moses made the Ark and placed the tablets in it .\n",
      "She was part of a strong congregation in Vlorë in the early 1930 ’ s .\n",
      "In the present , it is a different matter .\n",
      "Although at times singing is a means of establishing territory , it is also used in a large way to attract potential mates .\n",
      "□ What Scriptures authorize the giving of authority to the other sheep in Jehovah’s organization ?\n",
      "\n",
      "==> dev.sw <==\n",
      "Alipata mimba akiwa na umri wa miaka 17 .\n",
      "Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "Kukumbatia kwingi na busu nyingi .\n",
      "Yehova alimwonaje Ayubu ?\n",
      "Ingawa ni wazi kwamba Haruni aliweka mana katika mtungi wakati huo , ilibidi angoje Musa alitengeneze Sanduku na aingize yale mabamba kabla ya kuweka mtungi huo mbele ya ule Ushuhuda .\n",
      "Alishirikiana na kutaniko lenye bidii huko Vlorë katika miaka ya mapema ya 1930 .\n",
      "Kabla ya hapo si rahisi kuzijua .\n",
      "Ijapokuwa nyakati nyingine ndege huimba ili kumiliki eneo fulani , wao hufanya hivyo hasa ili kuwavutia wapenzi .\n",
      "□ Ni Maandiko yapi yaruhusuyo kupewa kwa mamlaka kondoo wengine katika tengenezo la Yehova ?\n"
     ]
    }
   ],
   "source": [
    "# This section does the split between train/dev for the parallel corpora then saves them as separate files\n",
    "# We use 1000 dev test and the given test set.\n",
    "import csv\n",
    "\n",
    "# Do the split between dev/train and create parallel corpora\n",
    "num_dev_patterns = 1000\n",
    "\n",
    "# Optional: lower case the corpora - this will make it easier to generalize, but without proper casing.\n",
    "if lc:  # Julia: making lowercasing optional\n",
    "    df_pp[\"source_sentence\"] = df_pp[\"source_sentence\"].str.lower()\n",
    "    df_pp[\"target_sentence\"] = df_pp[\"target_sentence\"].str.lower()\n",
    "\n",
    "# Julia: test sets are already generated\n",
    "dev = df_pp.tail(num_dev_patterns) # Herman: Error in original\n",
    "stripped = df_pp.drop(df_pp.tail(num_dev_patterns).index)\n",
    "\n",
    "with open(\"train.\"+source_language, \"w\") as src_file, open(\"train.\"+target_language, \"w\") as trg_file:\n",
    "  for index, row in stripped.iterrows():\n",
    "    src_file.write(row[\"source_sentence\"]+\"\\n\")\n",
    "    trg_file.write(row[\"target_sentence\"]+\"\\n\")\n",
    "    \n",
    "with open(\"dev.\"+source_language, \"w\") as src_file, open(\"dev.\"+target_language, \"w\") as trg_file:\n",
    "  for index, row in dev.iterrows():\n",
    "    src_file.write(row[\"source_sentence\"]+\"\\n\")\n",
    "    trg_file.write(row[\"target_sentence\"]+\"\\n\")\n",
    "\n",
    "#stripped[[\"source_sentence\"]].to_csv(\"train.\"+source_language, header=False, index=False)  # Herman: Added `header=False` everywhere\n",
    "#stripped[[\"target_sentence\"]].to_csv(\"train.\"+target_language, header=False, index=False)  # Julia: Problematic handling of quotation marks.\n",
    "\n",
    "#dev[[\"source_sentence\"]].to_csv(\"dev.\"+source_language, header=False, index=False)\n",
    "#dev[[\"target_sentence\"]].to_csv(\"dev.\"+target_language, header=False, index=False)\n",
    "\n",
    "\n",
    "# Doublecheck the format below. There should be no extra quotation marks or weird characters.\n",
    "! head train.*\n",
    "! head dev.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "epeCydmCyS8X"
   },
   "source": [
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "## Installation of JoeyNMT\n",
    "\n",
    "JoeyNMT is a simple, minimalist NMT package which is useful for learning and teaching. Check out the documentation for JoeyNMT [here](https://joeynmt.readthedocs.io)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "iBRMm4kMxZ8L"
   },
   "outputs": [],
   "source": [
    "# Install JoeyNMT\n",
    "! git clone https://github.com/joeynmt/joeynmt.git\n",
    "! cd joeynmt; pip3 install ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AaE77Tcppex9"
   },
   "source": [
    "# Preprocessing the Data into Subword BPE Tokens\n",
    "\n",
    "- One of the most powerful improvements for agglutinative languages (a feature of most Bantu languages) is using BPE tokenization [ (Sennrich, 2015) ](https://arxiv.org/abs/1508.07909).\n",
    "\n",
    "- It was also shown that by optimizing the umber of BPE codes we significantly improve results for low-resourced languages [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021) [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)\n",
    "\n",
    "- Below we have the scripts for doing BPE tokenization of our data. We use 4000 tokens as recommended by [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021). You do not need to change anything. Simply running the below will be suitable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# One of the huge boosts in NMT performance was to use a different method of tokenizing. \n",
    "# Usually, NMT would tokenize by words. However, using a method called BPE gave amazing boosts to performance\n",
    "\n",
    "# Do subword NMT\n",
    "from os import path\n",
    "os.environ[\"src\"] = source_language # Sets them in bash as well, since we often use bash scripts\n",
    "os.environ[\"tgt\"] = target_language\n",
    "\n",
    "# Learn BPEs on the training data.\n",
    "os.environ[\"data_path\"] = path.join(\"joeynmt\", \"data\", source_language + target_language) # Herman! \n",
    "! subword-nmt learn-joint-bpe-and-vocab --input train.$src train.$tgt -s 4000 -o bpe.codes.4000 --write-vocabulary vocab.$src vocab.$tgt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Apply BPE splits to the development and test data.\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < train.$src > train.bpe.$src\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < train.$tgt > train.bpe.$tgt\n",
    "\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < dev.$src > dev.bpe.$src\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < dev.$tgt > dev.bpe.$tgt\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$src < test.$src > test.bpe.$src\n",
    "! subword-nmt apply-bpe -c bpe.codes.4000 --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bpe.codes.4000\tdev.en\t     test.bpe.sw     test.sw\t   train.en\r\n",
      "dev.bpe.en\tdev.sw\t     test.en\t     train.bpe.en  train.sw\r\n",
      "dev.bpe.sw\ttest.bpe.en  test.en-any.en  train.bpe.sw\r\n"
     ]
    }
   ],
   "source": [
    "# Create directory, move everyone we care about to the correct location\n",
    "! mkdir -p $data_path\n",
    "! cp train.* $data_path\n",
    "! cp test.* $data_path\n",
    "! cp dev.* $data_path\n",
    "! cp bpe.codes.4000 $data_path\n",
    "! ls $data_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cp: target '' is not a directory\n",
      "cp: target '' is not a directory\n",
      "cp: target '' is not a directory\n",
      "cp: cannot create regular file '': No such file or directory\n",
      "ls: cannot access '': No such file or directory\n"
     ]
    }
   ],
   "source": [
    "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n",
    "# ! cp train.* \"$gdrive_path\"\n",
    "# ! cp test.* \"$gdrive_path\"\n",
    "# ! cp dev.* \"$gdrive_path\"\n",
    "# ! cp bpe.codes.4000 \"$gdrive_path\"\n",
    "# ! ls \"$gdrive_path\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create that vocab using build_vocab\n",
    "# ! sudo chmod 777 joeynmt/scripts/build_vocab.py\n",
    "! /home/ks/code/tools/joeynmt/scripts/build_vocab.py joeynmt/data/$src$tgt/train.bpe.$src joeynmt/data/$src$tgt/train.bpe.$tgt --output_path joeynmt/data/$src$tgt/vocab.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "H-TyjtmXB1mL"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BPE Xhosa Sentences\n",
      "N@@ g@@ ao kubwa ya imani ( T@@ azama f@@ ungu la 12 hadi 14 )\n",
      "K@@ of@@ ia ya chu@@ ma ya wo@@ ko@@ vu ( T@@ azama f@@ ungu la 15 hadi 18 )\n",
      "N@@ im@@ et@@ ambua kwamba watu hu@@ it@@ ikia vizuri wanapo@@ ona uki@@ zungum@@ zia habari za Biblia kwa sha@@ uku na unapo@@ fanya yote una@@ yo@@ weza kuwasaidia . ”\n",
      "Up@@ anga wa roho ( T@@ azama f@@ ungu la 19 na 20 )\n",
      "Kwa msaada wa Yehova tunaweza kus@@ im@@ ama im@@ ara na kum@@ p@@ inga !\n",
      "Combined BPE Vocab\n",
      "о@@\n",
      "ḥ\n",
      "е@@\n",
      "с@@\n",
      "и\n",
      "х@@\n",
      "і@@\n",
      "д@@\n",
      "і\n",
      "➊\n"
     ]
    }
   ],
   "source": [
    "# Some output\n",
    "! echo \"BPE Xhosa Sentences\"\n",
    "! tail -n 5 test.bpe.$tgt\n",
    "! echo \"Combined BPE Vocab\"\n",
    "! tail -n 10 joeynmt/data/$src$tgt/vocab.txt  # Herman"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "IlMitUHR8Qy-"
   },
   "outputs": [],
   "source": [
    "# Also move everything we care about to a mounted location in google drive (relevant if running in colab) at gdrive_path\n",
    "! cp train.* \"$gdrive_path\"\n",
    "! cp test.* \"$gdrive_path\"\n",
    "! cp dev.* \"$gdrive_path\"\n",
    "! cp bpe.codes.4000 \"$gdrive_path\"\n",
    "! ls \"$gdrive_path\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Ixmzi60WsUZ8"
   },
   "source": [
    "# Creating the JoeyNMT Config\n",
    "\n",
    "JoeyNMT requires a yaml config. We provide a template below. We've also set a number of defaults with it, that you may play with!\n",
    "\n",
    "- We used Transformer architecture \n",
    "- We set our dropout to reasonably high: 0.3 (recommended in  [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021))\n",
    "\n",
    "Things worth playing with:\n",
    "- The batch size (also recommended to change for low-resourced languages)\n",
    "- The number of epochs (we've set it at 30 just so it runs in about an hour, for testing purposes)\n",
    "- The decoder options (beam_size, alpha)\n",
    "- Evaluation metrics (BLEU versus Crhf4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "PIs1lY2hxMsl"
   },
   "outputs": [],
   "source": [
    "# This creates the config file for our JoeyNMT system. It might seem overwhelming so we've provided a couple of useful parameters you'll need to update\n",
    "# (You can of course play with all the parameters if you'd like!)\n",
    "\n",
    "name = '%s%s' % (source_language, target_language)\n",
    "# gdrive_path = os.environ[\"gdrive_path\"]\n",
    "\n",
    "# Create the config\n",
    "config = \"\"\"\n",
    "name: \"{name}_filtered_transformer\"\n",
    "\n",
    "data:\n",
    "    src: \"{source_language}\"\n",
    "    trg: \"{target_language}\"\n",
    "    train: \"/home/ks/code/nlp/masakhane_sw2/joeynmt/data/{name}/train.bpe\"\n",
    "    dev:   \"/home/ks/code/nlp/masakhane_sw2/joeynmt/data/{name}/dev.bpe\"\n",
    "    test:  \"/home/ks/code/nlp/masakhane_sw2/joeynmt/data/{name}/test.bpe\"\n",
    "    level: \"bpe\"\n",
    "    lowercase: False\n",
    "    max_sent_length: 100\n",
    "    src_vocab: \"/home/ks/code/nlp/masakhane_sw2/joeynmt/data/{name}/vocab.txt\"\n",
    "    trg_vocab: \"/home/ks/code/nlp/masakhane_sw2/joeynmt/data/{name}/vocab.txt\"\n",
    "\n",
    "testing:\n",
    "    beam_size: 5\n",
    "    alpha: 1.0\n",
    "\n",
    "training:\n",
    "    #load_model: \"{gdrive_path}/models/{name}_transformer/1.ckpt\" # if uncommented, load a pre-trained model from this checkpoint\n",
    "    random_seed: 42\n",
    "    optimizer: \"adam\"\n",
    "    normalization: \"tokens\"\n",
    "    adam_betas: [0.9, 0.999] \n",
    "    scheduling: \"plateau\"           # TODO: try switching from plateau to Noam scheduling\n",
    "    patience: 5                     # For plateau: decrease learning rate by decrease_factor if validation score has not improved for this many validation rounds.\n",
    "    learning_rate_factor: 0.5       # factor for Noam scheduler (used with Transformer)\n",
    "    learning_rate_warmup: 1000      # warmup steps for Noam scheduler (used with Transformer)\n",
    "    decrease_factor: 0.7\n",
    "    loss: \"crossentropy\"\n",
    "    learning_rate: 0.0003\n",
    "    learning_rate_min: 0.00000001\n",
    "    weight_decay: 0.0\n",
    "    label_smoothing: 0.1\n",
    "    batch_size: 4096\n",
    "    batch_type: \"token\"\n",
    "    eval_batch_size: 3600\n",
    "    eval_batch_type: \"token\"\n",
    "    batch_multiplier: 1\n",
    "    early_stopping_metric: \"ppl\"\n",
    "    epochs: 30                     # TODO: Decrease for when playing around and checking of working. Around 30 is sufficient to check if its working at all\n",
    "    validation_freq: 1000          # TODO: Set to at least once per epoch.\n",
    "    logging_freq: 100\n",
    "    eval_metric: \"bleu\"\n",
    "    model_dir: \"/home/ks/code/tools/joeynmt/models/{name}_filtered_transformer\"\n",
    "    overwrite: False               # TODO: Set to True if you want to overwrite possibly existing models. \n",
    "    shuffle: True\n",
    "    use_cuda: True\n",
    "    max_output_length: 100\n",
    "    print_valid_sents: [0, 1, 2, 3]\n",
    "    keep_last_ckpts: 3\n",
    "\n",
    "model:\n",
    "    initializer: \"xavier\"\n",
    "    bias_initializer: \"zeros\"\n",
    "    init_gain: 1.0\n",
    "    embed_initializer: \"xavier\"\n",
    "    embed_init_gain: 1.0\n",
    "    tied_embeddings: True\n",
    "    tied_softmax: True\n",
    "    encoder:\n",
    "        type: \"transformer\"\n",
    "        num_layers: 6\n",
    "        num_heads: 4             # TODO: Increase to 8 for larger data.\n",
    "        embeddings:\n",
    "            embedding_dim: 256   # TODO: Increase to 512 for larger data.\n",
    "            scale: True\n",
    "            dropout: 0.2\n",
    "        # typically ff_size = 4 x hidden_size\n",
    "        hidden_size: 256         # TODO: Increase to 512 for larger data.\n",
    "        ff_size: 1024            # TODO: Increase to 2048 for larger data.\n",
    "        dropout: 0.3\n",
    "    decoder:\n",
    "        type: \"transformer\"\n",
    "        num_layers: 6\n",
    "        num_heads: 4              # TODO: Increase to 8 for larger data.\n",
    "        embeddings:\n",
    "            embedding_dim: 256    # TODO: Increase to 512 for larger data.\n",
    "            scale: True\n",
    "            dropout: 0.2\n",
    "        # typically ff_size = 4 x hidden_size\n",
    "        hidden_size: 256         # TODO: Increase to 512 for larger data.\n",
    "        ff_size: 1024            # TODO: Increase to 2048 for larger data.\n",
    "        dropout: 0.3\n",
    "\"\"\".format(name=name, gdrive_path=\"/home/ks/code/nlp/masakhane_sw2/joeynmt\", source_language=source_language, target_language=target_language)\n",
    "with open(\"/home/ks/code/tools/joeynmt/configs/transformer_filtered_{name}.yaml\".format(name=name),'w') as f:\n",
    "    f.write(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pIifxE3Qzuvs"
   },
   "source": [
    "# Train the Model\n",
    "\n",
    "This single line of joeynmt runs the training using the config we made above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6ZBPFwT94WpI"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "2019-10-30 11:09:55,235 Hello! This is Joey-NMT.\n",
      "2019-10-30 11:09:55,240 Total params: 12219648\n",
      "2019-10-30 11:09:55,240 Trainable parameters: ['decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.dec_layer_norm.bias', 'decoder.layers.0.dec_layer_norm.weight', 'decoder.layers.0.feed_forward.layer_norm.bias', 'decoder.layers.0.feed_forward.layer_norm.weight', 'decoder.layers.0.feed_forward.pwff_layer.0.bias', 'decoder.layers.0.feed_forward.pwff_layer.0.weight', 'decoder.layers.0.feed_forward.pwff_layer.3.bias', 'decoder.layers.0.feed_forward.pwff_layer.3.weight', 'decoder.layers.0.src_trg_att.k_layer.bias', 'decoder.layers.0.src_trg_att.k_layer.weight', 'decoder.layers.0.src_trg_att.output_layer.bias', 'decoder.layers.0.src_trg_att.output_layer.weight', 'decoder.layers.0.src_trg_att.q_layer.bias', 'decoder.layers.0.src_trg_att.q_layer.weight', 'decoder.layers.0.src_trg_att.v_layer.bias', 'decoder.layers.0.src_trg_att.v_layer.weight', 'decoder.layers.0.trg_trg_att.k_layer.bias', 'decoder.layers.0.trg_trg_att.k_layer.weight', 'decoder.layers.0.trg_trg_att.output_layer.bias', 'decoder.layers.0.trg_trg_att.output_layer.weight', 'decoder.layers.0.trg_trg_att.q_layer.bias', 'decoder.layers.0.trg_trg_att.q_layer.weight', 'decoder.layers.0.trg_trg_att.v_layer.bias', 'decoder.layers.0.trg_trg_att.v_layer.weight', 'decoder.layers.0.x_layer_norm.bias', 'decoder.layers.0.x_layer_norm.weight', 'decoder.layers.1.dec_layer_norm.bias', 'decoder.layers.1.dec_layer_norm.weight', 'decoder.layers.1.feed_forward.layer_norm.bias', 'decoder.layers.1.feed_forward.layer_norm.weight', 'decoder.layers.1.feed_forward.pwff_layer.0.bias', 'decoder.layers.1.feed_forward.pwff_layer.0.weight', 'decoder.layers.1.feed_forward.pwff_layer.3.bias', 'decoder.layers.1.feed_forward.pwff_layer.3.weight', 'decoder.layers.1.src_trg_att.k_layer.bias', 'decoder.layers.1.src_trg_att.k_layer.weight', 'decoder.layers.1.src_trg_att.output_layer.bias', 'decoder.layers.1.src_trg_att.output_layer.weight', 'decoder.layers.1.src_trg_att.q_layer.bias', 'decoder.layers.1.src_trg_att.q_layer.weight', 'decoder.layers.1.src_trg_att.v_layer.bias', 'decoder.layers.1.src_trg_att.v_layer.weight', 'decoder.layers.1.trg_trg_att.k_layer.bias', 'decoder.layers.1.trg_trg_att.k_layer.weight', 'decoder.layers.1.trg_trg_att.output_layer.bias', 'decoder.layers.1.trg_trg_att.output_layer.weight', 'decoder.layers.1.trg_trg_att.q_layer.bias', 'decoder.layers.1.trg_trg_att.q_layer.weight', 'decoder.layers.1.trg_trg_att.v_layer.bias', 'decoder.layers.1.trg_trg_att.v_layer.weight', 'decoder.layers.1.x_layer_norm.bias', 'decoder.layers.1.x_layer_norm.weight', 'decoder.layers.2.dec_layer_norm.bias', 'decoder.layers.2.dec_layer_norm.weight', 'decoder.layers.2.feed_forward.layer_norm.bias', 'decoder.layers.2.feed_forward.layer_norm.weight', 'decoder.layers.2.feed_forward.pwff_layer.0.bias', 'decoder.layers.2.feed_forward.pwff_layer.0.weight', 'decoder.layers.2.feed_forward.pwff_layer.3.bias', 'decoder.layers.2.feed_forward.pwff_layer.3.weight', 'decoder.layers.2.src_trg_att.k_layer.bias', 'decoder.layers.2.src_trg_att.k_layer.weight', 'decoder.layers.2.src_trg_att.output_layer.bias', 'decoder.layers.2.src_trg_att.output_layer.weight', 'decoder.layers.2.src_trg_att.q_layer.bias', 'decoder.layers.2.src_trg_att.q_layer.weight', 'decoder.layers.2.src_trg_att.v_layer.bias', 'decoder.layers.2.src_trg_att.v_layer.weight', 'decoder.layers.2.trg_trg_att.k_layer.bias', 'decoder.layers.2.trg_trg_att.k_layer.weight', 'decoder.layers.2.trg_trg_att.output_layer.bias', 'decoder.layers.2.trg_trg_att.output_layer.weight', 'decoder.layers.2.trg_trg_att.q_layer.bias', 'decoder.layers.2.trg_trg_att.q_layer.weight', 'decoder.layers.2.trg_trg_att.v_layer.bias', 'decoder.layers.2.trg_trg_att.v_layer.weight', 'decoder.layers.2.x_layer_norm.bias', 'decoder.layers.2.x_layer_norm.weight', 'decoder.layers.3.dec_layer_norm.bias', 'decoder.layers.3.dec_layer_norm.weight', 'decoder.layers.3.feed_forward.layer_norm.bias', 'decoder.layers.3.feed_forward.layer_norm.weight', 'decoder.layers.3.feed_forward.pwff_layer.0.bias', 'decoder.layers.3.feed_forward.pwff_layer.0.weight', 'decoder.layers.3.feed_forward.pwff_layer.3.bias', 'decoder.layers.3.feed_forward.pwff_layer.3.weight', 'decoder.layers.3.src_trg_att.k_layer.bias', 'decoder.layers.3.src_trg_att.k_layer.weight', 'decoder.layers.3.src_trg_att.output_layer.bias', 'decoder.layers.3.src_trg_att.output_layer.weight', 'decoder.layers.3.src_trg_att.q_layer.bias', 'decoder.layers.3.src_trg_att.q_layer.weight', 'decoder.layers.3.src_trg_att.v_layer.bias', 'decoder.layers.3.src_trg_att.v_layer.weight', 'decoder.layers.3.trg_trg_att.k_layer.bias', 'decoder.layers.3.trg_trg_att.k_layer.weight', 'decoder.layers.3.trg_trg_att.output_layer.bias', 'decoder.layers.3.trg_trg_att.output_layer.weight', 'decoder.layers.3.trg_trg_att.q_layer.bias', 'decoder.layers.3.trg_trg_att.q_layer.weight', 'decoder.layers.3.trg_trg_att.v_layer.bias', 'decoder.layers.3.trg_trg_att.v_layer.weight', 'decoder.layers.3.x_layer_norm.bias', 'decoder.layers.3.x_layer_norm.weight', 'decoder.layers.4.dec_layer_norm.bias', 'decoder.layers.4.dec_layer_norm.weight', 'decoder.layers.4.feed_forward.layer_norm.bias', 'decoder.layers.4.feed_forward.layer_norm.weight', 'decoder.layers.4.feed_forward.pwff_layer.0.bias', 'decoder.layers.4.feed_forward.pwff_layer.0.weight', 'decoder.layers.4.feed_forward.pwff_layer.3.bias', 'decoder.layers.4.feed_forward.pwff_layer.3.weight', 'decoder.layers.4.src_trg_att.k_layer.bias', 'decoder.layers.4.src_trg_att.k_layer.weight', 'decoder.layers.4.src_trg_att.output_layer.bias', 'decoder.layers.4.src_trg_att.output_layer.weight', 'decoder.layers.4.src_trg_att.q_layer.bias', 'decoder.layers.4.src_trg_att.q_layer.weight', 'decoder.layers.4.src_trg_att.v_layer.bias', 'decoder.layers.4.src_trg_att.v_layer.weight', 'decoder.layers.4.trg_trg_att.k_layer.bias', 'decoder.layers.4.trg_trg_att.k_layer.weight', 'decoder.layers.4.trg_trg_att.output_layer.bias', 'decoder.layers.4.trg_trg_att.output_layer.weight', 'decoder.layers.4.trg_trg_att.q_layer.bias', 'decoder.layers.4.trg_trg_att.q_layer.weight', 'decoder.layers.4.trg_trg_att.v_layer.bias', 'decoder.layers.4.trg_trg_att.v_layer.weight', 'decoder.layers.4.x_layer_norm.bias', 'decoder.layers.4.x_layer_norm.weight', 'decoder.layers.5.dec_layer_norm.bias', 'decoder.layers.5.dec_layer_norm.weight', 'decoder.layers.5.feed_forward.layer_norm.bias', 'decoder.layers.5.feed_forward.layer_norm.weight', 'decoder.layers.5.feed_forward.pwff_layer.0.bias', 'decoder.layers.5.feed_forward.pwff_layer.0.weight', 'decoder.layers.5.feed_forward.pwff_layer.3.bias', 'decoder.layers.5.feed_forward.pwff_layer.3.weight', 'decoder.layers.5.src_trg_att.k_layer.bias', 'decoder.layers.5.src_trg_att.k_layer.weight', 'decoder.layers.5.src_trg_att.output_layer.bias', 'decoder.layers.5.src_trg_att.output_layer.weight', 'decoder.layers.5.src_trg_att.q_layer.bias', 'decoder.layers.5.src_trg_att.q_layer.weight', 'decoder.layers.5.src_trg_att.v_layer.bias', 'decoder.layers.5.src_trg_att.v_layer.weight', 'decoder.layers.5.trg_trg_att.k_layer.bias', 'decoder.layers.5.trg_trg_att.k_layer.weight', 'decoder.layers.5.trg_trg_att.output_layer.bias', 'decoder.layers.5.trg_trg_att.output_layer.weight', 'decoder.layers.5.trg_trg_att.q_layer.bias', 'decoder.layers.5.trg_trg_att.q_layer.weight', 'decoder.layers.5.trg_trg_att.v_layer.bias', 'decoder.layers.5.trg_trg_att.v_layer.weight', 'decoder.layers.5.x_layer_norm.bias', 'decoder.layers.5.x_layer_norm.weight', 'encoder.layer_norm.bias', 'encoder.layer_norm.weight', 'encoder.layers.0.feed_forward.layer_norm.bias', 'encoder.layers.0.feed_forward.layer_norm.weight', 'encoder.layers.0.feed_forward.pwff_layer.0.bias', 'encoder.layers.0.feed_forward.pwff_layer.0.weight', 'encoder.layers.0.feed_forward.pwff_layer.3.bias', 'encoder.layers.0.feed_forward.pwff_layer.3.weight', 'encoder.layers.0.layer_norm.bias', 'encoder.layers.0.layer_norm.weight', 'encoder.layers.0.src_src_att.k_layer.bias', 'encoder.layers.0.src_src_att.k_layer.weight', 'encoder.layers.0.src_src_att.output_layer.bias', 'encoder.layers.0.src_src_att.output_layer.weight', 'encoder.layers.0.src_src_att.q_layer.bias', 'encoder.layers.0.src_src_att.q_layer.weight', 'encoder.layers.0.src_src_att.v_layer.bias', 'encoder.layers.0.src_src_att.v_layer.weight', 'encoder.layers.1.feed_forward.layer_norm.bias', 'encoder.layers.1.feed_forward.layer_norm.weight', 'encoder.layers.1.feed_forward.pwff_layer.0.bias', 'encoder.layers.1.feed_forward.pwff_layer.0.weight', 'encoder.layers.1.feed_forward.pwff_layer.3.bias', 'encoder.layers.1.feed_forward.pwff_layer.3.weight', 'encoder.layers.1.layer_norm.bias', 'encoder.layers.1.layer_norm.weight', 'encoder.layers.1.src_src_att.k_layer.bias', 'encoder.layers.1.src_src_att.k_layer.weight', 'encoder.layers.1.src_src_att.output_layer.bias', 'encoder.layers.1.src_src_att.output_layer.weight', 'encoder.layers.1.src_src_att.q_layer.bias', 'encoder.layers.1.src_src_att.q_layer.weight', 'encoder.layers.1.src_src_att.v_layer.bias', 'encoder.layers.1.src_src_att.v_layer.weight', 'encoder.layers.2.feed_forward.layer_norm.bias', 'encoder.layers.2.feed_forward.layer_norm.weight', 'encoder.layers.2.feed_forward.pwff_layer.0.bias', 'encoder.layers.2.feed_forward.pwff_layer.0.weight', 'encoder.layers.2.feed_forward.pwff_layer.3.bias', 'encoder.layers.2.feed_forward.pwff_layer.3.weight', 'encoder.layers.2.layer_norm.bias', 'encoder.layers.2.layer_norm.weight', 'encoder.layers.2.src_src_att.k_layer.bias', 'encoder.layers.2.src_src_att.k_layer.weight', 'encoder.layers.2.src_src_att.output_layer.bias', 'encoder.layers.2.src_src_att.output_layer.weight', 'encoder.layers.2.src_src_att.q_layer.bias', 'encoder.layers.2.src_src_att.q_layer.weight', 'encoder.layers.2.src_src_att.v_layer.bias', 'encoder.layers.2.src_src_att.v_layer.weight', 'encoder.layers.3.feed_forward.layer_norm.bias', 'encoder.layers.3.feed_forward.layer_norm.weight', 'encoder.layers.3.feed_forward.pwff_layer.0.bias', 'encoder.layers.3.feed_forward.pwff_layer.0.weight', 'encoder.layers.3.feed_forward.pwff_layer.3.bias', 'encoder.layers.3.feed_forward.pwff_layer.3.weight', 'encoder.layers.3.layer_norm.bias', 'encoder.layers.3.layer_norm.weight', 'encoder.layers.3.src_src_att.k_layer.bias', 'encoder.layers.3.src_src_att.k_layer.weight', 'encoder.layers.3.src_src_att.output_layer.bias', 'encoder.layers.3.src_src_att.output_layer.weight', 'encoder.layers.3.src_src_att.q_layer.bias', 'encoder.layers.3.src_src_att.q_layer.weight', 'encoder.layers.3.src_src_att.v_layer.bias', 'encoder.layers.3.src_src_att.v_layer.weight', 'encoder.layers.4.feed_forward.layer_norm.bias', 'encoder.layers.4.feed_forward.layer_norm.weight', 'encoder.layers.4.feed_forward.pwff_layer.0.bias', 'encoder.layers.4.feed_forward.pwff_layer.0.weight', 'encoder.layers.4.feed_forward.pwff_layer.3.bias', 'encoder.layers.4.feed_forward.pwff_layer.3.weight', 'encoder.layers.4.layer_norm.bias', 'encoder.layers.4.layer_norm.weight', 'encoder.layers.4.src_src_att.k_layer.bias', 'encoder.layers.4.src_src_att.k_layer.weight', 'encoder.layers.4.src_src_att.output_layer.bias', 'encoder.layers.4.src_src_att.output_layer.weight', 'encoder.layers.4.src_src_att.q_layer.bias', 'encoder.layers.4.src_src_att.q_layer.weight', 'encoder.layers.4.src_src_att.v_layer.bias', 'encoder.layers.4.src_src_att.v_layer.weight', 'encoder.layers.5.feed_forward.layer_norm.bias', 'encoder.layers.5.feed_forward.layer_norm.weight', 'encoder.layers.5.feed_forward.pwff_layer.0.bias', 'encoder.layers.5.feed_forward.pwff_layer.0.weight', 'encoder.layers.5.feed_forward.pwff_layer.3.bias', 'encoder.layers.5.feed_forward.pwff_layer.3.weight', 'encoder.layers.5.layer_norm.bias', 'encoder.layers.5.layer_norm.weight', 'encoder.layers.5.src_src_att.k_layer.bias', 'encoder.layers.5.src_src_att.k_layer.weight', 'encoder.layers.5.src_src_att.output_layer.bias', 'encoder.layers.5.src_src_att.output_layer.weight', 'encoder.layers.5.src_src_att.q_layer.bias', 'encoder.layers.5.src_src_att.q_layer.weight', 'encoder.layers.5.src_src_att.v_layer.bias', 'encoder.layers.5.src_src_att.v_layer.weight', 'src_embed.lut.weight']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 11:10:00,042 cfg.name                           : ensw_filtered_transformer\n",
      "2019-10-30 11:10:00,042 cfg.data.src                       : en\n",
      "2019-10-30 11:10:00,042 cfg.data.trg                       : sw\n",
      "2019-10-30 11:10:00,042 cfg.data.train                     : /home/ks/code/nlp/masakhane_sw2/joeynmt/data/ensw/train.bpe\n",
      "2019-10-30 11:10:00,042 cfg.data.dev                       : /home/ks/code/nlp/masakhane_sw2/joeynmt/data/ensw/dev.bpe\n",
      "2019-10-30 11:10:00,042 cfg.data.test                      : /home/ks/code/nlp/masakhane_sw2/joeynmt/data/ensw/test.bpe\n",
      "2019-10-30 11:10:00,042 cfg.data.level                     : bpe\n",
      "2019-10-30 11:10:00,042 cfg.data.lowercase                 : False\n",
      "2019-10-30 11:10:00,042 cfg.data.max_sent_length           : 100\n",
      "2019-10-30 11:10:00,042 cfg.data.src_vocab                 : /home/ks/code/nlp/masakhane_sw2/joeynmt/data/ensw/vocab.txt\n",
      "2019-10-30 11:10:00,042 cfg.data.trg_vocab                 : /home/ks/code/nlp/masakhane_sw2/joeynmt/data/ensw/vocab.txt\n",
      "2019-10-30 11:10:00,042 cfg.testing.beam_size              : 5\n",
      "2019-10-30 11:10:00,042 cfg.testing.alpha                  : 1.0\n",
      "2019-10-30 11:10:00,042 cfg.training.random_seed           : 42\n",
      "2019-10-30 11:10:00,042 cfg.training.optimizer             : adam\n",
      "2019-10-30 11:10:00,042 cfg.training.normalization         : tokens\n",
      "2019-10-30 11:10:00,042 cfg.training.adam_betas            : [0.9, 0.999]\n",
      "2019-10-30 11:10:00,042 cfg.training.scheduling            : plateau\n",
      "2019-10-30 11:10:00,042 cfg.training.patience              : 5\n",
      "2019-10-30 11:10:00,043 cfg.training.learning_rate_factor  : 0.5\n",
      "2019-10-30 11:10:00,043 cfg.training.learning_rate_warmup  : 1000\n",
      "2019-10-30 11:10:00,043 cfg.training.decrease_factor       : 0.7\n",
      "2019-10-30 11:10:00,043 cfg.training.loss                  : crossentropy\n",
      "2019-10-30 11:10:00,043 cfg.training.learning_rate         : 0.0003\n",
      "2019-10-30 11:10:00,043 cfg.training.learning_rate_min     : 1e-08\n",
      "2019-10-30 11:10:00,043 cfg.training.weight_decay          : 0.0\n",
      "2019-10-30 11:10:00,043 cfg.training.label_smoothing       : 0.1\n",
      "2019-10-30 11:10:00,043 cfg.training.batch_size            : 4096\n",
      "2019-10-30 11:10:00,043 cfg.training.batch_type            : token\n",
      "2019-10-30 11:10:00,043 cfg.training.eval_batch_size       : 3600\n",
      "2019-10-30 11:10:00,043 cfg.training.eval_batch_type       : token\n",
      "2019-10-30 11:10:00,043 cfg.training.batch_multiplier      : 1\n",
      "2019-10-30 11:10:00,043 cfg.training.early_stopping_metric : ppl\n",
      "2019-10-30 11:10:00,043 cfg.training.epochs                : 30\n",
      "2019-10-30 11:10:00,043 cfg.training.validation_freq       : 1000\n",
      "2019-10-30 11:10:00,043 cfg.training.logging_freq          : 100\n",
      "2019-10-30 11:10:00,043 cfg.training.eval_metric           : bleu\n",
      "2019-10-30 11:10:00,043 cfg.training.model_dir             : /home/ks/code/tools/joeynmt/models/ensw_filtered_transformer\n",
      "2019-10-30 11:10:00,043 cfg.training.overwrite             : False\n",
      "2019-10-30 11:10:00,043 cfg.training.shuffle               : True\n",
      "2019-10-30 11:10:00,043 cfg.training.use_cuda              : True\n",
      "2019-10-30 11:10:00,043 cfg.training.max_output_length     : 100\n",
      "2019-10-30 11:10:00,043 cfg.training.print_valid_sents     : [0, 1, 2, 3]\n",
      "2019-10-30 11:10:00,043 cfg.training.keep_last_ckpts       : 3\n",
      "2019-10-30 11:10:00,043 cfg.model.initializer              : xavier\n",
      "2019-10-30 11:10:00,043 cfg.model.bias_initializer         : zeros\n",
      "2019-10-30 11:10:00,043 cfg.model.init_gain                : 1.0\n",
      "2019-10-30 11:10:00,044 cfg.model.embed_initializer        : xavier\n",
      "2019-10-30 11:10:00,044 cfg.model.embed_init_gain          : 1.0\n",
      "2019-10-30 11:10:00,044 cfg.model.tied_embeddings          : True\n",
      "2019-10-30 11:10:00,044 cfg.model.tied_softmax             : True\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.type             : transformer\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.num_layers       : 6\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.num_heads        : 4\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.embeddings.embedding_dim : 256\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.embeddings.scale : True\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.embeddings.dropout : 0.2\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.hidden_size      : 256\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.ff_size          : 1024\n",
      "2019-10-30 11:10:00,044 cfg.model.encoder.dropout          : 0.3\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.type             : transformer\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.num_layers       : 6\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.num_heads        : 4\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.embeddings.embedding_dim : 256\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.embeddings.scale : True\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.embeddings.dropout : 0.2\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.hidden_size      : 256\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.ff_size          : 1024\n",
      "2019-10-30 11:10:00,044 cfg.model.decoder.dropout          : 0.3\n",
      "2019-10-30 11:10:00,044 Data set sizes: \n",
      "\ttrain 873384,\n",
      "\tvalid 1000,\n",
      "\ttest 2721\n",
      "2019-10-30 11:10:00,044 First training example:\n",
      "\t[SRC] To make matters wor@@ se , I made the bi@@ g mis@@ take of op@@ ening my m@@ outh to speak only when I was sure of my F@@ ren@@ ch .\n",
      "\t[TRG] Jambo baya hata zaidi nili@@ lo@@ fanya lilikuwa kuzungumza lugha hiyo wakati tu nili@@ pokuwa na uhakika kwamba K@@ if@@ ar@@ ansa ch@@ angu ni sahihi .\n",
      "2019-10-30 11:10:00,044 First 10 words (src): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) , (5) . (6) the (7) na (8) ya (9) to\n",
      "2019-10-30 11:10:00,045 First 10 words (trg): (0) <unk> (1) <pad> (2) <s> (3) </s> (4) , (5) . (6) the (7) na (8) ya (9) to\n",
      "2019-10-30 11:10:00,045 Number of Src words (types): 4529\n",
      "2019-10-30 11:10:00,045 Number of Trg words (types): 4529\n",
      "2019-10-30 11:10:00,045 Model(\n",
      "\tencoder=TransformerEncoder(num_layers=6, num_heads=4),\n",
      "\tdecoder=TransformerDecoder(num_layers=6, num_heads=4),\n",
      "\tsrc_embed=Embeddings(embedding_dim=256, vocab_size=4529),\n",
      "\ttrg_embed=Embeddings(embedding_dim=256, vocab_size=4529))\n",
      "2019-10-30 11:10:00,047 EPOCH 1\n",
      "2019-10-30 11:10:14,851 Epoch   1 Step:      100 Batch Loss:     5.683980 Tokens per Sec:    17581, Lr: 0.000300\n",
      "2019-10-30 11:10:28,500 Epoch   1 Step:      200 Batch Loss:     5.577012 Tokens per Sec:    18765, Lr: 0.000300\n",
      "2019-10-30 11:10:42,162 Epoch   1 Step:      300 Batch Loss:     5.317440 Tokens per Sec:    18459, Lr: 0.000300\n",
      "2019-10-30 11:10:55,987 Epoch   1 Step:      400 Batch Loss:     5.445060 Tokens per Sec:    18561, Lr: 0.000300\n",
      "2019-10-30 11:11:10,155 Epoch   1 Step:      500 Batch Loss:     4.950495 Tokens per Sec:    18615, Lr: 0.000300\n",
      "2019-10-30 11:11:24,064 Epoch   1 Step:      600 Batch Loss:     5.172194 Tokens per Sec:    18323, Lr: 0.000300\n",
      "2019-10-30 11:11:43,442 Epoch   1 Step:      700 Batch Loss:     4.581911 Tokens per Sec:    12830, Lr: 0.000300\n",
      "2019-10-30 11:12:14,716 Epoch   1 Step:      800 Batch Loss:     4.733677 Tokens per Sec:     8230, Lr: 0.000300\n",
      "2019-10-30 11:12:37,255 Epoch   1 Step:      900 Batch Loss:     4.646959 Tokens per Sec:    11359, Lr: 0.000300\n",
      "2019-10-30 11:12:52,705 Epoch   1 Step:     1000 Batch Loss:     4.547887 Tokens per Sec:    16294, Lr: 0.000300\n",
      "2019-10-30 11:13:34,622 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:13:34,622 Saving new checkpoint.\n",
      "2019-10-30 11:13:34,747 Example #0\n",
      "2019-10-30 11:13:34,747 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:13:34,748 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:13:34,748 \tHypothesis: Meno ya miaka ya miaka W.K .\n",
      "2019-10-30 11:13:34,748 Example #1\n",
      "2019-10-30 11:13:34,748 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:13:34,748 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:13:34,748 \tHypothesis: Kwa nini ni jambo wa Yehova .\n",
      "2019-10-30 11:13:34,748 Example #2\n",
      "2019-10-30 11:13:34,748 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:13:34,748 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:13:34,748 \tHypothesis: Kwa nini , na nini , na kuhitaji kufanya nini ?\n",
      "2019-10-30 11:13:34,748 Example #3\n",
      "2019-10-30 11:13:34,748 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:13:34,748 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:13:34,748 \tHypothesis: Mabi wa kujia .\n",
      "2019-10-30 11:13:34,748 Validation result at epoch   1, step     1000: bleu:   0.87, loss: 120892.7109, ppl:  81.3416, duration: 42.0430s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 11:13:49,033 Epoch   1 Step:     1100 Batch Loss:     4.341217 Tokens per Sec:    17921, Lr: 0.000300\n",
      "2019-10-30 11:14:03,124 Epoch   1 Step:     1200 Batch Loss:     4.113015 Tokens per Sec:    17886, Lr: 0.000300\n",
      "2019-10-30 11:14:17,278 Epoch   1 Step:     1300 Batch Loss:     4.046868 Tokens per Sec:    18521, Lr: 0.000300\n",
      "2019-10-30 11:14:31,368 Epoch   1 Step:     1400 Batch Loss:     4.203510 Tokens per Sec:    18247, Lr: 0.000300\n",
      "2019-10-30 11:14:45,529 Epoch   1 Step:     1500 Batch Loss:     4.009446 Tokens per Sec:    18183, Lr: 0.000300\n",
      "2019-10-30 11:14:59,485 Epoch   1 Step:     1600 Batch Loss:     4.068118 Tokens per Sec:    18034, Lr: 0.000300\n",
      "2019-10-30 11:15:13,727 Epoch   1 Step:     1700 Batch Loss:     3.987222 Tokens per Sec:    18242, Lr: 0.000300\n",
      "2019-10-30 11:15:27,974 Epoch   1 Step:     1800 Batch Loss:     3.617097 Tokens per Sec:    18360, Lr: 0.000300\n",
      "2019-10-30 11:15:42,104 Epoch   1 Step:     1900 Batch Loss:     3.629297 Tokens per Sec:    17854, Lr: 0.000300\n",
      "2019-10-30 11:15:56,274 Epoch   1 Step:     2000 Batch Loss:     3.862552 Tokens per Sec:    17900, Lr: 0.000300\n",
      "2019-10-30 11:16:38,575 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:16:38,575 Saving new checkpoint.\n",
      "2019-10-30 11:16:38,693 Example #0\n",
      "2019-10-30 11:16:38,693 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:16:38,693 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:16:38,693 \tHypothesis: Alikuwa na umri wa miaka mingi .\n",
      "2019-10-30 11:16:38,693 Example #1\n",
      "2019-10-30 11:16:38,693 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:16:38,693 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:16:38,693 \tHypothesis: Biblia hutenda kwa njia ya kundu .\n",
      "2019-10-30 11:16:38,694 Example #2\n",
      "2019-10-30 11:16:38,694 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:16:38,694 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:16:38,694 \tHypothesis: Ni nini kwamba kuna jambo la kunaweza kuchunguza jinsi gani ?\n",
      "2019-10-30 11:16:38,694 Example #3\n",
      "2019-10-30 11:16:38,694 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:16:38,694 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:16:38,694 \tHypothesis: Ujapokuwa na mamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamamaa .\n",
      "2019-10-30 11:16:38,694 Validation result at epoch   1, step     2000: bleu:   2.81, loss: 99126.6953, ppl:  36.8445, duration: 42.4199s\n",
      "2019-10-30 11:16:53,111 Epoch   1 Step:     2100 Batch Loss:     3.676395 Tokens per Sec:    18021, Lr: 0.000300\n",
      "2019-10-30 11:17:07,541 Epoch   1 Step:     2200 Batch Loss:     3.503572 Tokens per Sec:    17732, Lr: 0.000300\n",
      "2019-10-30 11:17:22,144 Epoch   1 Step:     2300 Batch Loss:     3.581944 Tokens per Sec:    17435, Lr: 0.000300\n",
      "2019-10-30 11:17:36,548 Epoch   1 Step:     2400 Batch Loss:     3.568376 Tokens per Sec:    17822, Lr: 0.000300\n",
      "2019-10-30 11:17:50,792 Epoch   1 Step:     2500 Batch Loss:     3.716051 Tokens per Sec:    17766, Lr: 0.000300\n",
      "2019-10-30 11:18:05,508 Epoch   1 Step:     2600 Batch Loss:     3.060312 Tokens per Sec:    17610, Lr: 0.000300\n",
      "2019-10-30 11:18:20,116 Epoch   1 Step:     2700 Batch Loss:     3.083419 Tokens per Sec:    17374, Lr: 0.000300\n",
      "2019-10-30 11:18:34,413 Epoch   1 Step:     2800 Batch Loss:     3.603699 Tokens per Sec:    17861, Lr: 0.000300\n",
      "2019-10-30 11:18:48,665 Epoch   1 Step:     2900 Batch Loss:     3.753938 Tokens per Sec:    18048, Lr: 0.000300\n",
      "2019-10-30 11:19:03,149 Epoch   1 Step:     3000 Batch Loss:     3.385772 Tokens per Sec:    18115, Lr: 0.000300\n",
      "2019-10-30 11:19:45,623 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:19:45,623 Saving new checkpoint.\n",
      "2019-10-30 11:19:45,747 Example #0\n",
      "2019-10-30 11:19:45,747 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:19:45,747 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:19:45,747 \tHypothesis: Alikuwa na umri wa miaka 16 .\n",
      "2019-10-30 11:19:45,747 Example #1\n",
      "2019-10-30 11:19:45,747 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:19:45,747 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:19:45,747 \tHypothesis: Biblia hawatazamia uovu wa kiadili .\n",
      "2019-10-30 11:19:45,747 Example #2\n",
      "2019-10-30 11:19:45,747 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:19:45,747 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:19:45,747 \tHypothesis: Ni jambo gani kwamba kuna hatari gani , na kunaweza kuchunguza mambo yanayoweza kufautiana ?\n",
      "2019-10-30 11:19:45,747 Example #3\n",
      "2019-10-30 11:19:45,747 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:19:45,747 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:19:45,747 \tHypothesis: Maduni ya mawingizi na madogo .\n",
      "2019-10-30 11:19:45,748 Validation result at epoch   1, step     3000: bleu:   5.12, loss: 85728.5703, ppl:  22.6287, duration: 42.5976s\n",
      "2019-10-30 11:20:00,184 Epoch   1 Step:     3100 Batch Loss:     2.763184 Tokens per Sec:    17545, Lr: 0.000300\n",
      "2019-10-30 11:20:15,804 Epoch   1 Step:     3200 Batch Loss:     3.362744 Tokens per Sec:    17065, Lr: 0.000300\n",
      "2019-10-30 11:20:31,434 Epoch   1 Step:     3300 Batch Loss:     3.099243 Tokens per Sec:    16633, Lr: 0.000300\n",
      "2019-10-30 11:20:46,688 Epoch   1 Step:     3400 Batch Loss:     3.209504 Tokens per Sec:    16760, Lr: 0.000300\n",
      "2019-10-30 11:21:00,918 Epoch   1 Step:     3500 Batch Loss:     2.925356 Tokens per Sec:    17941, Lr: 0.000300\n",
      "2019-10-30 11:21:15,760 Epoch   1 Step:     3600 Batch Loss:     3.599034 Tokens per Sec:    17020, Lr: 0.000300\n",
      "2019-10-30 11:21:30,017 Epoch   1 Step:     3700 Batch Loss:     3.019784 Tokens per Sec:    17808, Lr: 0.000300\n",
      "2019-10-30 11:21:45,475 Epoch   1 Step:     3800 Batch Loss:     3.354308 Tokens per Sec:    17267, Lr: 0.000300\n",
      "2019-10-30 11:21:59,786 Epoch   1 Step:     3900 Batch Loss:     2.837762 Tokens per Sec:    17534, Lr: 0.000300\n",
      "2019-10-30 11:22:14,368 Epoch   1 Step:     4000 Batch Loss:     3.020594 Tokens per Sec:    18252, Lr: 0.000300\n",
      "2019-10-30 11:22:58,303 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:22:58,304 Saving new checkpoint.\n",
      "2019-10-30 11:22:58,434 Example #0\n",
      "2019-10-30 11:22:58,435 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:22:58,435 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:22:58,435 \tHypothesis: Alikuwa amekuwa na umri wa miaka 17 .\n",
      "2019-10-30 11:22:58,435 Example #1\n",
      "2019-10-30 11:22:58,435 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:22:58,435 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:22:58,435 \tHypothesis: Biblia hayaswi kuharibiwa .\n",
      "2019-10-30 11:22:58,435 Example #2\n",
      "2019-10-30 11:22:58,435 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:22:58,435 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:22:58,435 \tHypothesis: Ni mambo gani ambayo yanaweza kufautiana , na jinsi gani kunaweza kufaulana ?\n",
      "2019-10-30 11:22:58,435 Example #3\n",
      "2019-10-30 11:22:58,435 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:22:58,435 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:22:58,435 \tHypothesis: Maduni na mazoezi .\n",
      "2019-10-30 11:22:58,435 Validation result at epoch   1, step     4000: bleu:   8.43, loss: 77592.1094, ppl:  16.8302, duration: 44.0672s\n",
      "2019-10-30 11:23:13,051 Epoch   1 Step:     4100 Batch Loss:     3.210609 Tokens per Sec:    17609, Lr: 0.000300\n",
      "2019-10-30 11:23:27,822 Epoch   1 Step:     4200 Batch Loss:     2.986566 Tokens per Sec:    17149, Lr: 0.000300\n",
      "2019-10-30 11:23:42,336 Epoch   1 Step:     4300 Batch Loss:     3.055120 Tokens per Sec:    17950, Lr: 0.000300\n",
      "2019-10-30 11:23:56,875 Epoch   1 Step:     4400 Batch Loss:     3.073128 Tokens per Sec:    17133, Lr: 0.000300\n",
      "2019-10-30 11:24:11,539 Epoch   1 Step:     4500 Batch Loss:     2.822597 Tokens per Sec:    17585, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 11:24:26,137 Epoch   1 Step:     4600 Batch Loss:     2.670850 Tokens per Sec:    17279, Lr: 0.000300\n",
      "2019-10-30 11:24:40,383 Epoch   1 Step:     4700 Batch Loss:     2.753960 Tokens per Sec:    17992, Lr: 0.000300\n",
      "2019-10-30 11:24:54,637 Epoch   1 Step:     4800 Batch Loss:     3.159465 Tokens per Sec:    18338, Lr: 0.000300\n",
      "2019-10-30 11:25:09,078 Epoch   1 Step:     4900 Batch Loss:     2.992566 Tokens per Sec:    17680, Lr: 0.000300\n",
      "2019-10-30 11:25:23,598 Epoch   1 Step:     5000 Batch Loss:     3.456820 Tokens per Sec:    17614, Lr: 0.000300\n",
      "2019-10-30 11:26:06,438 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:26:06,438 Saving new checkpoint.\n",
      "2019-10-30 11:26:06,567 Example #0\n",
      "2019-10-30 11:26:06,567 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:26:06,567 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:26:06,567 \tHypothesis: Alikuwa amekuwa na umri wa miaka 17 .\n",
      "2019-10-30 11:26:06,567 Example #1\n",
      "2019-10-30 11:26:06,567 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:26:06,567 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:26:06,567 \tHypothesis: Biblia haitastahili kuhusiana na uumbaji .\n",
      "2019-10-30 11:26:06,567 Example #2\n",
      "2019-10-30 11:26:06,568 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:26:06,568 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:26:06,568 \tHypothesis: Ni mambo gani ambayo yanaweza kuathiri , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:26:06,568 Example #3\n",
      "2019-10-30 11:26:06,568 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:26:06,568 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:26:06,568 \tHypothesis: Madaktari na mazoezi .\n",
      "2019-10-30 11:26:06,568 Validation result at epoch   1, step     5000: bleu:  10.88, loss: 71383.7031, ppl:  13.4272, duration: 42.9690s\n",
      "2019-10-30 11:26:21,999 Epoch   1 Step:     5100 Batch Loss:     3.004599 Tokens per Sec:    17063, Lr: 0.000300\n",
      "2019-10-30 11:26:37,150 Epoch   1 Step:     5200 Batch Loss:     2.950419 Tokens per Sec:    16948, Lr: 0.000300\n",
      "2019-10-30 11:26:51,660 Epoch   1 Step:     5300 Batch Loss:     2.783336 Tokens per Sec:    18053, Lr: 0.000300\n",
      "2019-10-30 11:27:06,796 Epoch   1 Step:     5400 Batch Loss:     2.948578 Tokens per Sec:    17454, Lr: 0.000300\n",
      "2019-10-30 11:27:22,374 Epoch   1 Step:     5500 Batch Loss:     2.928643 Tokens per Sec:    16564, Lr: 0.000300\n",
      "2019-10-30 11:27:37,808 Epoch   1 Step:     5600 Batch Loss:     2.782459 Tokens per Sec:    16590, Lr: 0.000300\n",
      "2019-10-30 11:27:53,558 Epoch   1 Step:     5700 Batch Loss:     2.815937 Tokens per Sec:    16337, Lr: 0.000300\n",
      "2019-10-30 11:28:09,068 Epoch   1 Step:     5800 Batch Loss:     2.516478 Tokens per Sec:    16286, Lr: 0.000300\n",
      "2019-10-30 11:28:24,236 Epoch   1 Step:     5900 Batch Loss:     3.048704 Tokens per Sec:    17009, Lr: 0.000300\n",
      "2019-10-30 11:28:38,776 Epoch   1 Step:     6000 Batch Loss:     2.637599 Tokens per Sec:    17991, Lr: 0.000300\n",
      "2019-10-30 11:29:23,661 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:29:23,661 Saving new checkpoint.\n",
      "2019-10-30 11:29:23,828 Example #0\n",
      "2019-10-30 11:29:23,828 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:29:23,828 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:29:23,828 \tHypothesis: Alikuwa amehubiri katika umri wa miaka 17 .\n",
      "2019-10-30 11:29:23,828 Example #1\n",
      "2019-10-30 11:29:23,828 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:29:23,828 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:29:23,829 \tHypothesis: Biblia haitegemei uumbaji wa kimbingu .\n",
      "2019-10-30 11:29:23,829 Example #2\n",
      "2019-10-30 11:29:23,829 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:29:23,829 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:29:23,829 \tHypothesis: Ni mambo gani ambayo yanaweza kufahamu , na ni nini kinachoweza kuathiri hali ya kulevya ?\n",
      "2019-10-30 11:29:23,829 Example #3\n",
      "2019-10-30 11:29:23,829 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:29:23,829 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:29:23,829 \tHypothesis: Madhara na mashine .\n",
      "2019-10-30 11:29:23,829 Validation result at epoch   1, step     6000: bleu:  12.84, loss: 67318.6406, ppl:  11.5811, duration: 45.0529s\n",
      "2019-10-30 11:29:39,557 Epoch   1 Step:     6100 Batch Loss:     2.886109 Tokens per Sec:    16243, Lr: 0.000300\n",
      "2019-10-30 11:29:55,263 Epoch   1 Step:     6200 Batch Loss:     2.918218 Tokens per Sec:    16559, Lr: 0.000300\n",
      "2019-10-30 11:30:09,596 Epoch   1 Step:     6300 Batch Loss:     2.349925 Tokens per Sec:    17635, Lr: 0.000300\n",
      "2019-10-30 11:30:24,203 Epoch   1 Step:     6400 Batch Loss:     3.122527 Tokens per Sec:    17837, Lr: 0.000300\n",
      "2019-10-30 11:30:39,334 Epoch   1 Step:     6500 Batch Loss:     2.539003 Tokens per Sec:    16857, Lr: 0.000300\n",
      "2019-10-30 11:30:53,936 Epoch   1 Step:     6600 Batch Loss:     2.603079 Tokens per Sec:    17497, Lr: 0.000300\n",
      "2019-10-30 11:31:08,311 Epoch   1 Step:     6700 Batch Loss:     2.571902 Tokens per Sec:    17805, Lr: 0.000300\n",
      "2019-10-30 11:31:22,683 Epoch   1 Step:     6800 Batch Loss:     2.775760 Tokens per Sec:    17802, Lr: 0.000300\n",
      "2019-10-30 11:31:37,034 Epoch   1 Step:     6900 Batch Loss:     2.346484 Tokens per Sec:    17715, Lr: 0.000300\n",
      "2019-10-30 11:31:51,638 Epoch   1 Step:     7000 Batch Loss:     2.613517 Tokens per Sec:    17709, Lr: 0.000300\n",
      "2019-10-30 11:32:33,364 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:32:33,364 Saving new checkpoint.\n",
      "2019-10-30 11:32:33,491 Example #0\n",
      "2019-10-30 11:32:33,491 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:32:33,491 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:32:33,492 \tHypothesis: Alikuwa akifanya kazi ya kukaa katika umri wa miaka 17 .\n",
      "2019-10-30 11:32:33,492 Example #1\n",
      "2019-10-30 11:32:33,492 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:32:33,492 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:32:33,492 \tHypothesis: Biblia haitaswi kunawakilisha uumbaji .\n",
      "2019-10-30 11:32:33,492 Example #2\n",
      "2019-10-30 11:32:33,492 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:32:33,492 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:32:33,492 \tHypothesis: Ni mambo gani ambayo yanaweza kuathiri sana , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:32:33,492 Example #3\n",
      "2019-10-30 11:32:33,492 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:32:33,492 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:32:33,492 \tHypothesis: Madaktari na mashine .\n",
      "2019-10-30 11:32:33,492 Validation result at epoch   1, step     7000: bleu:  15.15, loss: 63303.4883, ppl:  10.0070, duration: 41.8534s\n",
      "2019-10-30 11:32:47,502 Epoch   1 Step:     7100 Batch Loss:     2.586793 Tokens per Sec:    17922, Lr: 0.000300\n",
      "2019-10-30 11:33:01,895 Epoch   1 Step:     7200 Batch Loss:     2.508797 Tokens per Sec:    17934, Lr: 0.000300\n",
      "2019-10-30 11:33:16,202 Epoch   1 Step:     7300 Batch Loss:     2.715571 Tokens per Sec:    17747, Lr: 0.000300\n",
      "2019-10-30 11:33:30,440 Epoch   1 Step:     7400 Batch Loss:     2.072850 Tokens per Sec:    18247, Lr: 0.000300\n",
      "2019-10-30 11:33:44,657 Epoch   1 Step:     7500 Batch Loss:     2.615349 Tokens per Sec:    18104, Lr: 0.000300\n",
      "2019-10-30 11:33:58,909 Epoch   1 Step:     7600 Batch Loss:     2.261326 Tokens per Sec:    18142, Lr: 0.000300\n",
      "2019-10-30 11:34:13,138 Epoch   1 Step:     7700 Batch Loss:     2.251446 Tokens per Sec:    18048, Lr: 0.000300\n",
      "2019-10-30 11:34:27,097 Epoch   1 Step:     7800 Batch Loss:     2.752595 Tokens per Sec:    18105, Lr: 0.000300\n",
      "2019-10-30 11:34:41,128 Epoch   1 Step:     7900 Batch Loss:     2.916224 Tokens per Sec:    18170, Lr: 0.000300\n",
      "2019-10-30 11:34:55,410 Epoch   1 Step:     8000 Batch Loss:     2.358280 Tokens per Sec:    17824, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 11:35:51,320 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:35:51,320 Saving new checkpoint.\n",
      "2019-10-30 11:35:51,861 Example #0\n",
      "2019-10-30 11:35:51,862 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:35:51,862 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:35:51,862 \tHypothesis: Alikuwa akitafuta umri wa miaka 17 .\n",
      "2019-10-30 11:35:51,862 Example #1\n",
      "2019-10-30 11:35:51,863 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:35:51,863 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:35:51,864 \tHypothesis: Biblia haitegemei uumbaji wa uumbaji .\n",
      "2019-10-30 11:35:51,864 Example #2\n",
      "2019-10-30 11:35:51,864 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:35:51,865 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:35:51,865 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na yanaweza kuathiri hali gani ?\n",
      "2019-10-30 11:35:51,865 Example #3\n",
      "2019-10-30 11:35:51,865 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:35:51,866 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:35:51,866 \tHypothesis: Sifa za kuleza na mashine .\n",
      "2019-10-30 11:35:51,866 Validation result at epoch   1, step     8000: bleu:  16.16, loss: 60214.6562, ppl:   8.9432, duration: 56.4559s\n",
      "2019-10-30 11:36:24,372 Epoch   1 Step:     8100 Batch Loss:     2.418090 Tokens per Sec:     7713, Lr: 0.000300\n",
      "2019-10-30 11:36:52,480 Epoch   1 Step:     8200 Batch Loss:     2.295199 Tokens per Sec:     9063, Lr: 0.000300\n",
      "2019-10-30 11:37:08,792 Epoch   1 Step:     8300 Batch Loss:     2.395776 Tokens per Sec:    15684, Lr: 0.000300\n",
      "2019-10-30 11:37:25,319 Epoch   1 Step:     8400 Batch Loss:     2.841190 Tokens per Sec:    15643, Lr: 0.000300\n",
      "2019-10-30 11:37:40,287 Epoch   1 Step:     8500 Batch Loss:     2.333640 Tokens per Sec:    16772, Lr: 0.000300\n",
      "2019-10-30 11:37:55,095 Epoch   1 Step:     8600 Batch Loss:     2.224446 Tokens per Sec:    17451, Lr: 0.000300\n",
      "2019-10-30 11:38:10,394 Epoch   1 Step:     8700 Batch Loss:     2.670107 Tokens per Sec:    16921, Lr: 0.000300\n",
      "2019-10-30 11:38:25,360 Epoch   1 Step:     8800 Batch Loss:     2.796523 Tokens per Sec:    17543, Lr: 0.000300\n",
      "2019-10-30 11:38:39,474 Epoch   1 Step:     8900 Batch Loss:     2.096144 Tokens per Sec:    17841, Lr: 0.000300\n",
      "2019-10-30 11:38:53,803 Epoch   1 Step:     9000 Batch Loss:     2.155795 Tokens per Sec:    18002, Lr: 0.000300\n",
      "2019-10-30 11:39:35,708 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:39:35,708 Saving new checkpoint.\n",
      "2019-10-30 11:39:35,839 Example #0\n",
      "2019-10-30 11:39:35,839 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:39:35,839 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:39:35,839 \tHypothesis: Alikuwa mwangalifu wa umri wa miaka 17 .\n",
      "2019-10-30 11:39:35,839 Example #1\n",
      "2019-10-30 11:39:35,839 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:39:35,839 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:39:35,839 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 11:39:35,839 Example #2\n",
      "2019-10-30 11:39:35,839 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:39:35,839 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:39:35,839 \tHypothesis: Ni mambo gani yanayosababisha matatizo , na ni nini kinachoweza kuhangaika ?\n",
      "2019-10-30 11:39:35,839 Example #3\n",
      "2019-10-30 11:39:35,839 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:39:35,839 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:39:35,840 \tHypothesis: Mtazamo wa mashine na maua .\n",
      "2019-10-30 11:39:35,840 Validation result at epoch   1, step     9000: bleu:  17.94, loss: 57998.0156, ppl:   8.2503, duration: 42.0362s\n",
      "2019-10-30 11:39:49,986 Epoch   1 Step:     9100 Batch Loss:     2.555076 Tokens per Sec:    18306, Lr: 0.000300\n",
      "2019-10-30 11:40:04,272 Epoch   1 Step:     9200 Batch Loss:     2.133990 Tokens per Sec:    18028, Lr: 0.000300\n",
      "2019-10-30 11:40:18,793 Epoch   1 Step:     9300 Batch Loss:     2.246813 Tokens per Sec:    18134, Lr: 0.000300\n",
      "2019-10-30 11:40:31,771 Epoch   1: total training loss 29908.74\n",
      "2019-10-30 11:40:31,771 EPOCH 2\n",
      "2019-10-30 11:40:34,304 Epoch   2 Step:     9400 Batch Loss:     2.160109 Tokens per Sec:     9841, Lr: 0.000300\n",
      "2019-10-30 11:40:48,521 Epoch   2 Step:     9500 Batch Loss:     2.184427 Tokens per Sec:    17795, Lr: 0.000300\n",
      "2019-10-30 11:41:03,007 Epoch   2 Step:     9600 Batch Loss:     2.117386 Tokens per Sec:    17530, Lr: 0.000300\n",
      "2019-10-30 11:41:17,904 Epoch   2 Step:     9700 Batch Loss:     2.435861 Tokens per Sec:    17270, Lr: 0.000300\n",
      "2019-10-30 11:41:32,287 Epoch   2 Step:     9800 Batch Loss:     2.261851 Tokens per Sec:    17737, Lr: 0.000300\n",
      "2019-10-30 11:41:46,731 Epoch   2 Step:     9900 Batch Loss:     2.233311 Tokens per Sec:    17803, Lr: 0.000300\n",
      "2019-10-30 11:42:01,068 Epoch   2 Step:    10000 Batch Loss:     2.326972 Tokens per Sec:    17722, Lr: 0.000300\n",
      "2019-10-30 11:42:44,365 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:42:44,365 Saving new checkpoint.\n",
      "2019-10-30 11:42:44,500 Example #0\n",
      "2019-10-30 11:42:44,500 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:42:44,500 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:42:44,500 \tHypothesis: Alikuwa akifanya kazi katika umri wa miaka 17 .\n",
      "2019-10-30 11:42:44,500 Example #1\n",
      "2019-10-30 11:42:44,500 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:42:44,500 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:42:44,501 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 11:42:44,501 Example #2\n",
      "2019-10-30 11:42:44,501 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:42:44,501 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:42:44,501 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:42:44,501 Example #3\n",
      "2019-10-30 11:42:44,501 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:42:44,501 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:42:44,501 \tHypothesis: Mtazamo wa maua na maua .\n",
      "2019-10-30 11:42:44,501 Validation result at epoch   2, step    10000: bleu:  18.77, loss: 55727.7344, ppl:   7.5962, duration: 43.4323s\n",
      "2019-10-30 11:42:59,124 Epoch   2 Step:    10100 Batch Loss:     2.414272 Tokens per Sec:    17524, Lr: 0.000300\n",
      "2019-10-30 11:43:13,746 Epoch   2 Step:    10200 Batch Loss:     2.342788 Tokens per Sec:    17875, Lr: 0.000300\n",
      "2019-10-30 11:43:28,568 Epoch   2 Step:    10300 Batch Loss:     1.998764 Tokens per Sec:    17174, Lr: 0.000300\n",
      "2019-10-30 11:43:43,495 Epoch   2 Step:    10400 Batch Loss:     1.973090 Tokens per Sec:    17363, Lr: 0.000300\n",
      "2019-10-30 11:43:58,797 Epoch   2 Step:    10500 Batch Loss:     2.043303 Tokens per Sec:    16694, Lr: 0.000300\n",
      "2019-10-30 11:44:13,845 Epoch   2 Step:    10600 Batch Loss:     2.039910 Tokens per Sec:    17028, Lr: 0.000300\n",
      "2019-10-30 11:44:29,072 Epoch   2 Step:    10700 Batch Loss:     2.323383 Tokens per Sec:    17199, Lr: 0.000300\n",
      "2019-10-30 11:44:43,962 Epoch   2 Step:    10800 Batch Loss:     2.263069 Tokens per Sec:    17090, Lr: 0.000300\n",
      "2019-10-30 11:44:58,349 Epoch   2 Step:    10900 Batch Loss:     2.096847 Tokens per Sec:    17497, Lr: 0.000300\n",
      "2019-10-30 11:45:13,156 Epoch   2 Step:    11000 Batch Loss:     2.465977 Tokens per Sec:    17682, Lr: 0.000300\n",
      "2019-10-30 11:45:55,463 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:45:55,463 Saving new checkpoint.\n",
      "2019-10-30 11:45:55,636 Example #0\n",
      "2019-10-30 11:45:55,637 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:45:55,637 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:45:55,637 \tHypothesis: Alikuwa akifanya kazi katika umri wa miaka 17 .\n",
      "2019-10-30 11:45:55,637 Example #1\n",
      "2019-10-30 11:45:55,637 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:45:55,637 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:45:55,637 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 11:45:55,637 Example #2\n",
      "2019-10-30 11:45:55,637 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:45:55,637 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:45:55,637 \tHypothesis: Ni mambo gani ambayo husababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:45:55,637 Example #3\n",
      "2019-10-30 11:45:55,637 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:45:55,637 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:45:55,637 \tHypothesis: Migino ya maua na matumizi .\n",
      "2019-10-30 11:45:55,637 Validation result at epoch   2, step    11000: bleu:  20.55, loss: 54107.0312, ppl:   7.1612, duration: 42.4805s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 11:46:09,762 Epoch   2 Step:    11100 Batch Loss:     2.431263 Tokens per Sec:    18562, Lr: 0.000300\n",
      "2019-10-30 11:46:24,715 Epoch   2 Step:    11200 Batch Loss:     2.085107 Tokens per Sec:    16813, Lr: 0.000300\n",
      "2019-10-30 11:46:38,877 Epoch   2 Step:    11300 Batch Loss:     2.336582 Tokens per Sec:    18341, Lr: 0.000300\n",
      "2019-10-30 11:46:53,064 Epoch   2 Step:    11400 Batch Loss:     2.406187 Tokens per Sec:    18317, Lr: 0.000300\n",
      "2019-10-30 11:47:06,996 Epoch   2 Step:    11500 Batch Loss:     2.149143 Tokens per Sec:    18293, Lr: 0.000300\n",
      "2019-10-30 11:47:20,849 Epoch   2 Step:    11600 Batch Loss:     2.235575 Tokens per Sec:    18151, Lr: 0.000300\n",
      "2019-10-30 11:47:34,730 Epoch   2 Step:    11700 Batch Loss:     2.224729 Tokens per Sec:    18275, Lr: 0.000300\n",
      "2019-10-30 11:47:48,883 Epoch   2 Step:    11800 Batch Loss:     2.059717 Tokens per Sec:    18490, Lr: 0.000300\n",
      "2019-10-30 11:48:02,826 Epoch   2 Step:    11900 Batch Loss:     2.162276 Tokens per Sec:    18164, Lr: 0.000300\n",
      "2019-10-30 11:48:16,964 Epoch   2 Step:    12000 Batch Loss:     2.212478 Tokens per Sec:    18171, Lr: 0.000300\n",
      "2019-10-30 11:49:15,156 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:49:15,157 Saving new checkpoint.\n",
      "2019-10-30 11:49:15,651 Example #0\n",
      "2019-10-30 11:49:15,652 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:49:15,652 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:49:15,653 \tHypothesis: Alikuwa mzee wa umri wa miaka 17 .\n",
      "2019-10-30 11:49:15,653 Example #1\n",
      "2019-10-30 11:49:15,653 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:49:15,653 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:49:15,654 \tHypothesis: Biblia haitegemei uumbaji wa uumbaji .\n",
      "2019-10-30 11:49:15,654 Example #2\n",
      "2019-10-30 11:49:15,654 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:49:15,655 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:49:15,655 \tHypothesis: Ni mambo gani ambayo yanaweza kuhangaika , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:49:15,655 Example #3\n",
      "2019-10-30 11:49:15,655 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:49:15,656 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:49:15,656 \tHypothesis: Mapema ya maua na misiba .\n",
      "2019-10-30 11:49:15,656 Validation result at epoch   2, step    12000: bleu:  21.35, loss: 52610.3281, ppl:   6.7816, duration: 58.6916s\n",
      "2019-10-30 11:49:46,029 Epoch   2 Step:    12100 Batch Loss:     1.966652 Tokens per Sec:     8374, Lr: 0.000300\n",
      "2019-10-30 11:50:06,629 Epoch   2 Step:    12200 Batch Loss:     2.225543 Tokens per Sec:    12517, Lr: 0.000300\n",
      "2019-10-30 11:50:21,838 Epoch   2 Step:    12300 Batch Loss:     1.989405 Tokens per Sec:    16904, Lr: 0.000300\n",
      "2019-10-30 11:50:36,597 Epoch   2 Step:    12400 Batch Loss:     1.843273 Tokens per Sec:    17479, Lr: 0.000300\n",
      "2019-10-30 11:50:50,925 Epoch   2 Step:    12500 Batch Loss:     2.165266 Tokens per Sec:    17830, Lr: 0.000300\n",
      "2019-10-30 11:51:05,310 Epoch   2 Step:    12600 Batch Loss:     2.250498 Tokens per Sec:    18114, Lr: 0.000300\n",
      "2019-10-30 11:51:19,375 Epoch   2 Step:    12700 Batch Loss:     2.158380 Tokens per Sec:    18371, Lr: 0.000300\n",
      "2019-10-30 11:51:33,267 Epoch   2 Step:    12800 Batch Loss:     2.376029 Tokens per Sec:    18263, Lr: 0.000300\n",
      "2019-10-30 11:51:47,184 Epoch   2 Step:    12900 Batch Loss:     1.854855 Tokens per Sec:    18424, Lr: 0.000300\n",
      "2019-10-30 11:52:01,165 Epoch   2 Step:    13000 Batch Loss:     2.209381 Tokens per Sec:    18579, Lr: 0.000300\n",
      "2019-10-30 11:52:42,284 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:52:42,284 Saving new checkpoint.\n",
      "2019-10-30 11:52:42,412 Example #0\n",
      "2019-10-30 11:52:42,412 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:52:42,412 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:52:42,412 \tHypothesis: Alikuwa amekuwa akitumia umri wa miaka 17 .\n",
      "2019-10-30 11:52:42,412 Example #1\n",
      "2019-10-30 11:52:42,412 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:52:42,412 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:52:42,412 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 11:52:42,413 Example #2\n",
      "2019-10-30 11:52:42,413 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:52:42,413 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:52:42,413 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuhangaika ?\n",
      "2019-10-30 11:52:42,413 Example #3\n",
      "2019-10-30 11:52:42,413 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:52:42,413 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:52:42,413 \tHypothesis: Sifa za kulevya na misiba .\n",
      "2019-10-30 11:52:42,413 Validation result at epoch   2, step    13000: bleu:  22.81, loss: 50830.5547, ppl:   6.3564, duration: 41.2476s\n",
      "2019-10-30 11:52:56,532 Epoch   2 Step:    13100 Batch Loss:     2.490395 Tokens per Sec:    18289, Lr: 0.000300\n",
      "2019-10-30 11:53:10,674 Epoch   2 Step:    13200 Batch Loss:     1.917078 Tokens per Sec:    18436, Lr: 0.000300\n",
      "2019-10-30 11:53:24,725 Epoch   2 Step:    13300 Batch Loss:     2.155499 Tokens per Sec:    18378, Lr: 0.000300\n",
      "2019-10-30 11:53:38,812 Epoch   2 Step:    13400 Batch Loss:     2.077295 Tokens per Sec:    18480, Lr: 0.000300\n",
      "2019-10-30 11:53:52,763 Epoch   2 Step:    13500 Batch Loss:     1.971167 Tokens per Sec:    17843, Lr: 0.000300\n",
      "2019-10-30 11:54:06,884 Epoch   2 Step:    13600 Batch Loss:     2.271123 Tokens per Sec:    18321, Lr: 0.000300\n",
      "2019-10-30 11:54:21,220 Epoch   2 Step:    13700 Batch Loss:     1.938679 Tokens per Sec:    18424, Lr: 0.000300\n",
      "2019-10-30 11:54:35,322 Epoch   2 Step:    13800 Batch Loss:     2.171431 Tokens per Sec:    18023, Lr: 0.000300\n",
      "2019-10-30 11:54:49,552 Epoch   2 Step:    13900 Batch Loss:     2.110486 Tokens per Sec:    18237, Lr: 0.000300\n",
      "2019-10-30 11:55:03,687 Epoch   2 Step:    14000 Batch Loss:     2.113237 Tokens per Sec:    18358, Lr: 0.000300\n",
      "2019-10-30 11:56:04,232 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:56:04,233 Saving new checkpoint.\n",
      "2019-10-30 11:56:04,707 Example #0\n",
      "2019-10-30 11:56:04,707 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:56:04,707 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:56:04,708 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 11:56:04,708 Example #1\n",
      "2019-10-30 11:56:04,708 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:56:04,708 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:56:04,709 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 11:56:04,709 Example #2\n",
      "2019-10-30 11:56:04,709 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:56:04,709 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:56:04,710 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:56:04,710 Example #3\n",
      "2019-10-30 11:56:04,710 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:56:04,710 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:56:04,711 \tHypothesis: Madhara ya maua na maua .\n",
      "2019-10-30 11:56:04,711 Validation result at epoch   2, step    14000: bleu:  23.40, loss: 49828.4883, ppl:   6.1288, duration: 61.0237s\n",
      "2019-10-30 11:56:34,934 Epoch   2 Step:    14100 Batch Loss:     2.279988 Tokens per Sec:     8607, Lr: 0.000300\n",
      "2019-10-30 11:56:55,562 Epoch   2 Step:    14200 Batch Loss:     1.849141 Tokens per Sec:    12203, Lr: 0.000300\n",
      "2019-10-30 11:57:10,691 Epoch   2 Step:    14300 Batch Loss:     2.062511 Tokens per Sec:    16907, Lr: 0.000300\n",
      "2019-10-30 11:57:25,650 Epoch   2 Step:    14400 Batch Loss:     2.229585 Tokens per Sec:    17781, Lr: 0.000300\n",
      "2019-10-30 11:57:40,348 Epoch   2 Step:    14500 Batch Loss:     2.230510 Tokens per Sec:    17606, Lr: 0.000300\n",
      "2019-10-30 11:57:54,718 Epoch   2 Step:    14600 Batch Loss:     1.932376 Tokens per Sec:    17525, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 11:58:08,974 Epoch   2 Step:    14700 Batch Loss:     1.859396 Tokens per Sec:    17908, Lr: 0.000300\n",
      "2019-10-30 11:58:22,972 Epoch   2 Step:    14800 Batch Loss:     1.997756 Tokens per Sec:    17988, Lr: 0.000300\n",
      "2019-10-30 11:58:37,229 Epoch   2 Step:    14900 Batch Loss:     2.311798 Tokens per Sec:    18149, Lr: 0.000300\n",
      "2019-10-30 11:58:51,424 Epoch   2 Step:    15000 Batch Loss:     2.487770 Tokens per Sec:    18255, Lr: 0.000300\n",
      "2019-10-30 11:59:33,353 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 11:59:33,353 Saving new checkpoint.\n",
      "2019-10-30 11:59:33,487 Example #0\n",
      "2019-10-30 11:59:33,487 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 11:59:33,487 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 11:59:33,487 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 11:59:33,487 Example #1\n",
      "2019-10-30 11:59:33,487 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 11:59:33,488 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 11:59:33,488 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 11:59:33,488 Example #2\n",
      "2019-10-30 11:59:33,488 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 11:59:33,488 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 11:59:33,488 \tHypothesis: Ni mambo gani ambayo husababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 11:59:33,488 Example #3\n",
      "2019-10-30 11:59:33,488 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 11:59:33,488 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 11:59:33,488 \tHypothesis: Mapenzi ya maua na maua .\n",
      "2019-10-30 11:59:33,488 Validation result at epoch   2, step    15000: bleu:  23.88, loss: 48824.3750, ppl:   5.9089, duration: 42.0634s\n",
      "2019-10-30 11:59:47,601 Epoch   2 Step:    15100 Batch Loss:     1.822730 Tokens per Sec:    17617, Lr: 0.000300\n",
      "2019-10-30 12:00:01,835 Epoch   2 Step:    15200 Batch Loss:     2.157866 Tokens per Sec:    18147, Lr: 0.000300\n",
      "2019-10-30 12:00:15,954 Epoch   2 Step:    15300 Batch Loss:     2.238588 Tokens per Sec:    18000, Lr: 0.000300\n",
      "2019-10-30 12:00:30,088 Epoch   2 Step:    15400 Batch Loss:     1.857473 Tokens per Sec:    18013, Lr: 0.000300\n",
      "2019-10-30 12:00:44,289 Epoch   2 Step:    15500 Batch Loss:     1.907517 Tokens per Sec:    18081, Lr: 0.000300\n",
      "2019-10-30 12:00:58,578 Epoch   2 Step:    15600 Batch Loss:     2.286308 Tokens per Sec:    17958, Lr: 0.000300\n",
      "2019-10-30 12:01:12,901 Epoch   2 Step:    15700 Batch Loss:     1.955860 Tokens per Sec:    18039, Lr: 0.000300\n",
      "2019-10-30 12:01:27,251 Epoch   2 Step:    15800 Batch Loss:     1.880741 Tokens per Sec:    18456, Lr: 0.000300\n",
      "2019-10-30 12:01:41,888 Epoch   2 Step:    15900 Batch Loss:     2.143711 Tokens per Sec:    17644, Lr: 0.000300\n",
      "2019-10-30 12:01:56,325 Epoch   2 Step:    16000 Batch Loss:     1.869092 Tokens per Sec:    17834, Lr: 0.000300\n",
      "2019-10-30 12:02:38,970 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:02:38,970 Saving new checkpoint.\n",
      "2019-10-30 12:02:39,105 Example #0\n",
      "2019-10-30 12:02:39,105 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:02:39,105 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:02:39,105 \tHypothesis: Alikuwa mzee wa umri wa miaka 17 .\n",
      "2019-10-30 12:02:39,105 Example #1\n",
      "2019-10-30 12:02:39,106 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:02:39,106 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:02:39,106 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 12:02:39,106 Example #2\n",
      "2019-10-30 12:02:39,106 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:02:39,106 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:02:39,106 \tHypothesis: Ni mambo gani ambayo husababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 12:02:39,106 Example #3\n",
      "2019-10-30 12:02:39,106 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:02:39,106 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:02:39,106 \tHypothesis: Mapigano ya maua na misuli .\n",
      "2019-10-30 12:02:39,106 Validation result at epoch   2, step    16000: bleu:  24.77, loss: 47932.8008, ppl:   5.7203, duration: 42.7807s\n",
      "2019-10-30 12:02:53,624 Epoch   2 Step:    16100 Batch Loss:     1.993386 Tokens per Sec:    17968, Lr: 0.000300\n",
      "2019-10-30 12:03:07,960 Epoch   2 Step:    16200 Batch Loss:     2.058195 Tokens per Sec:    18065, Lr: 0.000300\n",
      "2019-10-30 12:03:22,123 Epoch   2 Step:    16300 Batch Loss:     1.808439 Tokens per Sec:    17715, Lr: 0.000300\n",
      "2019-10-30 12:03:36,450 Epoch   2 Step:    16400 Batch Loss:     1.816901 Tokens per Sec:    18082, Lr: 0.000300\n",
      "2019-10-30 12:03:50,525 Epoch   2 Step:    16500 Batch Loss:     2.314456 Tokens per Sec:    17800, Lr: 0.000300\n",
      "2019-10-30 12:04:04,735 Epoch   2 Step:    16600 Batch Loss:     1.696109 Tokens per Sec:    18185, Lr: 0.000300\n",
      "2019-10-30 12:04:19,236 Epoch   2 Step:    16700 Batch Loss:     2.002882 Tokens per Sec:    17893, Lr: 0.000300\n",
      "2019-10-30 12:04:33,514 Epoch   2 Step:    16800 Batch Loss:     1.958494 Tokens per Sec:    17577, Lr: 0.000300\n",
      "2019-10-30 12:04:47,827 Epoch   2 Step:    16900 Batch Loss:     2.292253 Tokens per Sec:    18012, Lr: 0.000300\n",
      "2019-10-30 12:05:02,219 Epoch   2 Step:    17000 Batch Loss:     1.983234 Tokens per Sec:    17951, Lr: 0.000300\n",
      "2019-10-30 12:05:44,371 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:05:44,371 Saving new checkpoint.\n",
      "2019-10-30 12:05:44,502 Example #0\n",
      "2019-10-30 12:05:44,502 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:05:44,503 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:05:44,503 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:05:44,503 Example #1\n",
      "2019-10-30 12:05:44,503 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:05:44,503 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:05:44,503 \tHypothesis: Biblia haipatani uumbaji .\n",
      "2019-10-30 12:05:44,503 Example #2\n",
      "2019-10-30 12:05:44,503 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:05:44,503 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:05:44,503 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 12:05:44,503 Example #3\n",
      "2019-10-30 12:05:44,503 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:05:44,503 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:05:44,503 \tHypothesis: Mapendwa na wavi .\n",
      "2019-10-30 12:05:44,503 Validation result at epoch   2, step    17000: bleu:  25.35, loss: 47172.9258, ppl:   5.5643, duration: 42.2833s\n",
      "2019-10-30 12:05:58,822 Epoch   2 Step:    17100 Batch Loss:     1.937456 Tokens per Sec:    17783, Lr: 0.000300\n",
      "2019-10-30 12:06:13,298 Epoch   2 Step:    17200 Batch Loss:     2.157204 Tokens per Sec:    17822, Lr: 0.000300\n",
      "2019-10-30 12:06:27,590 Epoch   2 Step:    17300 Batch Loss:     1.800814 Tokens per Sec:    17881, Lr: 0.000300\n",
      "2019-10-30 12:06:41,839 Epoch   2 Step:    17400 Batch Loss:     1.731453 Tokens per Sec:    17969, Lr: 0.000300\n",
      "2019-10-30 12:06:56,148 Epoch   2 Step:    17500 Batch Loss:     1.890215 Tokens per Sec:    18131, Lr: 0.000300\n",
      "2019-10-30 12:07:10,513 Epoch   2 Step:    17600 Batch Loss:     2.182040 Tokens per Sec:    18142, Lr: 0.000300\n",
      "2019-10-30 12:07:24,835 Epoch   2 Step:    17700 Batch Loss:     1.993890 Tokens per Sec:    18240, Lr: 0.000300\n",
      "2019-10-30 12:07:39,051 Epoch   2 Step:    17800 Batch Loss:     1.909092 Tokens per Sec:    17814, Lr: 0.000300\n",
      "2019-10-30 12:07:53,430 Epoch   2 Step:    17900 Batch Loss:     1.753155 Tokens per Sec:    18140, Lr: 0.000300\n",
      "2019-10-30 12:08:07,669 Epoch   2 Step:    18000 Batch Loss:     2.058256 Tokens per Sec:    17907, Lr: 0.000300\n",
      "2019-10-30 12:08:50,229 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:08:50,229 Saving new checkpoint.\n",
      "2019-10-30 12:08:50,360 Example #0\n",
      "2019-10-30 12:08:50,360 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:08:50,360 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:08:50,360 \tHypothesis: Alikuwa mzee katika umri wa miaka 17 .\n",
      "2019-10-30 12:08:50,360 Example #1\n",
      "2019-10-30 12:08:50,360 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:08:50,361 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:08:50,361 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:08:50,361 Example #2\n",
      "2019-10-30 12:08:50,361 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:08:50,361 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:08:50,361 \tHypothesis: Ni mambo gani ambayo yanafanya hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 12:08:50,361 Example #3\n",
      "2019-10-30 12:08:50,361 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:08:50,361 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:08:50,361 \tHypothesis: Madhara ya vicheo na misuli .\n",
      "2019-10-30 12:08:50,361 Validation result at epoch   2, step    18000: bleu:  25.92, loss: 46360.5352, ppl:   5.4023, duration: 42.6916s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 12:09:04,716 Epoch   2 Step:    18100 Batch Loss:     2.027657 Tokens per Sec:    17731, Lr: 0.000300\n",
      "2019-10-30 12:09:19,045 Epoch   2 Step:    18200 Batch Loss:     1.850734 Tokens per Sec:    17989, Lr: 0.000300\n",
      "2019-10-30 12:09:33,445 Epoch   2 Step:    18300 Batch Loss:     1.717876 Tokens per Sec:    18000, Lr: 0.000300\n",
      "2019-10-30 12:09:47,732 Epoch   2 Step:    18400 Batch Loss:     1.831221 Tokens per Sec:    17691, Lr: 0.000300\n",
      "2019-10-30 12:10:02,189 Epoch   2 Step:    18500 Batch Loss:     2.193217 Tokens per Sec:    18118, Lr: 0.000300\n",
      "2019-10-30 12:10:16,518 Epoch   2 Step:    18600 Batch Loss:     1.776230 Tokens per Sec:    18051, Lr: 0.000300\n",
      "2019-10-30 12:10:30,687 Epoch   2 Step:    18700 Batch Loss:     2.029525 Tokens per Sec:    17717, Lr: 0.000300\n",
      "2019-10-30 12:10:41,587 Epoch   2: total training loss 19412.54\n",
      "2019-10-30 12:10:41,588 EPOCH 3\n",
      "2019-10-30 12:10:46,170 Epoch   3 Step:    18800 Batch Loss:     1.674996 Tokens per Sec:    13659, Lr: 0.000300\n",
      "2019-10-30 12:11:00,358 Epoch   3 Step:    18900 Batch Loss:     2.112353 Tokens per Sec:    18004, Lr: 0.000300\n",
      "2019-10-30 12:11:14,571 Epoch   3 Step:    19000 Batch Loss:     1.688622 Tokens per Sec:    17764, Lr: 0.000300\n",
      "2019-10-30 12:11:56,864 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:11:56,864 Saving new checkpoint.\n",
      "2019-10-30 12:11:57,038 Example #0\n",
      "2019-10-30 12:11:57,038 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:11:57,038 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:11:57,038 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:11:57,038 Example #1\n",
      "2019-10-30 12:11:57,038 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:11:57,038 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:11:57,038 \tHypothesis: Biblia haisemi uumbaji wa uumbaji .\n",
      "2019-10-30 12:11:57,038 Example #2\n",
      "2019-10-30 12:11:57,038 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:11:57,038 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:11:57,038 \tHypothesis: Ni mambo gani ambayo husababisha hangaiko , na ni mahangaiko gani yanayoweza kuathiri ?\n",
      "2019-10-30 12:11:57,039 Example #3\n",
      "2019-10-30 12:11:57,039 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:11:57,039 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:11:57,039 \tHypothesis: Mapigano ya maua na magari .\n",
      "2019-10-30 12:11:57,039 Validation result at epoch   3, step    19000: bleu:  25.97, loss: 46063.8711, ppl:   5.3443, duration: 42.4671s\n",
      "2019-10-30 12:12:11,353 Epoch   3 Step:    19100 Batch Loss:     1.614666 Tokens per Sec:    18122, Lr: 0.000300\n",
      "2019-10-30 12:12:25,617 Epoch   3 Step:    19200 Batch Loss:     2.378168 Tokens per Sec:    17760, Lr: 0.000300\n",
      "2019-10-30 12:12:39,937 Epoch   3 Step:    19300 Batch Loss:     1.855685 Tokens per Sec:    17971, Lr: 0.000300\n",
      "2019-10-30 12:12:54,101 Epoch   3 Step:    19400 Batch Loss:     1.936510 Tokens per Sec:    17909, Lr: 0.000300\n",
      "2019-10-30 12:13:08,414 Epoch   3 Step:    19500 Batch Loss:     1.960813 Tokens per Sec:    17798, Lr: 0.000300\n",
      "2019-10-30 12:13:22,750 Epoch   3 Step:    19600 Batch Loss:     1.747046 Tokens per Sec:    18136, Lr: 0.000300\n",
      "2019-10-30 12:13:37,141 Epoch   3 Step:    19700 Batch Loss:     1.574521 Tokens per Sec:    18016, Lr: 0.000300\n",
      "2019-10-30 12:13:51,477 Epoch   3 Step:    19800 Batch Loss:     1.620596 Tokens per Sec:    17861, Lr: 0.000300\n",
      "2019-10-30 12:14:05,905 Epoch   3 Step:    19900 Batch Loss:     1.854833 Tokens per Sec:    17776, Lr: 0.000300\n",
      "2019-10-30 12:14:20,226 Epoch   3 Step:    20000 Batch Loss:     1.867777 Tokens per Sec:    17800, Lr: 0.000300\n",
      "2019-10-30 12:15:02,644 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:15:02,644 Saving new checkpoint.\n",
      "2019-10-30 12:15:02,776 Example #0\n",
      "2019-10-30 12:15:02,776 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:15:02,776 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:15:02,776 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:15:02,776 Example #1\n",
      "2019-10-30 12:15:02,776 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:15:02,776 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:15:02,776 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 12:15:02,776 Example #2\n",
      "2019-10-30 12:15:02,776 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:15:02,776 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:15:02,776 \tHypothesis: Ni mambo gani ambayo yanaweza kusababisha hangaiko , na ni hangaiko gani ?\n",
      "2019-10-30 12:15:02,776 Example #3\n",
      "2019-10-30 12:15:02,777 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:15:02,777 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:15:02,777 \tHypothesis: Mapendezi na misiba .\n",
      "2019-10-30 12:15:02,777 Validation result at epoch   3, step    20000: bleu:  26.61, loss: 45050.6680, ppl:   5.1508, duration: 42.5503s\n",
      "2019-10-30 12:15:17,144 Epoch   3 Step:    20100 Batch Loss:     1.860777 Tokens per Sec:    17946, Lr: 0.000300\n",
      "2019-10-30 12:15:31,683 Epoch   3 Step:    20200 Batch Loss:     2.699334 Tokens per Sec:    18315, Lr: 0.000300\n",
      "2019-10-30 12:15:45,995 Epoch   3 Step:    20300 Batch Loss:     1.759064 Tokens per Sec:    17770, Lr: 0.000300\n",
      "2019-10-30 12:16:00,317 Epoch   3 Step:    20400 Batch Loss:     1.826943 Tokens per Sec:    18047, Lr: 0.000300\n",
      "2019-10-30 12:16:14,508 Epoch   3 Step:    20500 Batch Loss:     2.016803 Tokens per Sec:    17767, Lr: 0.000300\n",
      "2019-10-30 12:16:28,705 Epoch   3 Step:    20600 Batch Loss:     1.814761 Tokens per Sec:    17896, Lr: 0.000300\n",
      "2019-10-30 12:16:42,995 Epoch   3 Step:    20700 Batch Loss:     1.641840 Tokens per Sec:    17615, Lr: 0.000300\n",
      "2019-10-30 12:16:57,372 Epoch   3 Step:    20800 Batch Loss:     1.864856 Tokens per Sec:    17857, Lr: 0.000300\n",
      "2019-10-30 12:17:11,697 Epoch   3 Step:    20900 Batch Loss:     1.837613 Tokens per Sec:    17606, Lr: 0.000300\n",
      "2019-10-30 12:17:26,035 Epoch   3 Step:    21000 Batch Loss:     1.872529 Tokens per Sec:    17748, Lr: 0.000300\n",
      "2019-10-30 12:18:08,555 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:18:08,555 Saving new checkpoint.\n",
      "2019-10-30 12:18:08,693 Example #0\n",
      "2019-10-30 12:18:08,693 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:18:08,693 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:18:08,693 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:18:08,694 Example #1\n",
      "2019-10-30 12:18:08,694 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:18:08,694 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:18:08,694 \tHypothesis: Biblia haisemi uumbaji wa uumbaji .\n",
      "2019-10-30 12:18:08,694 Example #2\n",
      "2019-10-30 12:18:08,694 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:18:08,694 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:18:08,694 \tHypothesis: Ni mambo gani ambayo husababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 12:18:08,694 Example #3\n",
      "2019-10-30 12:18:08,694 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:18:08,694 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:18:08,694 \tHypothesis: Mengi ya maua na mishipa .\n",
      "2019-10-30 12:18:08,694 Validation result at epoch   3, step    21000: bleu:  26.45, loss: 44776.5508, ppl:   5.0997, duration: 42.6589s\n",
      "2019-10-30 12:18:23,284 Epoch   3 Step:    21100 Batch Loss:     1.698489 Tokens per Sec:    17459, Lr: 0.000300\n",
      "2019-10-30 12:18:37,642 Epoch   3 Step:    21200 Batch Loss:     1.811205 Tokens per Sec:    17778, Lr: 0.000300\n",
      "2019-10-30 12:18:52,081 Epoch   3 Step:    21300 Batch Loss:     1.786747 Tokens per Sec:    18141, Lr: 0.000300\n",
      "2019-10-30 12:19:06,348 Epoch   3 Step:    21400 Batch Loss:     1.786142 Tokens per Sec:    18207, Lr: 0.000300\n",
      "2019-10-30 12:19:20,761 Epoch   3 Step:    21500 Batch Loss:     1.660817 Tokens per Sec:    18366, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 12:19:34,996 Epoch   3 Step:    21600 Batch Loss:     1.812444 Tokens per Sec:    17870, Lr: 0.000300\n",
      "2019-10-30 12:19:49,145 Epoch   3 Step:    21700 Batch Loss:     1.856580 Tokens per Sec:    17716, Lr: 0.000300\n",
      "2019-10-30 12:20:03,557 Epoch   3 Step:    21800 Batch Loss:     1.746711 Tokens per Sec:    17575, Lr: 0.000300\n",
      "2019-10-30 12:20:17,978 Epoch   3 Step:    21900 Batch Loss:     1.510386 Tokens per Sec:    17840, Lr: 0.000300\n",
      "2019-10-30 12:20:32,389 Epoch   3 Step:    22000 Batch Loss:     1.882905 Tokens per Sec:    17978, Lr: 0.000300\n",
      "2019-10-30 12:21:15,217 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:21:15,217 Saving new checkpoint.\n",
      "2019-10-30 12:21:15,359 Example #0\n",
      "2019-10-30 12:21:15,359 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:21:15,359 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:21:15,359 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:21:15,359 Example #1\n",
      "2019-10-30 12:21:15,359 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:21:15,359 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:21:15,359 \tHypothesis: Biblia haitoi uumbaji .\n",
      "2019-10-30 12:21:15,359 Example #2\n",
      "2019-10-30 12:21:15,359 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:21:15,359 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:21:15,359 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuchangia mahangaiko ?\n",
      "2019-10-30 12:21:15,359 Example #3\n",
      "2019-10-30 12:21:15,360 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:21:15,360 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:21:15,360 \tHypothesis: Madokezo ya maua na mizigo .\n",
      "2019-10-30 12:21:15,360 Validation result at epoch   3, step    22000: bleu:  27.41, loss: 44034.1602, ppl:   4.9638, duration: 42.9704s\n",
      "2019-10-30 12:21:30,036 Epoch   3 Step:    22100 Batch Loss:     1.652390 Tokens per Sec:    17708, Lr: 0.000300\n",
      "2019-10-30 12:21:44,455 Epoch   3 Step:    22200 Batch Loss:     1.600336 Tokens per Sec:    18184, Lr: 0.000300\n",
      "2019-10-30 12:21:58,681 Epoch   3 Step:    22300 Batch Loss:     1.579208 Tokens per Sec:    17865, Lr: 0.000300\n",
      "2019-10-30 12:22:13,156 Epoch   3 Step:    22400 Batch Loss:     1.710020 Tokens per Sec:    17974, Lr: 0.000300\n",
      "2019-10-30 12:22:27,765 Epoch   3 Step:    22500 Batch Loss:     1.546571 Tokens per Sec:    18041, Lr: 0.000300\n",
      "2019-10-30 12:22:42,177 Epoch   3 Step:    22600 Batch Loss:     1.852573 Tokens per Sec:    17925, Lr: 0.000300\n",
      "2019-10-30 12:22:56,536 Epoch   3 Step:    22700 Batch Loss:     1.931437 Tokens per Sec:    17875, Lr: 0.000300\n",
      "2019-10-30 12:23:11,026 Epoch   3 Step:    22800 Batch Loss:     1.966926 Tokens per Sec:    17748, Lr: 0.000300\n",
      "2019-10-30 12:23:25,343 Epoch   3 Step:    22900 Batch Loss:     1.676519 Tokens per Sec:    17949, Lr: 0.000300\n",
      "2019-10-30 12:23:39,791 Epoch   3 Step:    23000 Batch Loss:     1.578513 Tokens per Sec:    18243, Lr: 0.000300\n",
      "2019-10-30 12:24:22,079 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:24:22,079 Saving new checkpoint.\n",
      "2019-10-30 12:24:22,211 Example #0\n",
      "2019-10-30 12:24:22,212 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:24:22,212 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:24:22,212 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:24:22,212 Example #1\n",
      "2019-10-30 12:24:22,212 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:24:22,212 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:24:22,212 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:24:22,212 Example #2\n",
      "2019-10-30 12:24:22,212 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:24:22,212 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:24:22,212 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuchangia mahangaiko ?\n",
      "2019-10-30 12:24:22,212 Example #3\n",
      "2019-10-30 12:24:22,212 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:24:22,212 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:24:22,212 \tHypothesis: Mapenzi ya maua na mishipa .\n",
      "2019-10-30 12:24:22,213 Validation result at epoch   3, step    23000: bleu:  27.42, loss: 43545.2695, ppl:   4.8763, duration: 42.4206s\n",
      "2019-10-30 12:24:36,423 Epoch   3 Step:    23100 Batch Loss:     1.824455 Tokens per Sec:    17741, Lr: 0.000300\n",
      "2019-10-30 12:24:50,757 Epoch   3 Step:    23200 Batch Loss:     1.781704 Tokens per Sec:    18036, Lr: 0.000300\n",
      "2019-10-30 12:25:05,283 Epoch   3 Step:    23300 Batch Loss:     1.954730 Tokens per Sec:    17935, Lr: 0.000300\n",
      "2019-10-30 12:25:19,755 Epoch   3 Step:    23400 Batch Loss:     1.592439 Tokens per Sec:    18171, Lr: 0.000300\n",
      "2019-10-30 12:25:34,048 Epoch   3 Step:    23500 Batch Loss:     1.668018 Tokens per Sec:    17964, Lr: 0.000300\n",
      "2019-10-30 12:25:48,236 Epoch   3 Step:    23600 Batch Loss:     1.823569 Tokens per Sec:    17765, Lr: 0.000300\n",
      "2019-10-30 12:26:02,556 Epoch   3 Step:    23700 Batch Loss:     1.634111 Tokens per Sec:    18060, Lr: 0.000300\n",
      "2019-10-30 12:26:16,948 Epoch   3 Step:    23800 Batch Loss:     1.683555 Tokens per Sec:    18258, Lr: 0.000300\n",
      "2019-10-30 12:26:31,137 Epoch   3 Step:    23900 Batch Loss:     1.785422 Tokens per Sec:    17803, Lr: 0.000300\n",
      "2019-10-30 12:26:45,224 Epoch   3 Step:    24000 Batch Loss:     1.766779 Tokens per Sec:    17714, Lr: 0.000300\n",
      "2019-10-30 12:27:27,414 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:27:27,414 Saving new checkpoint.\n",
      "2019-10-30 12:27:27,543 Example #0\n",
      "2019-10-30 12:27:27,544 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:27:27,544 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:27:27,544 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:27:27,544 Example #1\n",
      "2019-10-30 12:27:27,544 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:27:27,544 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:27:27,544 \tHypothesis: Biblia haitoshi uumbaji wa uumbaji .\n",
      "2019-10-30 12:27:27,544 Example #2\n",
      "2019-10-30 12:27:27,544 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:27:27,544 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:27:27,544 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 12:27:27,544 Example #3\n",
      "2019-10-30 12:27:27,544 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:27:27,544 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:27:27,544 \tHypothesis: Mbinu za ununyevu na magari .\n",
      "2019-10-30 12:27:27,544 Validation result at epoch   3, step    24000: bleu:  27.59, loss: 43294.8984, ppl:   4.8321, duration: 42.3202s\n",
      "2019-10-30 12:27:41,743 Epoch   3 Step:    24100 Batch Loss:     1.826069 Tokens per Sec:    18075, Lr: 0.000300\n",
      "2019-10-30 12:27:56,060 Epoch   3 Step:    24200 Batch Loss:     1.588056 Tokens per Sec:    18145, Lr: 0.000300\n",
      "2019-10-30 12:28:10,353 Epoch   3 Step:    24300 Batch Loss:     1.390271 Tokens per Sec:    18114, Lr: 0.000300\n",
      "2019-10-30 12:28:24,375 Epoch   3 Step:    24400 Batch Loss:     1.620476 Tokens per Sec:    18031, Lr: 0.000300\n",
      "2019-10-30 12:28:38,485 Epoch   3 Step:    24500 Batch Loss:     1.843006 Tokens per Sec:    18016, Lr: 0.000300\n",
      "2019-10-30 12:28:52,772 Epoch   3 Step:    24600 Batch Loss:     2.059666 Tokens per Sec:    18357, Lr: 0.000300\n",
      "2019-10-30 12:29:06,849 Epoch   3 Step:    24700 Batch Loss:     1.862025 Tokens per Sec:    17979, Lr: 0.000300\n",
      "2019-10-30 12:29:20,979 Epoch   3 Step:    24800 Batch Loss:     1.862438 Tokens per Sec:    18311, Lr: 0.000300\n",
      "2019-10-30 12:29:35,054 Epoch   3 Step:    24900 Batch Loss:     1.610338 Tokens per Sec:    18148, Lr: 0.000300\n",
      "2019-10-30 12:29:48,970 Epoch   3 Step:    25000 Batch Loss:     1.771996 Tokens per Sec:    17960, Lr: 0.000300\n",
      "2019-10-30 12:30:30,877 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:30:30,878 Saving new checkpoint.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 12:30:31,006 Example #0\n",
      "2019-10-30 12:30:31,006 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:30:31,006 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:30:31,006 \tHypothesis: Alikuwa mzee wa umri wa miaka 17 .\n",
      "2019-10-30 12:30:31,006 Example #1\n",
      "2019-10-30 12:30:31,006 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:30:31,006 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:30:31,006 \tHypothesis: Biblia haitegemei uumbaji .\n",
      "2019-10-30 12:30:31,006 Example #2\n",
      "2019-10-30 12:30:31,006 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:30:31,006 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:30:31,006 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 12:30:31,006 Example #3\n",
      "2019-10-30 12:30:31,007 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:30:31,007 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:30:31,007 \tHypothesis: Mapenzi ya maua na mizigo .\n",
      "2019-10-30 12:30:31,007 Validation result at epoch   3, step    25000: bleu:  28.40, loss: 42622.9258, ppl:   4.7154, duration: 42.0366s\n",
      "2019-10-30 12:30:45,052 Epoch   3 Step:    25100 Batch Loss:     1.928742 Tokens per Sec:    18424, Lr: 0.000300\n",
      "2019-10-30 12:30:59,088 Epoch   3 Step:    25200 Batch Loss:     1.641738 Tokens per Sec:    18028, Lr: 0.000300\n",
      "2019-10-30 12:31:13,276 Epoch   3 Step:    25300 Batch Loss:     1.868585 Tokens per Sec:    18513, Lr: 0.000300\n",
      "2019-10-30 12:31:27,460 Epoch   3 Step:    25400 Batch Loss:     2.039883 Tokens per Sec:    18042, Lr: 0.000300\n",
      "2019-10-30 12:31:41,450 Epoch   3 Step:    25500 Batch Loss:     1.914617 Tokens per Sec:    17956, Lr: 0.000300\n",
      "2019-10-30 12:31:55,530 Epoch   3 Step:    25600 Batch Loss:     1.759983 Tokens per Sec:    18282, Lr: 0.000300\n",
      "2019-10-30 12:32:09,664 Epoch   3 Step:    25700 Batch Loss:     1.828772 Tokens per Sec:    18091, Lr: 0.000300\n",
      "2019-10-30 12:32:23,715 Epoch   3 Step:    25800 Batch Loss:     1.789065 Tokens per Sec:    18042, Lr: 0.000300\n",
      "2019-10-30 12:32:37,667 Epoch   3 Step:    25900 Batch Loss:     1.785413 Tokens per Sec:    18130, Lr: 0.000300\n",
      "2019-10-30 12:32:51,823 Epoch   3 Step:    26000 Batch Loss:     1.652664 Tokens per Sec:    18510, Lr: 0.000300\n",
      "2019-10-30 12:33:33,660 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:33:33,660 Saving new checkpoint.\n",
      "2019-10-30 12:33:33,789 Example #0\n",
      "2019-10-30 12:33:33,789 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:33:33,789 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:33:33,789 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:33:33,789 Example #1\n",
      "2019-10-30 12:33:33,789 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:33:33,789 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:33:33,790 \tHypothesis: Biblia haisemi uumbaji wa uumbaji .\n",
      "2019-10-30 12:33:33,790 Example #2\n",
      "2019-10-30 12:33:33,790 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:33:33,790 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:33:33,790 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na ni nini kinachoweza kuchangamana ?\n",
      "2019-10-30 12:33:33,790 Example #3\n",
      "2019-10-30 12:33:33,790 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:33:33,790 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:33:33,790 \tHypothesis: Mapenda - mabawa na magari .\n",
      "2019-10-30 12:33:33,790 Validation result at epoch   3, step    26000: bleu:  28.18, loss: 42317.8633, ppl:   4.6633, duration: 41.9664s\n",
      "2019-10-30 12:33:47,743 Epoch   3 Step:    26100 Batch Loss:     1.581351 Tokens per Sec:    18005, Lr: 0.000300\n",
      "2019-10-30 12:34:01,930 Epoch   3 Step:    26200 Batch Loss:     1.564553 Tokens per Sec:    17888, Lr: 0.000300\n",
      "2019-10-30 12:34:16,151 Epoch   3 Step:    26300 Batch Loss:     1.799204 Tokens per Sec:    18351, Lr: 0.000300\n",
      "2019-10-30 12:34:30,275 Epoch   3 Step:    26400 Batch Loss:     1.599608 Tokens per Sec:    18396, Lr: 0.000300\n",
      "2019-10-30 12:34:44,390 Epoch   3 Step:    26500 Batch Loss:     1.610226 Tokens per Sec:    18430, Lr: 0.000300\n",
      "2019-10-30 12:34:58,478 Epoch   3 Step:    26600 Batch Loss:     1.600829 Tokens per Sec:    17888, Lr: 0.000300\n",
      "2019-10-30 12:35:12,747 Epoch   3 Step:    26700 Batch Loss:     1.655505 Tokens per Sec:    18381, Lr: 0.000300\n",
      "2019-10-30 12:35:26,929 Epoch   3 Step:    26800 Batch Loss:     1.752181 Tokens per Sec:    18134, Lr: 0.000300\n",
      "2019-10-30 12:35:41,083 Epoch   3 Step:    26900 Batch Loss:     1.797540 Tokens per Sec:    18105, Lr: 0.000300\n",
      "2019-10-30 12:35:55,230 Epoch   3 Step:    27000 Batch Loss:     1.806403 Tokens per Sec:    18187, Lr: 0.000300\n",
      "2019-10-30 12:36:37,328 Example #0\n",
      "2019-10-30 12:36:37,328 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:36:37,328 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:36:37,328 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:36:37,328 Example #1\n",
      "2019-10-30 12:36:37,328 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:36:37,328 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:36:37,328 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:36:37,328 Example #2\n",
      "2019-10-30 12:36:37,328 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:36:37,328 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:36:37,328 \tHypothesis: Ni mambo gani ambayo yanaweza kusababisha wasiwasi , na ni nini kinachoweza kuhangaika ?\n",
      "2019-10-30 12:36:37,328 Example #3\n",
      "2019-10-30 12:36:37,328 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:36:37,328 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:36:37,329 \tHypothesis: Mapendekezo ya nyuki na misiba .\n",
      "2019-10-30 12:36:37,329 Validation result at epoch   3, step    27000: bleu:  28.81, loss: 42408.5703, ppl:   4.6787, duration: 42.0985s\n",
      "2019-10-30 12:36:51,639 Epoch   3 Step:    27100 Batch Loss:     1.612879 Tokens per Sec:    17971, Lr: 0.000300\n",
      "2019-10-30 12:37:05,809 Epoch   3 Step:    27200 Batch Loss:     1.523602 Tokens per Sec:    18195, Lr: 0.000300\n",
      "2019-10-30 12:37:19,922 Epoch   3 Step:    27300 Batch Loss:     1.640847 Tokens per Sec:    18149, Lr: 0.000300\n",
      "2019-10-30 12:37:34,036 Epoch   3 Step:    27400 Batch Loss:     1.751133 Tokens per Sec:    18369, Lr: 0.000300\n",
      "2019-10-30 12:37:48,014 Epoch   3 Step:    27500 Batch Loss:     1.406017 Tokens per Sec:    18085, Lr: 0.000300\n",
      "2019-10-30 12:38:02,185 Epoch   3 Step:    27600 Batch Loss:     1.648213 Tokens per Sec:    18002, Lr: 0.000300\n",
      "2019-10-30 12:38:16,374 Epoch   3 Step:    27700 Batch Loss:     1.659708 Tokens per Sec:    18472, Lr: 0.000300\n",
      "2019-10-30 12:38:30,569 Epoch   3 Step:    27800 Batch Loss:     1.699628 Tokens per Sec:    18534, Lr: 0.000300\n",
      "2019-10-30 12:38:44,459 Epoch   3 Step:    27900 Batch Loss:     1.626025 Tokens per Sec:    17978, Lr: 0.000300\n",
      "2019-10-30 12:38:58,647 Epoch   3 Step:    28000 Batch Loss:     1.916330 Tokens per Sec:    18390, Lr: 0.000300\n",
      "2019-10-30 12:39:40,337 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:39:40,337 Saving new checkpoint.\n",
      "2019-10-30 12:39:40,464 Example #0\n",
      "2019-10-30 12:39:40,464 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:39:40,464 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:39:40,464 \tHypothesis: Alikuwa na umri wa miaka 17 .\n",
      "2019-10-30 12:39:40,464 Example #1\n",
      "2019-10-30 12:39:40,464 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:39:40,464 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:39:40,465 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:39:40,465 Example #2\n",
      "2019-10-30 12:39:40,465 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:39:40,465 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:39:40,465 \tHypothesis: Mambo fulani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 12:39:40,465 Example #3\n",
      "2019-10-30 12:39:40,465 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:39:40,465 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:39:40,465 \tHypothesis: Mapenzi ya maua na mizigo .\n",
      "2019-10-30 12:39:40,465 Validation result at epoch   3, step    28000: bleu:  29.25, loss: 41475.0781, ppl:   4.5225, duration: 41.8180s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 12:39:54,494 Epoch   3 Step:    28100 Batch Loss:     1.621880 Tokens per Sec:    18062, Lr: 0.000300\n",
      "2019-10-30 12:40:03,583 Epoch   3: total training loss 16838.32\n",
      "2019-10-30 12:40:03,584 EPOCH 4\n",
      "2019-10-30 12:40:09,735 Epoch   4 Step:    28200 Batch Loss:     1.638353 Tokens per Sec:    14928, Lr: 0.000300\n",
      "2019-10-30 12:40:23,643 Epoch   4 Step:    28300 Batch Loss:     2.567002 Tokens per Sec:    18006, Lr: 0.000300\n",
      "2019-10-30 12:40:37,896 Epoch   4 Step:    28400 Batch Loss:     1.828672 Tokens per Sec:    18417, Lr: 0.000300\n",
      "2019-10-30 12:40:52,198 Epoch   4 Step:    28500 Batch Loss:     1.710598 Tokens per Sec:    18033, Lr: 0.000300\n",
      "2019-10-30 12:41:06,495 Epoch   4 Step:    28600 Batch Loss:     1.614343 Tokens per Sec:    17923, Lr: 0.000300\n",
      "2019-10-30 12:41:20,643 Epoch   4 Step:    28700 Batch Loss:     1.584770 Tokens per Sec:    18142, Lr: 0.000300\n",
      "2019-10-30 12:41:34,698 Epoch   4 Step:    28800 Batch Loss:     1.883095 Tokens per Sec:    18157, Lr: 0.000300\n",
      "2019-10-30 12:41:48,946 Epoch   4 Step:    28900 Batch Loss:     1.820126 Tokens per Sec:    18239, Lr: 0.000300\n",
      "2019-10-30 12:42:03,056 Epoch   4 Step:    29000 Batch Loss:     1.588791 Tokens per Sec:    18119, Lr: 0.000300\n",
      "2019-10-30 12:42:44,929 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:42:44,929 Saving new checkpoint.\n",
      "2019-10-30 12:42:45,061 Example #0\n",
      "2019-10-30 12:42:45,061 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:42:45,061 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:42:45,061 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:42:45,062 Example #1\n",
      "2019-10-30 12:42:45,062 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:42:45,062 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:42:45,062 \tHypothesis: Biblia haionyeshi uumbaji .\n",
      "2019-10-30 12:42:45,062 Example #2\n",
      "2019-10-30 12:42:45,062 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:42:45,062 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:42:45,062 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 12:42:45,062 Example #3\n",
      "2019-10-30 12:42:45,062 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:42:45,062 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:42:45,062 \tHypothesis: Upendo wa nyuki na mistari .\n",
      "2019-10-30 12:42:45,062 Validation result at epoch   4, step    29000: bleu:  28.84, loss: 41200.6016, ppl:   4.4775, duration: 42.0054s\n",
      "2019-10-30 12:42:59,348 Epoch   4 Step:    29100 Batch Loss:     1.468450 Tokens per Sec:    18310, Lr: 0.000300\n",
      "2019-10-30 12:43:13,475 Epoch   4 Step:    29200 Batch Loss:     1.644696 Tokens per Sec:    18027, Lr: 0.000300\n",
      "2019-10-30 12:43:27,649 Epoch   4 Step:    29300 Batch Loss:     1.405469 Tokens per Sec:    18371, Lr: 0.000300\n",
      "2019-10-30 12:43:41,834 Epoch   4 Step:    29400 Batch Loss:     1.788920 Tokens per Sec:    18576, Lr: 0.000300\n",
      "2019-10-30 12:43:55,798 Epoch   4 Step:    29500 Batch Loss:     1.736027 Tokens per Sec:    18502, Lr: 0.000300\n",
      "2019-10-30 12:44:10,104 Epoch   4 Step:    29600 Batch Loss:     1.650035 Tokens per Sec:    18615, Lr: 0.000300\n",
      "2019-10-30 12:44:24,083 Epoch   4 Step:    29700 Batch Loss:     1.456809 Tokens per Sec:    18158, Lr: 0.000300\n",
      "2019-10-30 12:44:38,073 Epoch   4 Step:    29800 Batch Loss:     1.745042 Tokens per Sec:    18345, Lr: 0.000300\n",
      "2019-10-30 12:44:52,199 Epoch   4 Step:    29900 Batch Loss:     1.768766 Tokens per Sec:    18342, Lr: 0.000300\n",
      "2019-10-30 12:45:06,187 Epoch   4 Step:    30000 Batch Loss:     1.956672 Tokens per Sec:    18240, Lr: 0.000300\n",
      "2019-10-30 12:46:10,713 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:46:10,713 Saving new checkpoint.\n",
      "2019-10-30 12:46:11,316 Example #0\n",
      "2019-10-30 12:46:11,317 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:46:11,317 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:46:11,318 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:46:11,318 Example #1\n",
      "2019-10-30 12:46:11,318 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:46:11,319 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:46:11,319 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:46:11,319 Example #2\n",
      "2019-10-30 12:46:11,320 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:46:11,320 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:46:11,320 \tHypothesis: Ni mambo gani ambayo husababisha hangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 12:46:11,320 Example #3\n",
      "2019-10-30 12:46:11,321 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:46:11,321 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:46:11,321 \tHypothesis: Mapenzi ya aina na mishipa .\n",
      "2019-10-30 12:46:11,321 Validation result at epoch   4, step    30000: bleu:  29.90, loss: 40869.1641, ppl:   4.4239, duration: 65.1334s\n",
      "2019-10-30 12:46:39,433 Epoch   4 Step:    30100 Batch Loss:     1.766267 Tokens per Sec:     9248, Lr: 0.000300\n",
      "2019-10-30 12:46:54,972 Epoch   4 Step:    30200 Batch Loss:     1.495109 Tokens per Sec:    16312, Lr: 0.000300\n",
      "2019-10-30 12:47:10,217 Epoch   4 Step:    30300 Batch Loss:     1.561520 Tokens per Sec:    17308, Lr: 0.000300\n",
      "2019-10-30 12:47:24,766 Epoch   4 Step:    30400 Batch Loss:     1.791916 Tokens per Sec:    17193, Lr: 0.000300\n",
      "2019-10-30 12:47:39,009 Epoch   4 Step:    30500 Batch Loss:     1.391699 Tokens per Sec:    17705, Lr: 0.000300\n",
      "2019-10-30 12:47:53,317 Epoch   4 Step:    30600 Batch Loss:     1.759136 Tokens per Sec:    18445, Lr: 0.000300\n",
      "2019-10-30 12:48:07,386 Epoch   4 Step:    30700 Batch Loss:     1.542374 Tokens per Sec:    17851, Lr: 0.000300\n",
      "2019-10-30 12:48:21,362 Epoch   4 Step:    30800 Batch Loss:     1.644400 Tokens per Sec:    18095, Lr: 0.000300\n",
      "2019-10-30 12:48:35,113 Epoch   4 Step:    30900 Batch Loss:     1.488512 Tokens per Sec:    18269, Lr: 0.000300\n",
      "2019-10-30 12:48:49,033 Epoch   4 Step:    31000 Batch Loss:     1.522325 Tokens per Sec:    18454, Lr: 0.000300\n",
      "2019-10-30 12:49:30,341 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:49:30,341 Saving new checkpoint.\n",
      "2019-10-30 12:49:30,468 Example #0\n",
      "2019-10-30 12:49:30,468 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:49:30,468 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:49:30,468 \tHypothesis: Alikuwa mzee mwenye umri wa miaka 17 .\n",
      "2019-10-30 12:49:30,468 Example #1\n",
      "2019-10-30 12:49:30,468 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:49:30,468 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:49:30,468 \tHypothesis: Biblia haisemi uzuri .\n",
      "2019-10-30 12:49:30,468 Example #2\n",
      "2019-10-30 12:49:30,468 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:49:30,468 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:49:30,468 \tHypothesis: Ni mambo gani fulani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiriwa na nini ?\n",
      "2019-10-30 12:49:30,468 Example #3\n",
      "2019-10-30 12:49:30,469 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:49:30,469 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:49:30,469 \tHypothesis: Mapenzi ya nyuki na mabawa .\n",
      "2019-10-30 12:49:30,469 Validation result at epoch   4, step    31000: bleu:  29.48, loss: 40537.2930, ppl:   4.3708, duration: 41.4354s\n",
      "2019-10-30 12:49:44,485 Epoch   4 Step:    31100 Batch Loss:     1.646977 Tokens per Sec:    18425, Lr: 0.000300\n",
      "2019-10-30 12:49:58,480 Epoch   4 Step:    31200 Batch Loss:     1.495394 Tokens per Sec:    18511, Lr: 0.000300\n",
      "2019-10-30 12:50:12,484 Epoch   4 Step:    31300 Batch Loss:     1.396285 Tokens per Sec:    18247, Lr: 0.000300\n",
      "2019-10-30 12:50:26,351 Epoch   4 Step:    31400 Batch Loss:     1.815011 Tokens per Sec:    18275, Lr: 0.000300\n",
      "2019-10-30 12:50:40,485 Epoch   4 Step:    31500 Batch Loss:     1.767150 Tokens per Sec:    18510, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 12:50:54,574 Epoch   4 Step:    31600 Batch Loss:     1.633710 Tokens per Sec:    18593, Lr: 0.000300\n",
      "2019-10-30 12:51:08,573 Epoch   4 Step:    31700 Batch Loss:     1.855283 Tokens per Sec:    18548, Lr: 0.000300\n",
      "2019-10-30 12:51:22,553 Epoch   4 Step:    31800 Batch Loss:     1.812481 Tokens per Sec:    18416, Lr: 0.000300\n",
      "2019-10-30 12:51:36,580 Epoch   4 Step:    31900 Batch Loss:     1.507688 Tokens per Sec:    18063, Lr: 0.000300\n",
      "2019-10-30 12:51:50,603 Epoch   4 Step:    32000 Batch Loss:     1.733472 Tokens per Sec:    18470, Lr: 0.000300\n",
      "2019-10-30 12:52:39,188 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:52:39,188 Saving new checkpoint.\n",
      "2019-10-30 12:52:39,666 Example #0\n",
      "2019-10-30 12:52:39,667 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:52:39,667 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:52:39,667 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 12:52:39,667 Example #1\n",
      "2019-10-30 12:52:39,668 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:52:39,668 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:52:39,668 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:52:39,668 Example #2\n",
      "2019-10-30 12:52:39,669 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:52:39,669 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:52:39,669 \tHypothesis: Ni mambo gani fulani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 12:52:39,669 Example #3\n",
      "2019-10-30 12:52:39,670 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:52:39,670 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:52:39,670 \tHypothesis: Wapenda makaburi na mishahara .\n",
      "2019-10-30 12:52:39,670 Validation result at epoch   4, step    32000: bleu:  29.73, loss: 40215.5820, ppl:   4.3199, duration: 49.0671s\n",
      "2019-10-30 12:53:09,926 Epoch   4 Step:    32100 Batch Loss:     1.686234 Tokens per Sec:     8348, Lr: 0.000300\n",
      "2019-10-30 12:53:40,588 Epoch   4 Step:    32200 Batch Loss:     1.639914 Tokens per Sec:     8064, Lr: 0.000300\n",
      "2019-10-30 12:53:56,707 Epoch   4 Step:    32300 Batch Loss:     1.819942 Tokens per Sec:    15926, Lr: 0.000300\n",
      "2019-10-30 12:54:11,792 Epoch   4 Step:    32400 Batch Loss:     1.580621 Tokens per Sec:    16904, Lr: 0.000300\n",
      "2019-10-30 12:54:26,691 Epoch   4 Step:    32500 Batch Loss:     1.727517 Tokens per Sec:    17378, Lr: 0.000300\n",
      "2019-10-30 12:54:41,209 Epoch   4 Step:    32600 Batch Loss:     1.706188 Tokens per Sec:    17667, Lr: 0.000300\n",
      "2019-10-30 12:54:55,450 Epoch   4 Step:    32700 Batch Loss:     1.745537 Tokens per Sec:    18017, Lr: 0.000300\n",
      "2019-10-30 12:55:09,445 Epoch   4 Step:    32800 Batch Loss:     1.630658 Tokens per Sec:    18228, Lr: 0.000300\n",
      "2019-10-30 12:55:23,521 Epoch   4 Step:    32900 Batch Loss:     1.925041 Tokens per Sec:    18284, Lr: 0.000300\n",
      "2019-10-30 12:55:37,525 Epoch   4 Step:    33000 Batch Loss:     1.672146 Tokens per Sec:    18256, Lr: 0.000300\n",
      "2019-10-30 12:56:18,889 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:56:18,890 Saving new checkpoint.\n",
      "2019-10-30 12:56:19,019 Example #0\n",
      "2019-10-30 12:56:19,020 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:56:19,020 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:56:19,020 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:56:19,020 Example #1\n",
      "2019-10-30 12:56:19,020 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:56:19,020 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:56:19,020 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 12:56:19,020 Example #2\n",
      "2019-10-30 12:56:19,020 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:56:19,020 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:56:19,020 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiriwa na nini ?\n",
      "2019-10-30 12:56:19,020 Example #3\n",
      "2019-10-30 12:56:19,020 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:56:19,020 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:56:19,020 \tHypothesis: Wapendao na washindi .\n",
      "2019-10-30 12:56:19,020 Validation result at epoch   4, step    33000: bleu:  30.28, loss: 40052.1875, ppl:   4.2943, duration: 41.4950s\n",
      "2019-10-30 12:56:33,157 Epoch   4 Step:    33100 Batch Loss:     1.696776 Tokens per Sec:    18399, Lr: 0.000300\n",
      "2019-10-30 12:56:47,161 Epoch   4 Step:    33200 Batch Loss:     1.612405 Tokens per Sec:    18177, Lr: 0.000300\n",
      "2019-10-30 12:57:01,218 Epoch   4 Step:    33300 Batch Loss:     1.784528 Tokens per Sec:    18301, Lr: 0.000300\n",
      "2019-10-30 12:57:15,339 Epoch   4 Step:    33400 Batch Loss:     1.579383 Tokens per Sec:    18532, Lr: 0.000300\n",
      "2019-10-30 12:57:29,089 Epoch   4 Step:    33500 Batch Loss:     1.533076 Tokens per Sec:    18196, Lr: 0.000300\n",
      "2019-10-30 12:57:43,100 Epoch   4 Step:    33600 Batch Loss:     1.600290 Tokens per Sec:    18273, Lr: 0.000300\n",
      "2019-10-30 12:57:57,270 Epoch   4 Step:    33700 Batch Loss:     1.476711 Tokens per Sec:    18439, Lr: 0.000300\n",
      "2019-10-30 12:58:11,418 Epoch   4 Step:    33800 Batch Loss:     1.741171 Tokens per Sec:    18298, Lr: 0.000300\n",
      "2019-10-30 12:58:25,454 Epoch   4 Step:    33900 Batch Loss:     1.588861 Tokens per Sec:    18315, Lr: 0.000300\n",
      "2019-10-30 12:58:39,550 Epoch   4 Step:    34000 Batch Loss:     1.768170 Tokens per Sec:    18445, Lr: 0.000300\n",
      "2019-10-30 12:59:38,423 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 12:59:38,424 Saving new checkpoint.\n",
      "2019-10-30 12:59:38,907 Example #0\n",
      "2019-10-30 12:59:38,908 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 12:59:38,908 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 12:59:38,909 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 12:59:38,909 Example #1\n",
      "2019-10-30 12:59:38,909 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 12:59:38,909 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 12:59:38,910 \tHypothesis: Biblia haisemi uumbaji wa uumbaji .\n",
      "2019-10-30 12:59:38,910 Example #2\n",
      "2019-10-30 12:59:38,910 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 12:59:38,910 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 12:59:38,911 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kuathiriwa na nini ?\n",
      "2019-10-30 12:59:38,911 Example #3\n",
      "2019-10-30 12:59:38,911 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 12:59:38,911 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 12:59:38,912 \tHypothesis: Madokezo ya unyenyekevu na mishale .\n",
      "2019-10-30 12:59:38,912 Validation result at epoch   4, step    34000: bleu:  30.38, loss: 39650.9023, ppl:   4.2321, duration: 59.3613s\n",
      "2019-10-30 13:00:09,032 Epoch   4 Step:    34100 Batch Loss:     1.606019 Tokens per Sec:     8510, Lr: 0.000300\n",
      "2019-10-30 13:00:27,228 Epoch   4 Step:    34200 Batch Loss:     1.692521 Tokens per Sec:    14280, Lr: 0.000300\n",
      "2019-10-30 13:00:42,508 Epoch   4 Step:    34300 Batch Loss:     1.569147 Tokens per Sec:    17030, Lr: 0.000300\n",
      "2019-10-30 13:00:57,322 Epoch   4 Step:    34400 Batch Loss:     1.650231 Tokens per Sec:    17544, Lr: 0.000300\n",
      "2019-10-30 13:01:12,057 Epoch   4 Step:    34500 Batch Loss:     1.663544 Tokens per Sec:    17395, Lr: 0.000300\n",
      "2019-10-30 13:01:26,293 Epoch   4 Step:    34600 Batch Loss:     1.651224 Tokens per Sec:    18078, Lr: 0.000300\n",
      "2019-10-30 13:01:40,185 Epoch   4 Step:    34700 Batch Loss:     1.674614 Tokens per Sec:    18271, Lr: 0.000300\n",
      "2019-10-30 13:01:54,133 Epoch   4 Step:    34800 Batch Loss:     1.640776 Tokens per Sec:    18049, Lr: 0.000300\n",
      "2019-10-30 13:02:08,157 Epoch   4 Step:    34900 Batch Loss:     1.481087 Tokens per Sec:    18618, Lr: 0.000300\n",
      "2019-10-30 13:02:22,161 Epoch   4 Step:    35000 Batch Loss:     1.566235 Tokens per Sec:    18477, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 13:03:03,473 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:03:03,473 Saving new checkpoint.\n",
      "2019-10-30 13:03:03,599 Example #0\n",
      "2019-10-30 13:03:03,599 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:03:03,599 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:03:03,599 \tHypothesis: Alikuwa mwangalifu katika umri wa miaka 17 .\n",
      "2019-10-30 13:03:03,599 Example #1\n",
      "2019-10-30 13:03:03,599 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:03:03,599 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:03:03,599 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:03:03,599 Example #2\n",
      "2019-10-30 13:03:03,599 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:03:03,599 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:03:03,599 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 13:03:03,599 Example #3\n",
      "2019-10-30 13:03:03,599 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:03:03,599 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:03:03,599 \tHypothesis: Wapendao nyuki na washindi .\n",
      "2019-10-30 13:03:03,600 Validation result at epoch   4, step    35000: bleu:  30.12, loss: 39420.6094, ppl:   4.1968, duration: 41.4384s\n",
      "2019-10-30 13:03:17,656 Epoch   4 Step:    35100 Batch Loss:     1.709002 Tokens per Sec:    18396, Lr: 0.000300\n",
      "2019-10-30 13:03:31,723 Epoch   4 Step:    35200 Batch Loss:     1.626453 Tokens per Sec:    18378, Lr: 0.000300\n",
      "2019-10-30 13:03:45,852 Epoch   4 Step:    35300 Batch Loss:     1.446569 Tokens per Sec:    18395, Lr: 0.000300\n",
      "2019-10-30 13:03:59,669 Epoch   4 Step:    35400 Batch Loss:     1.475019 Tokens per Sec:    18373, Lr: 0.000300\n",
      "2019-10-30 13:04:13,704 Epoch   4 Step:    35500 Batch Loss:     1.562230 Tokens per Sec:    18565, Lr: 0.000300\n",
      "2019-10-30 13:04:27,662 Epoch   4 Step:    35600 Batch Loss:     1.602848 Tokens per Sec:    18294, Lr: 0.000300\n",
      "2019-10-30 13:04:41,570 Epoch   4 Step:    35700 Batch Loss:     1.645252 Tokens per Sec:    18251, Lr: 0.000300\n",
      "2019-10-30 13:04:55,600 Epoch   4 Step:    35800 Batch Loss:     1.541327 Tokens per Sec:    18299, Lr: 0.000300\n",
      "2019-10-30 13:05:10,878 Epoch   4 Step:    35900 Batch Loss:     2.274315 Tokens per Sec:    16636, Lr: 0.000300\n",
      "2019-10-30 13:05:25,698 Epoch   4 Step:    36000 Batch Loss:     1.574535 Tokens per Sec:    17272, Lr: 0.000300\n",
      "2019-10-30 13:06:07,541 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:06:07,541 Saving new checkpoint.\n",
      "2019-10-30 13:06:07,673 Example #0\n",
      "2019-10-30 13:06:07,673 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:06:07,673 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:06:07,673 \tHypothesis: Alikuwa mzee mwenye umri wa miaka 17 .\n",
      "2019-10-30 13:06:07,673 Example #1\n",
      "2019-10-30 13:06:07,673 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:06:07,673 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:06:07,673 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:06:07,673 Example #2\n",
      "2019-10-30 13:06:07,673 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:06:07,673 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:06:07,673 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 13:06:07,673 Example #3\n",
      "2019-10-30 13:06:07,674 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:06:07,674 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:06:07,674 \tHypothesis: Wapendao nyuki na mabawa .\n",
      "2019-10-30 13:06:07,674 Validation result at epoch   4, step    36000: bleu:  30.78, loss: 39353.9844, ppl:   4.1866, duration: 41.9752s\n",
      "2019-10-30 13:06:21,598 Epoch   4 Step:    36100 Batch Loss:     1.715094 Tokens per Sec:    18379, Lr: 0.000300\n",
      "2019-10-30 13:06:35,520 Epoch   4 Step:    36200 Batch Loss:     1.467574 Tokens per Sec:    18133, Lr: 0.000300\n",
      "2019-10-30 13:06:49,853 Epoch   4 Step:    36300 Batch Loss:     1.687448 Tokens per Sec:    17827, Lr: 0.000300\n",
      "2019-10-30 13:07:04,868 Epoch   4 Step:    36400 Batch Loss:     1.745754 Tokens per Sec:    17248, Lr: 0.000300\n",
      "2019-10-30 13:07:19,723 Epoch   4 Step:    36500 Batch Loss:     1.845273 Tokens per Sec:    17126, Lr: 0.000300\n",
      "2019-10-30 13:07:34,046 Epoch   4 Step:    36600 Batch Loss:     1.870411 Tokens per Sec:    17971, Lr: 0.000300\n",
      "2019-10-30 13:07:48,256 Epoch   4 Step:    36700 Batch Loss:     1.819311 Tokens per Sec:    18241, Lr: 0.000300\n",
      "2019-10-30 13:08:02,231 Epoch   4 Step:    36800 Batch Loss:     1.606280 Tokens per Sec:    18344, Lr: 0.000300\n",
      "2019-10-30 13:08:16,528 Epoch   4 Step:    36900 Batch Loss:     1.712472 Tokens per Sec:    18251, Lr: 0.000300\n",
      "2019-10-30 13:08:30,569 Epoch   4 Step:    37000 Batch Loss:     1.646102 Tokens per Sec:    18304, Lr: 0.000300\n",
      "2019-10-30 13:09:24,685 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:09:24,686 Saving new checkpoint.\n",
      "2019-10-30 13:09:25,166 Example #0\n",
      "2019-10-30 13:09:25,167 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:09:25,167 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:09:25,168 \tHypothesis: Alikuwa mzee wa miaka 17 .\n",
      "2019-10-30 13:09:25,168 Example #1\n",
      "2019-10-30 13:09:25,168 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:09:25,168 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:09:25,169 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:09:25,169 Example #2\n",
      "2019-10-30 13:09:25,169 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:09:25,169 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:09:25,170 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni mahangaiko gani yanayoweza kuathiri ?\n",
      "2019-10-30 13:09:25,170 Example #3\n",
      "2019-10-30 13:09:25,170 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:09:25,170 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:09:25,171 \tHypothesis: Mapenzi ya maua na maua .\n",
      "2019-10-30 13:09:25,171 Validation result at epoch   4, step    37000: bleu:  30.61, loss: 39078.6055, ppl:   4.1449, duration: 54.6010s\n",
      "2019-10-30 13:09:56,212 Epoch   4 Step:    37100 Batch Loss:     1.839740 Tokens per Sec:     8345, Lr: 0.000300\n",
      "2019-10-30 13:10:20,615 Epoch   4 Step:    37200 Batch Loss:     1.666659 Tokens per Sec:    10236, Lr: 0.000300\n",
      "2019-10-30 13:10:36,212 Epoch   4 Step:    37300 Batch Loss:     1.446012 Tokens per Sec:    17048, Lr: 0.000300\n",
      "2019-10-30 13:10:51,125 Epoch   4 Step:    37400 Batch Loss:     1.686894 Tokens per Sec:    17246, Lr: 0.000300\n",
      "2019-10-30 13:11:05,757 Epoch   4 Step:    37500 Batch Loss:     1.353941 Tokens per Sec:    17343, Lr: 0.000300\n",
      "2019-10-30 13:11:12,416 Epoch   4: total training loss 15522.09\n",
      "2019-10-30 13:11:12,416 EPOCH 5\n",
      "2019-10-30 13:11:21,385 Epoch   5 Step:    37600 Batch Loss:     1.450309 Tokens per Sec:    15553, Lr: 0.000300\n",
      "2019-10-30 13:11:36,367 Epoch   5 Step:    37700 Batch Loss:     1.413741 Tokens per Sec:    16989, Lr: 0.000300\n",
      "2019-10-30 13:11:52,048 Epoch   5 Step:    37800 Batch Loss:     1.629038 Tokens per Sec:    16410, Lr: 0.000300\n",
      "2019-10-30 13:12:06,942 Epoch   5 Step:    37900 Batch Loss:     1.880242 Tokens per Sec:    17216, Lr: 0.000300\n",
      "2019-10-30 13:12:21,933 Epoch   5 Step:    38000 Batch Loss:     1.691585 Tokens per Sec:    17277, Lr: 0.000300\n",
      "2019-10-30 13:13:03,380 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:13:03,381 Saving new checkpoint.\n",
      "2019-10-30 13:13:03,561 Example #0\n",
      "2019-10-30 13:13:03,561 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:13:03,561 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:13:03,561 \tHypothesis: Alikuwa mwangalifu akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:13:03,561 Example #1\n",
      "2019-10-30 13:13:03,561 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:13:03,562 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:13:03,562 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:13:03,562 Example #2\n",
      "2019-10-30 13:13:03,562 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:13:03,562 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:13:03,562 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yaweza kufanyaje ?\n",
      "2019-10-30 13:13:03,562 Example #3\n",
      "2019-10-30 13:13:03,562 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:13:03,562 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:13:03,562 \tHypothesis: Mapenzi ya nyuki na mabawa .\n",
      "2019-10-30 13:13:03,562 Validation result at epoch   5, step    38000: bleu:  31.09, loss: 38805.4102, ppl:   4.1039, duration: 41.6282s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 13:13:18,251 Epoch   5 Step:    38100 Batch Loss:     1.543994 Tokens per Sec:    18123, Lr: 0.000300\n",
      "2019-10-30 13:13:32,366 Epoch   5 Step:    38200 Batch Loss:     1.825300 Tokens per Sec:    17930, Lr: 0.000300\n",
      "2019-10-30 13:13:46,493 Epoch   5 Step:    38300 Batch Loss:     1.697662 Tokens per Sec:    18211, Lr: 0.000300\n",
      "2019-10-30 13:14:00,602 Epoch   5 Step:    38400 Batch Loss:     1.486660 Tokens per Sec:    18352, Lr: 0.000300\n",
      "2019-10-30 13:14:14,808 Epoch   5 Step:    38500 Batch Loss:     1.441533 Tokens per Sec:    18500, Lr: 0.000300\n",
      "2019-10-30 13:14:29,008 Epoch   5 Step:    38600 Batch Loss:     1.384709 Tokens per Sec:    18392, Lr: 0.000300\n",
      "2019-10-30 13:14:43,377 Epoch   5 Step:    38700 Batch Loss:     1.590280 Tokens per Sec:    17974, Lr: 0.000300\n",
      "2019-10-30 13:14:58,220 Epoch   5 Step:    38800 Batch Loss:     1.911928 Tokens per Sec:    17822, Lr: 0.000300\n",
      "2019-10-30 13:15:12,522 Epoch   5 Step:    38900 Batch Loss:     1.512883 Tokens per Sec:    17654, Lr: 0.000300\n",
      "2019-10-30 13:15:26,997 Epoch   5 Step:    39000 Batch Loss:     1.527575 Tokens per Sec:    17664, Lr: 0.000300\n",
      "2019-10-30 13:16:12,887 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:16:12,887 Saving new checkpoint.\n",
      "2019-10-30 13:16:13,036 Example #0\n",
      "2019-10-30 13:16:13,036 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:16:13,036 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:16:13,036 \tHypothesis: Alikuwa mwangalifu alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 13:16:13,036 Example #1\n",
      "2019-10-30 13:16:13,036 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:16:13,036 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:16:13,036 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:16:13,036 Example #2\n",
      "2019-10-30 13:16:13,037 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:16:13,037 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:16:13,037 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiriwa na nini ?\n",
      "2019-10-30 13:16:13,037 Example #3\n",
      "2019-10-30 13:16:13,037 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:16:13,037 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:16:13,037 \tHypothesis: Wapendao nyuki na wadudu .\n",
      "2019-10-30 13:16:13,037 Validation result at epoch   5, step    39000: bleu:  31.06, loss: 38516.1445, ppl:   4.0609, duration: 46.0392s\n",
      "2019-10-30 13:16:28,649 Epoch   5 Step:    39100 Batch Loss:     1.521006 Tokens per Sec:    16845, Lr: 0.000300\n",
      "2019-10-30 13:16:43,332 Epoch   5 Step:    39200 Batch Loss:     1.562559 Tokens per Sec:    17552, Lr: 0.000300\n",
      "2019-10-30 13:16:57,850 Epoch   5 Step:    39300 Batch Loss:     1.563031 Tokens per Sec:    17491, Lr: 0.000300\n",
      "2019-10-30 13:17:12,527 Epoch   5 Step:    39400 Batch Loss:     1.520467 Tokens per Sec:    17398, Lr: 0.000300\n",
      "2019-10-30 13:17:27,514 Epoch   5 Step:    39500 Batch Loss:     1.694707 Tokens per Sec:    17035, Lr: 0.000300\n",
      "2019-10-30 13:17:41,935 Epoch   5 Step:    39600 Batch Loss:     1.436661 Tokens per Sec:    17572, Lr: 0.000300\n",
      "2019-10-30 13:17:56,529 Epoch   5 Step:    39700 Batch Loss:     1.346115 Tokens per Sec:    17800, Lr: 0.000300\n",
      "2019-10-30 13:18:11,321 Epoch   5 Step:    39800 Batch Loss:     1.615543 Tokens per Sec:    17730, Lr: 0.000300\n",
      "2019-10-30 13:18:25,855 Epoch   5 Step:    39900 Batch Loss:     1.627761 Tokens per Sec:    17425, Lr: 0.000300\n",
      "2019-10-30 13:18:41,219 Epoch   5 Step:    40000 Batch Loss:     1.693105 Tokens per Sec:    16967, Lr: 0.000300\n",
      "2019-10-30 13:19:24,555 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:19:24,555 Saving new checkpoint.\n",
      "2019-10-30 13:19:24,692 Example #0\n",
      "2019-10-30 13:19:24,692 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:19:24,692 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:19:24,692 \tHypothesis: Alikuwa mwangalifu akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:19:24,692 Example #1\n",
      "2019-10-30 13:19:24,692 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:19:24,692 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:19:24,692 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:19:24,692 Example #2\n",
      "2019-10-30 13:19:24,692 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:19:24,692 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:19:24,692 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiriwa na nini ?\n",
      "2019-10-30 13:19:24,692 Example #3\n",
      "2019-10-30 13:19:24,693 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:19:24,693 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:19:24,693 \tHypothesis: Wapendao nyuki na wadudu .\n",
      "2019-10-30 13:19:24,693 Validation result at epoch   5, step    40000: bleu:  31.37, loss: 38459.9141, ppl:   4.0526, duration: 43.4729s\n",
      "2019-10-30 13:19:39,082 Epoch   5 Step:    40100 Batch Loss:     1.501790 Tokens per Sec:    17325, Lr: 0.000300\n",
      "2019-10-30 13:19:53,705 Epoch   5 Step:    40200 Batch Loss:     1.577877 Tokens per Sec:    17649, Lr: 0.000300\n",
      "2019-10-30 13:20:08,276 Epoch   5 Step:    40300 Batch Loss:     1.717201 Tokens per Sec:    17617, Lr: 0.000300\n",
      "2019-10-30 13:20:22,704 Epoch   5 Step:    40400 Batch Loss:     1.465286 Tokens per Sec:    17169, Lr: 0.000300\n",
      "2019-10-30 13:20:37,367 Epoch   5 Step:    40500 Batch Loss:     1.525022 Tokens per Sec:    17977, Lr: 0.000300\n",
      "2019-10-30 13:20:51,844 Epoch   5 Step:    40600 Batch Loss:     1.375390 Tokens per Sec:    17430, Lr: 0.000300\n",
      "2019-10-30 13:21:06,128 Epoch   5 Step:    40700 Batch Loss:     1.533342 Tokens per Sec:    17710, Lr: 0.000300\n",
      "2019-10-30 13:21:20,732 Epoch   5 Step:    40800 Batch Loss:     1.544061 Tokens per Sec:    17814, Lr: 0.000300\n",
      "2019-10-30 13:21:35,448 Epoch   5 Step:    40900 Batch Loss:     1.594629 Tokens per Sec:    17980, Lr: 0.000300\n",
      "2019-10-30 13:21:50,211 Epoch   5 Step:    41000 Batch Loss:     1.589144 Tokens per Sec:    17267, Lr: 0.000300\n",
      "2019-10-30 13:22:32,747 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:22:32,747 Saving new checkpoint.\n",
      "2019-10-30 13:22:32,881 Example #0\n",
      "2019-10-30 13:22:32,881 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:22:32,881 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:22:32,881 \tHypothesis: Alikuwa mwangalifu akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:22:32,881 Example #1\n",
      "2019-10-30 13:22:32,881 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:22:32,881 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:22:32,881 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:22:32,881 Example #2\n",
      "2019-10-30 13:22:32,881 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:22:32,881 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:22:32,882 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 13:22:32,882 Example #3\n",
      "2019-10-30 13:22:32,882 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:22:32,882 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:22:32,882 \tHypothesis: Wapenda maua na maua .\n",
      "2019-10-30 13:22:32,882 Validation result at epoch   5, step    41000: bleu:  31.17, loss: 38292.4570, ppl:   4.0280, duration: 42.6701s\n",
      "2019-10-30 13:22:48,289 Epoch   5 Step:    41100 Batch Loss:     1.586220 Tokens per Sec:    16685, Lr: 0.000300\n",
      "2019-10-30 13:23:02,888 Epoch   5 Step:    41200 Batch Loss:     1.410004 Tokens per Sec:    17640, Lr: 0.000300\n",
      "2019-10-30 13:23:17,467 Epoch   5 Step:    41300 Batch Loss:     1.720812 Tokens per Sec:    17692, Lr: 0.000300\n",
      "2019-10-30 13:23:31,952 Epoch   5 Step:    41400 Batch Loss:     1.556862 Tokens per Sec:    17621, Lr: 0.000300\n",
      "2019-10-30 13:23:46,607 Epoch   5 Step:    41500 Batch Loss:     1.538660 Tokens per Sec:    17465, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 13:24:01,102 Epoch   5 Step:    41600 Batch Loss:     1.496756 Tokens per Sec:    17894, Lr: 0.000300\n",
      "2019-10-30 13:24:15,330 Epoch   5 Step:    41700 Batch Loss:     1.692984 Tokens per Sec:    17836, Lr: 0.000300\n",
      "2019-10-30 13:24:29,771 Epoch   5 Step:    41800 Batch Loss:     1.471447 Tokens per Sec:    17623, Lr: 0.000300\n",
      "2019-10-30 13:24:44,264 Epoch   5 Step:    41900 Batch Loss:     1.614058 Tokens per Sec:    17787, Lr: 0.000300\n",
      "2019-10-30 13:24:58,692 Epoch   5 Step:    42000 Batch Loss:     1.643598 Tokens per Sec:    17573, Lr: 0.000300\n",
      "2019-10-30 13:25:42,657 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:25:42,657 Saving new checkpoint.\n",
      "2019-10-30 13:25:42,804 Example #0\n",
      "2019-10-30 13:25:42,805 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:25:42,805 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:25:42,805 \tHypothesis: Alikuwa mwangalifu alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 13:25:42,805 Example #1\n",
      "2019-10-30 13:25:42,805 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:25:42,805 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:25:42,805 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:25:42,805 Example #2\n",
      "2019-10-30 13:25:42,805 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:25:42,805 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:25:42,805 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni mahangaiko gani yanayoweza kuathiri ?\n",
      "2019-10-30 13:25:42,805 Example #3\n",
      "2019-10-30 13:25:42,805 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:25:42,805 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:25:42,805 \tHypothesis: Wapendao maua na maua .\n",
      "2019-10-30 13:25:42,806 Validation result at epoch   5, step    42000: bleu:  31.57, loss: 37927.4102, ppl:   3.9748, duration: 44.1127s\n",
      "2019-10-30 13:25:58,153 Epoch   5 Step:    42100 Batch Loss:     1.457392 Tokens per Sec:    17072, Lr: 0.000300\n",
      "2019-10-30 13:26:13,123 Epoch   5 Step:    42200 Batch Loss:     1.656636 Tokens per Sec:    17111, Lr: 0.000300\n",
      "2019-10-30 13:26:27,882 Epoch   5 Step:    42300 Batch Loss:     1.740556 Tokens per Sec:    17602, Lr: 0.000300\n",
      "2019-10-30 13:26:42,693 Epoch   5 Step:    42400 Batch Loss:     1.616049 Tokens per Sec:    17696, Lr: 0.000300\n",
      "2019-10-30 13:26:57,285 Epoch   5 Step:    42500 Batch Loss:     1.465241 Tokens per Sec:    17537, Lr: 0.000300\n",
      "2019-10-30 13:27:11,857 Epoch   5 Step:    42600 Batch Loss:     1.404777 Tokens per Sec:    17731, Lr: 0.000300\n",
      "2019-10-30 13:27:26,290 Epoch   5 Step:    42700 Batch Loss:     1.460516 Tokens per Sec:    17379, Lr: 0.000300\n",
      "2019-10-30 13:27:40,828 Epoch   5 Step:    42800 Batch Loss:     1.516344 Tokens per Sec:    17691, Lr: 0.000300\n",
      "2019-10-30 13:27:55,530 Epoch   5 Step:    42900 Batch Loss:     1.667441 Tokens per Sec:    17776, Lr: 0.000300\n",
      "2019-10-30 13:28:10,156 Epoch   5 Step:    43000 Batch Loss:     1.552556 Tokens per Sec:    17309, Lr: 0.000300\n",
      "2019-10-30 13:28:53,879 Example #0\n",
      "2019-10-30 13:28:53,879 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:28:53,879 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:28:53,879 \tHypothesis: Alikuwa mwangalifu akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:28:53,879 Example #1\n",
      "2019-10-30 13:28:53,879 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:28:53,879 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:28:53,879 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:28:53,879 Example #2\n",
      "2019-10-30 13:28:53,879 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:28:53,879 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:28:53,879 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuchangamsha ?\n",
      "2019-10-30 13:28:53,880 Example #3\n",
      "2019-10-30 13:28:53,880 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:28:53,880 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:28:53,880 \tHypothesis: Mapenzi ya maua na maua .\n",
      "2019-10-30 13:28:53,880 Validation result at epoch   5, step    43000: bleu:  32.00, loss: 38000.1680, ppl:   3.9854, duration: 43.7234s\n",
      "2019-10-30 13:29:08,690 Epoch   5 Step:    43100 Batch Loss:     1.540684 Tokens per Sec:    17502, Lr: 0.000300\n",
      "2019-10-30 13:29:23,386 Epoch   5 Step:    43200 Batch Loss:     1.576023 Tokens per Sec:    17681, Lr: 0.000300\n",
      "2019-10-30 13:29:37,944 Epoch   5 Step:    43300 Batch Loss:     1.628590 Tokens per Sec:    17424, Lr: 0.000300\n",
      "2019-10-30 13:29:52,670 Epoch   5 Step:    43400 Batch Loss:     1.344337 Tokens per Sec:    17665, Lr: 0.000300\n",
      "2019-10-30 13:30:07,389 Epoch   5 Step:    43500 Batch Loss:     1.339729 Tokens per Sec:    17375, Lr: 0.000300\n",
      "2019-10-30 13:30:22,028 Epoch   5 Step:    43600 Batch Loss:     1.440332 Tokens per Sec:    17341, Lr: 0.000300\n",
      "2019-10-30 13:30:36,805 Epoch   5 Step:    43700 Batch Loss:     1.544000 Tokens per Sec:    17620, Lr: 0.000300\n",
      "2019-10-30 13:30:51,328 Epoch   5 Step:    43800 Batch Loss:     1.380545 Tokens per Sec:    17424, Lr: 0.000300\n",
      "2019-10-30 13:31:05,944 Epoch   5 Step:    43900 Batch Loss:     1.664442 Tokens per Sec:    17723, Lr: 0.000300\n",
      "2019-10-30 13:31:20,559 Epoch   5 Step:    44000 Batch Loss:     1.536628 Tokens per Sec:    17384, Lr: 0.000300\n",
      "2019-10-30 13:32:03,808 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:32:03,809 Saving new checkpoint.\n",
      "2019-10-30 13:32:03,940 Example #0\n",
      "2019-10-30 13:32:03,940 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:32:03,940 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:32:03,940 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 13:32:03,941 Example #1\n",
      "2019-10-30 13:32:03,941 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:32:03,941 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:32:03,941 \tHypothesis: Biblia haisemi uzuri .\n",
      "2019-10-30 13:32:03,941 Example #2\n",
      "2019-10-30 13:32:03,941 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:32:03,941 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:32:03,941 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 13:32:03,941 Example #3\n",
      "2019-10-30 13:32:03,941 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:32:03,941 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:32:03,941 \tHypothesis: Wapendao na wabudu .\n",
      "2019-10-30 13:32:03,941 Validation result at epoch   5, step    44000: bleu:  31.56, loss: 37800.3320, ppl:   3.9565, duration: 43.3822s\n",
      "2019-10-30 13:32:18,538 Epoch   5 Step:    44100 Batch Loss:     1.988828 Tokens per Sec:    17529, Lr: 0.000300\n",
      "2019-10-30 13:32:33,165 Epoch   5 Step:    44200 Batch Loss:     1.570448 Tokens per Sec:    17510, Lr: 0.000300\n",
      "2019-10-30 13:32:47,800 Epoch   5 Step:    44300 Batch Loss:     1.493045 Tokens per Sec:    17441, Lr: 0.000300\n",
      "2019-10-30 13:33:02,476 Epoch   5 Step:    44400 Batch Loss:     1.404138 Tokens per Sec:    17711, Lr: 0.000300\n",
      "2019-10-30 13:33:16,781 Epoch   5 Step:    44500 Batch Loss:     1.348065 Tokens per Sec:    17850, Lr: 0.000300\n",
      "2019-10-30 13:33:31,119 Epoch   5 Step:    44600 Batch Loss:     1.398480 Tokens per Sec:    18184, Lr: 0.000300\n",
      "2019-10-30 13:33:45,527 Epoch   5 Step:    44700 Batch Loss:     1.669257 Tokens per Sec:    18095, Lr: 0.000300\n",
      "2019-10-30 13:33:59,900 Epoch   5 Step:    44800 Batch Loss:     1.356399 Tokens per Sec:    17951, Lr: 0.000300\n",
      "2019-10-30 13:34:14,276 Epoch   5 Step:    44900 Batch Loss:     1.709660 Tokens per Sec:    17946, Lr: 0.000300\n",
      "2019-10-30 13:34:28,667 Epoch   5 Step:    45000 Batch Loss:     1.617100 Tokens per Sec:    17696, Lr: 0.000300\n",
      "2019-10-30 13:35:48,213 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:35:48,213 Saving new checkpoint.\n",
      "2019-10-30 13:35:48,388 Example #0\n",
      "2019-10-30 13:35:48,389 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:35:48,389 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:35:48,389 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:35:48,389 Example #1\n",
      "2019-10-30 13:35:48,389 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:35:48,389 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:35:48,389 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 13:35:48,389 Example #2\n",
      "2019-10-30 13:35:48,389 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:35:48,389 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:35:48,389 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na ni mahangaiko gani yanayoweza kutokea ?\n",
      "2019-10-30 13:35:48,389 Example #3\n",
      "2019-10-30 13:35:48,390 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:35:48,390 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:35:48,390 \tHypothesis: Madhabahu na mabawa .\n",
      "2019-10-30 13:35:48,390 Validation result at epoch   5, step    45000: bleu:  32.24, loss: 37423.0859, ppl:   3.9026, duration: 79.7223s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 13:36:03,875 Epoch   5 Step:    45100 Batch Loss:     1.538431 Tokens per Sec:    16018, Lr: 0.000300\n",
      "2019-10-30 13:36:19,137 Epoch   5 Step:    45200 Batch Loss:     1.690335 Tokens per Sec:    17218, Lr: 0.000300\n",
      "2019-10-30 13:36:34,014 Epoch   5 Step:    45300 Batch Loss:     1.450749 Tokens per Sec:    17158, Lr: 0.000300\n",
      "2019-10-30 13:36:48,729 Epoch   5 Step:    45400 Batch Loss:     1.865606 Tokens per Sec:    17594, Lr: 0.000300\n",
      "2019-10-30 13:37:03,037 Epoch   5 Step:    45500 Batch Loss:     1.517508 Tokens per Sec:    17459, Lr: 0.000300\n",
      "2019-10-30 13:37:17,280 Epoch   5 Step:    45600 Batch Loss:     1.765623 Tokens per Sec:    17597, Lr: 0.000300\n",
      "2019-10-30 13:37:31,441 Epoch   5 Step:    45700 Batch Loss:     1.239390 Tokens per Sec:    18061, Lr: 0.000300\n",
      "2019-10-30 13:37:45,612 Epoch   5 Step:    45800 Batch Loss:     1.421365 Tokens per Sec:    17947, Lr: 0.000300\n",
      "2019-10-30 13:37:59,785 Epoch   5 Step:    45900 Batch Loss:     1.623074 Tokens per Sec:    17952, Lr: 0.000300\n",
      "2019-10-30 13:38:14,462 Epoch   5 Step:    46000 Batch Loss:     1.544830 Tokens per Sec:    17800, Lr: 0.000300\n",
      "2019-10-30 13:38:56,606 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:38:56,606 Saving new checkpoint.\n",
      "2019-10-30 13:38:56,735 Example #0\n",
      "2019-10-30 13:38:56,736 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:38:56,736 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:38:56,736 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 13:38:56,736 Example #1\n",
      "2019-10-30 13:38:56,736 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:38:56,736 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:38:56,736 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 13:38:56,736 Example #2\n",
      "2019-10-30 13:38:56,736 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:38:56,736 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:38:56,736 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuchanganua ?\n",
      "2019-10-30 13:38:56,736 Example #3\n",
      "2019-10-30 13:38:56,736 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:38:56,736 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:38:56,736 \tHypothesis: Wapenzi wa nyuki na washindi .\n",
      "2019-10-30 13:38:56,736 Validation result at epoch   5, step    46000: bleu:  32.14, loss: 37306.1016, ppl:   3.8860, duration: 42.2739s\n",
      "2019-10-30 13:39:11,023 Epoch   5 Step:    46100 Batch Loss:     1.969111 Tokens per Sec:    17939, Lr: 0.000300\n",
      "2019-10-30 13:39:25,361 Epoch   5 Step:    46200 Batch Loss:     1.499679 Tokens per Sec:    17938, Lr: 0.000300\n",
      "2019-10-30 13:39:39,693 Epoch   5 Step:    46300 Batch Loss:     1.669068 Tokens per Sec:    17930, Lr: 0.000300\n",
      "2019-10-30 13:39:53,818 Epoch   5 Step:    46400 Batch Loss:     1.468818 Tokens per Sec:    17837, Lr: 0.000300\n",
      "2019-10-30 13:40:08,266 Epoch   5 Step:    46500 Batch Loss:     1.405232 Tokens per Sec:    18129, Lr: 0.000300\n",
      "2019-10-30 13:40:22,605 Epoch   5 Step:    46600 Batch Loss:     1.442549 Tokens per Sec:    17969, Lr: 0.000300\n",
      "2019-10-30 13:40:37,056 Epoch   5 Step:    46700 Batch Loss:     1.499178 Tokens per Sec:    17742, Lr: 0.000300\n",
      "2019-10-30 13:40:51,483 Epoch   5 Step:    46800 Batch Loss:     1.378848 Tokens per Sec:    17729, Lr: 0.000300\n",
      "2019-10-30 13:41:05,860 Epoch   5 Step:    46900 Batch Loss:     1.453453 Tokens per Sec:    17847, Lr: 0.000300\n",
      "2019-10-30 13:41:10,336 Epoch   5: total training loss 14713.34\n",
      "2019-10-30 13:41:10,336 EPOCH 6\n",
      "2019-10-30 13:41:21,285 Epoch   6 Step:    47000 Batch Loss:     1.641383 Tokens per Sec:    16164, Lr: 0.000300\n",
      "2019-10-30 13:42:03,714 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:42:03,714 Saving new checkpoint.\n",
      "2019-10-30 13:42:03,843 Example #0\n",
      "2019-10-30 13:42:03,844 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:42:03,844 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:42:03,844 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:42:03,844 Example #1\n",
      "2019-10-30 13:42:03,844 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:42:03,844 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:42:03,844 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:42:03,844 Example #2\n",
      "2019-10-30 13:42:03,844 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:42:03,844 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:42:03,844 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 13:42:03,844 Example #3\n",
      "2019-10-30 13:42:03,844 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:42:03,844 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:42:03,844 \tHypothesis: Wanyama wa nyuki na washindi .\n",
      "2019-10-30 13:42:03,844 Validation result at epoch   6, step    47000: bleu:  31.42, loss: 37149.2305, ppl:   3.8639, duration: 42.5592s\n",
      "2019-10-30 13:42:18,132 Epoch   6 Step:    47100 Batch Loss:     1.529978 Tokens per Sec:    17869, Lr: 0.000300\n",
      "2019-10-30 13:42:32,385 Epoch   6 Step:    47200 Batch Loss:     1.449582 Tokens per Sec:    17985, Lr: 0.000300\n",
      "2019-10-30 13:42:46,661 Epoch   6 Step:    47300 Batch Loss:     1.617093 Tokens per Sec:    17851, Lr: 0.000300\n",
      "2019-10-30 13:43:01,567 Epoch   6 Step:    47400 Batch Loss:     1.831697 Tokens per Sec:    17021, Lr: 0.000300\n",
      "2019-10-30 13:43:17,088 Epoch   6 Step:    47500 Batch Loss:     1.629752 Tokens per Sec:    16689, Lr: 0.000300\n",
      "2019-10-30 13:43:31,896 Epoch   6 Step:    47600 Batch Loss:     1.364921 Tokens per Sec:    17055, Lr: 0.000300\n",
      "2019-10-30 13:43:46,988 Epoch   6 Step:    47700 Batch Loss:     1.475556 Tokens per Sec:    17224, Lr: 0.000300\n",
      "2019-10-30 13:44:02,072 Epoch   6 Step:    47800 Batch Loss:     2.216881 Tokens per Sec:    16792, Lr: 0.000300\n",
      "2019-10-30 13:44:18,057 Epoch   6 Step:    47900 Batch Loss:     1.361166 Tokens per Sec:    16488, Lr: 0.000300\n",
      "2019-10-30 13:44:33,164 Epoch   6 Step:    48000 Batch Loss:     1.652260 Tokens per Sec:    17283, Lr: 0.000300\n",
      "2019-10-30 13:45:17,305 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:45:17,305 Saving new checkpoint.\n",
      "2019-10-30 13:45:17,442 Example #0\n",
      "2019-10-30 13:45:17,442 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:45:17,442 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:45:17,442 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:45:17,442 Example #1\n",
      "2019-10-30 13:45:17,443 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:45:17,443 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:45:17,443 \tHypothesis: Biblia haisemi uzinzi .\n",
      "2019-10-30 13:45:17,443 Example #2\n",
      "2019-10-30 13:45:17,443 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:45:17,443 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:45:17,443 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kuwa nini ?\n",
      "2019-10-30 13:45:17,443 Example #3\n",
      "2019-10-30 13:45:17,443 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:45:17,443 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:45:17,443 \tHypothesis: Mapenzi ya nyuki na mabawa .\n",
      "2019-10-30 13:45:17,443 Validation result at epoch   6, step    48000: bleu:  32.35, loss: 36985.1992, ppl:   3.8409, duration: 44.2786s\n",
      "2019-10-30 13:45:32,556 Epoch   6 Step:    48100 Batch Loss:     1.538430 Tokens per Sec:    17363, Lr: 0.000300\n",
      "2019-10-30 13:45:47,476 Epoch   6 Step:    48200 Batch Loss:     1.451912 Tokens per Sec:    16862, Lr: 0.000300\n",
      "2019-10-30 13:46:02,494 Epoch   6 Step:    48300 Batch Loss:     1.623017 Tokens per Sec:    17014, Lr: 0.000300\n",
      "2019-10-30 13:46:17,598 Epoch   6 Step:    48400 Batch Loss:     1.356428 Tokens per Sec:    16866, Lr: 0.000300\n",
      "2019-10-30 13:46:32,460 Epoch   6 Step:    48500 Batch Loss:     1.409509 Tokens per Sec:    16936, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 13:46:47,511 Epoch   6 Step:    48600 Batch Loss:     1.358018 Tokens per Sec:    17254, Lr: 0.000300\n",
      "2019-10-30 13:47:02,526 Epoch   6 Step:    48700 Batch Loss:     1.391653 Tokens per Sec:    17053, Lr: 0.000300\n",
      "2019-10-30 13:47:17,797 Epoch   6 Step:    48800 Batch Loss:     1.493745 Tokens per Sec:    17117, Lr: 0.000300\n",
      "2019-10-30 13:47:32,915 Epoch   6 Step:    48900 Batch Loss:     1.387908 Tokens per Sec:    17162, Lr: 0.000300\n",
      "2019-10-30 13:47:47,832 Epoch   6 Step:    49000 Batch Loss:     1.408634 Tokens per Sec:    17150, Lr: 0.000300\n",
      "2019-10-30 13:48:32,448 Example #0\n",
      "2019-10-30 13:48:32,449 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:48:32,449 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:48:32,449 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:48:32,449 Example #1\n",
      "2019-10-30 13:48:32,449 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:48:32,449 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:48:32,449 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:48:32,449 Example #2\n",
      "2019-10-30 13:48:32,449 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:48:32,449 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:48:32,449 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuchangamsha ?\n",
      "2019-10-30 13:48:32,449 Example #3\n",
      "2019-10-30 13:48:32,449 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:48:32,449 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:48:32,449 \tHypothesis: Mapenzi ya nyuki na kuyapiga .\n",
      "2019-10-30 13:48:32,449 Validation result at epoch   6, step    49000: bleu:  32.25, loss: 37501.1719, ppl:   3.9137, duration: 44.6171s\n",
      "2019-10-30 13:48:47,579 Epoch   6 Step:    49100 Batch Loss:     1.615234 Tokens per Sec:    17199, Lr: 0.000300\n",
      "2019-10-30 13:49:02,496 Epoch   6 Step:    49200 Batch Loss:     1.653108 Tokens per Sec:    16968, Lr: 0.000300\n",
      "2019-10-30 13:49:17,510 Epoch   6 Step:    49300 Batch Loss:     1.462302 Tokens per Sec:    17194, Lr: 0.000300\n",
      "2019-10-30 13:49:32,705 Epoch   6 Step:    49400 Batch Loss:     1.493829 Tokens per Sec:    16933, Lr: 0.000300\n",
      "2019-10-30 13:49:47,745 Epoch   6 Step:    49500 Batch Loss:     1.397742 Tokens per Sec:    17170, Lr: 0.000300\n",
      "2019-10-30 13:50:02,684 Epoch   6 Step:    49600 Batch Loss:     2.916059 Tokens per Sec:    16873, Lr: 0.000300\n",
      "2019-10-30 13:50:17,569 Epoch   6 Step:    49700 Batch Loss:     1.539415 Tokens per Sec:    16925, Lr: 0.000300\n",
      "2019-10-30 13:50:32,437 Epoch   6 Step:    49800 Batch Loss:     1.521696 Tokens per Sec:    16816, Lr: 0.000300\n",
      "2019-10-30 13:50:47,760 Epoch   6 Step:    49900 Batch Loss:     1.716788 Tokens per Sec:    16934, Lr: 0.000300\n",
      "2019-10-30 13:51:03,053 Epoch   6 Step:    50000 Batch Loss:     1.600123 Tokens per Sec:    16985, Lr: 0.000300\n",
      "2019-10-30 13:51:47,443 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:51:47,443 Saving new checkpoint.\n",
      "2019-10-30 13:51:47,580 Example #0\n",
      "2019-10-30 13:51:47,580 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:51:47,580 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:51:47,580 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:51:47,580 Example #1\n",
      "2019-10-30 13:51:47,580 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:51:47,580 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:51:47,580 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:51:47,581 Example #2\n",
      "2019-10-30 13:51:47,581 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:51:47,581 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:51:47,581 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri ?\n",
      "2019-10-30 13:51:47,581 Example #3\n",
      "2019-10-30 13:51:47,581 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:51:47,581 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:51:47,581 \tHypothesis: Upendo wa nyuki na wabudu .\n",
      "2019-10-30 13:51:47,581 Validation result at epoch   6, step    50000: bleu:  32.29, loss: 36772.9961, ppl:   3.8113, duration: 44.5274s\n",
      "2019-10-30 13:52:02,727 Epoch   6 Step:    50100 Batch Loss:     1.405452 Tokens per Sec:    17154, Lr: 0.000300\n",
      "2019-10-30 13:52:17,774 Epoch   6 Step:    50200 Batch Loss:     1.452881 Tokens per Sec:    17061, Lr: 0.000300\n",
      "2019-10-30 13:52:32,850 Epoch   6 Step:    50300 Batch Loss:     1.493993 Tokens per Sec:    17253, Lr: 0.000300\n",
      "2019-10-30 13:52:47,993 Epoch   6 Step:    50400 Batch Loss:     1.650306 Tokens per Sec:    17065, Lr: 0.000300\n",
      "2019-10-30 13:53:03,163 Epoch   6 Step:    50500 Batch Loss:     1.468618 Tokens per Sec:    16982, Lr: 0.000300\n",
      "2019-10-30 13:53:18,307 Epoch   6 Step:    50600 Batch Loss:     1.478941 Tokens per Sec:    17326, Lr: 0.000300\n",
      "2019-10-30 13:53:33,348 Epoch   6 Step:    50700 Batch Loss:     1.389094 Tokens per Sec:    17161, Lr: 0.000300\n",
      "2019-10-30 13:53:48,309 Epoch   6 Step:    50800 Batch Loss:     1.544041 Tokens per Sec:    17046, Lr: 0.000300\n",
      "2019-10-30 13:54:03,396 Epoch   6 Step:    50900 Batch Loss:     1.469669 Tokens per Sec:    17146, Lr: 0.000300\n",
      "2019-10-30 13:54:18,643 Epoch   6 Step:    51000 Batch Loss:     1.716125 Tokens per Sec:    17018, Lr: 0.000300\n",
      "2019-10-30 13:55:03,217 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:55:03,217 Saving new checkpoint.\n",
      "2019-10-30 13:55:03,366 Example #0\n",
      "2019-10-30 13:55:03,366 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:55:03,366 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:55:03,367 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:55:03,367 Example #1\n",
      "2019-10-30 13:55:03,367 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:55:03,367 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:55:03,367 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:55:03,367 Example #2\n",
      "2019-10-30 13:55:03,367 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:55:03,367 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:55:03,367 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 13:55:03,367 Example #3\n",
      "2019-10-30 13:55:03,367 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:55:03,367 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:55:03,367 \tHypothesis: Wapenda nyuki na wajumbe .\n",
      "2019-10-30 13:55:03,367 Validation result at epoch   6, step    51000: bleu:  32.57, loss: 36618.3789, ppl:   3.7899, duration: 44.7243s\n",
      "2019-10-30 13:55:18,739 Epoch   6 Step:    51100 Batch Loss:     1.398602 Tokens per Sec:    16872, Lr: 0.000300\n",
      "2019-10-30 13:55:33,878 Epoch   6 Step:    51200 Batch Loss:     1.576684 Tokens per Sec:    16969, Lr: 0.000300\n",
      "2019-10-30 13:55:48,708 Epoch   6 Step:    51300 Batch Loss:     1.480156 Tokens per Sec:    17010, Lr: 0.000300\n",
      "2019-10-30 13:56:03,653 Epoch   6 Step:    51400 Batch Loss:     1.561834 Tokens per Sec:    17049, Lr: 0.000300\n",
      "2019-10-30 13:56:18,696 Epoch   6 Step:    51500 Batch Loss:     1.243723 Tokens per Sec:    17148, Lr: 0.000300\n",
      "2019-10-30 13:56:33,720 Epoch   6 Step:    51600 Batch Loss:     1.467092 Tokens per Sec:    17337, Lr: 0.000300\n",
      "2019-10-30 13:56:48,674 Epoch   6 Step:    51700 Batch Loss:     1.402759 Tokens per Sec:    16870, Lr: 0.000300\n",
      "2019-10-30 13:57:03,748 Epoch   6 Step:    51800 Batch Loss:     1.414402 Tokens per Sec:    16989, Lr: 0.000300\n",
      "2019-10-30 13:57:18,785 Epoch   6 Step:    51900 Batch Loss:     1.659593 Tokens per Sec:    16989, Lr: 0.000300\n",
      "2019-10-30 13:57:33,787 Epoch   6 Step:    52000 Batch Loss:     1.437869 Tokens per Sec:    17072, Lr: 0.000300\n",
      "2019-10-30 13:58:18,070 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 13:58:18,070 Saving new checkpoint.\n",
      "2019-10-30 13:58:18,207 Example #0\n",
      "2019-10-30 13:58:18,207 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 13:58:18,207 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:58:18,207 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 13:58:18,207 Example #1\n",
      "2019-10-30 13:58:18,208 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 13:58:18,208 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 13:58:18,208 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 13:58:18,208 Example #2\n",
      "2019-10-30 13:58:18,208 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 13:58:18,208 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 13:58:18,208 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza mahangaiko gani ?\n",
      "2019-10-30 13:58:18,208 Example #3\n",
      "2019-10-30 13:58:18,208 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 13:58:18,208 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 13:58:18,208 \tHypothesis: Wapendao nyuki na wauaji .\n",
      "2019-10-30 13:58:18,208 Validation result at epoch   6, step    52000: bleu:  32.54, loss: 36361.3867, ppl:   3.7547, duration: 44.4204s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 13:58:33,364 Epoch   6 Step:    52100 Batch Loss:     1.661099 Tokens per Sec:    16926, Lr: 0.000300\n",
      "2019-10-30 13:58:48,906 Epoch   6 Step:    52200 Batch Loss:     1.521668 Tokens per Sec:    16202, Lr: 0.000300\n",
      "2019-10-30 13:59:03,837 Epoch   6 Step:    52300 Batch Loss:     1.542145 Tokens per Sec:    16843, Lr: 0.000300\n",
      "2019-10-30 13:59:18,742 Epoch   6 Step:    52400 Batch Loss:     1.516541 Tokens per Sec:    17207, Lr: 0.000300\n",
      "2019-10-30 13:59:33,591 Epoch   6 Step:    52500 Batch Loss:     1.519559 Tokens per Sec:    17279, Lr: 0.000300\n",
      "2019-10-30 13:59:48,647 Epoch   6 Step:    52600 Batch Loss:     1.547131 Tokens per Sec:    17200, Lr: 0.000300\n",
      "2019-10-30 14:00:03,655 Epoch   6 Step:    52700 Batch Loss:     1.551623 Tokens per Sec:    17072, Lr: 0.000300\n",
      "2019-10-30 14:00:18,640 Epoch   6 Step:    52800 Batch Loss:     1.658617 Tokens per Sec:    17125, Lr: 0.000300\n",
      "2019-10-30 14:00:33,487 Epoch   6 Step:    52900 Batch Loss:     1.283668 Tokens per Sec:    17077, Lr: 0.000300\n",
      "2019-10-30 14:00:48,636 Epoch   6 Step:    53000 Batch Loss:     1.472227 Tokens per Sec:    17247, Lr: 0.000300\n",
      "2019-10-30 14:01:33,132 Example #0\n",
      "2019-10-30 14:01:33,132 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:01:33,132 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:01:33,132 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:01:33,132 Example #1\n",
      "2019-10-30 14:01:33,132 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:01:33,132 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:01:33,133 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:01:33,133 Example #2\n",
      "2019-10-30 14:01:33,133 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:01:33,133 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:01:33,133 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 14:01:33,133 Example #3\n",
      "2019-10-30 14:01:33,133 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:01:33,133 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:01:33,133 \tHypothesis: Mapenzi ya nyuki na mauaji .\n",
      "2019-10-30 14:01:33,133 Validation result at epoch   6, step    53000: bleu:  32.98, loss: 36417.4453, ppl:   3.7623, duration: 44.4962s\n",
      "2019-10-30 14:01:48,160 Epoch   6 Step:    53100 Batch Loss:     1.457736 Tokens per Sec:    17118, Lr: 0.000300\n",
      "2019-10-30 14:02:02,391 Epoch   6 Step:    53200 Batch Loss:     1.684479 Tokens per Sec:    18265, Lr: 0.000300\n",
      "2019-10-30 14:02:16,583 Epoch   6 Step:    53300 Batch Loss:     1.423783 Tokens per Sec:    17898, Lr: 0.000300\n",
      "2019-10-30 14:02:30,893 Epoch   6 Step:    53400 Batch Loss:     1.419099 Tokens per Sec:    18482, Lr: 0.000300\n",
      "2019-10-30 14:02:45,111 Epoch   6 Step:    53500 Batch Loss:     1.711479 Tokens per Sec:    18199, Lr: 0.000300\n",
      "2019-10-30 14:02:59,269 Epoch   6 Step:    53600 Batch Loss:     1.670396 Tokens per Sec:    18246, Lr: 0.000300\n",
      "2019-10-30 14:03:13,580 Epoch   6 Step:    53700 Batch Loss:     1.536170 Tokens per Sec:    17953, Lr: 0.000300\n",
      "2019-10-30 14:03:27,673 Epoch   6 Step:    53800 Batch Loss:     1.620566 Tokens per Sec:    17858, Lr: 0.000300\n",
      "2019-10-30 14:03:42,069 Epoch   6 Step:    53900 Batch Loss:     1.305677 Tokens per Sec:    17484, Lr: 0.000300\n",
      "2019-10-30 14:03:56,468 Epoch   6 Step:    54000 Batch Loss:     1.556951 Tokens per Sec:    17801, Lr: 0.000300\n",
      "2019-10-30 14:04:39,364 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:04:39,364 Saving new checkpoint.\n",
      "2019-10-30 14:04:39,525 Example #0\n",
      "2019-10-30 14:04:39,525 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:04:39,525 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:04:39,525 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:04:39,525 Example #1\n",
      "2019-10-30 14:04:39,525 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:04:39,525 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:04:39,525 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:04:39,525 Example #2\n",
      "2019-10-30 14:04:39,526 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:04:39,526 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:04:39,526 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 14:04:39,526 Example #3\n",
      "2019-10-30 14:04:39,526 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:04:39,526 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:04:39,526 \tHypothesis: Upendo wa nyuki na wauaji .\n",
      "2019-10-30 14:04:39,526 Validation result at epoch   6, step    54000: bleu:  32.73, loss: 36260.9648, ppl:   3.7410, duration: 43.0575s\n",
      "2019-10-30 14:04:54,574 Epoch   6 Step:    54100 Batch Loss:     1.556854 Tokens per Sec:    17408, Lr: 0.000300\n",
      "2019-10-30 14:05:09,188 Epoch   6 Step:    54200 Batch Loss:     1.569410 Tokens per Sec:    17407, Lr: 0.000300\n",
      "2019-10-30 14:05:23,642 Epoch   6 Step:    54300 Batch Loss:     1.239484 Tokens per Sec:    17793, Lr: 0.000300\n",
      "2019-10-30 14:05:37,886 Epoch   6 Step:    54400 Batch Loss:     1.466433 Tokens per Sec:    18064, Lr: 0.000300\n",
      "2019-10-30 14:05:52,166 Epoch   6 Step:    54500 Batch Loss:     1.326031 Tokens per Sec:    18091, Lr: 0.000300\n",
      "2019-10-30 14:06:06,300 Epoch   6 Step:    54600 Batch Loss:     1.557739 Tokens per Sec:    18069, Lr: 0.000300\n",
      "2019-10-30 14:06:20,534 Epoch   6 Step:    54700 Batch Loss:     1.381606 Tokens per Sec:    18168, Lr: 0.000300\n",
      "2019-10-30 14:06:34,740 Epoch   6 Step:    54800 Batch Loss:     1.559092 Tokens per Sec:    18237, Lr: 0.000300\n",
      "2019-10-30 14:06:49,041 Epoch   6 Step:    54900 Batch Loss:     1.872403 Tokens per Sec:    18080, Lr: 0.000300\n",
      "2019-10-30 14:07:03,225 Epoch   6 Step:    55000 Batch Loss:     1.586997 Tokens per Sec:    17923, Lr: 0.000300\n",
      "2019-10-30 14:07:45,073 Example #0\n",
      "2019-10-30 14:07:45,073 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:07:45,073 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:07:45,073 \tHypothesis: Alikuwa mja wa miaka 17 .\n",
      "2019-10-30 14:07:45,073 Example #1\n",
      "2019-10-30 14:07:45,073 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:07:45,073 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:07:45,073 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:07:45,073 Example #2\n",
      "2019-10-30 14:07:45,074 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:07:45,074 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:07:45,074 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na hangaiko laweza kuathirije ?\n",
      "2019-10-30 14:07:45,074 Example #3\n",
      "2019-10-30 14:07:45,074 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:07:45,074 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:07:45,074 \tHypothesis: Wapendao na wabudu .\n",
      "2019-10-30 14:07:45,074 Validation result at epoch   6, step    55000: bleu:  32.65, loss: 36357.1602, ppl:   3.7541, duration: 41.8489s\n",
      "2019-10-30 14:07:59,278 Epoch   6 Step:    55100 Batch Loss:     1.393986 Tokens per Sec:    17997, Lr: 0.000300\n",
      "2019-10-30 14:08:13,439 Epoch   6 Step:    55200 Batch Loss:     1.323987 Tokens per Sec:    18107, Lr: 0.000300\n",
      "2019-10-30 14:08:27,671 Epoch   6 Step:    55300 Batch Loss:     1.446869 Tokens per Sec:    18276, Lr: 0.000300\n",
      "2019-10-30 14:08:41,816 Epoch   6 Step:    55400 Batch Loss:     1.499603 Tokens per Sec:    18189, Lr: 0.000300\n",
      "2019-10-30 14:08:55,997 Epoch   6 Step:    55500 Batch Loss:     1.525807 Tokens per Sec:    18089, Lr: 0.000300\n",
      "2019-10-30 14:09:10,340 Epoch   6 Step:    55600 Batch Loss:     1.353966 Tokens per Sec:    18229, Lr: 0.000300\n",
      "2019-10-30 14:09:24,596 Epoch   6 Step:    55700 Batch Loss:     1.240870 Tokens per Sec:    18176, Lr: 0.000300\n",
      "2019-10-30 14:09:38,877 Epoch   6 Step:    55800 Batch Loss:     1.715463 Tokens per Sec:    17926, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 14:09:53,516 Epoch   6 Step:    55900 Batch Loss:     1.522503 Tokens per Sec:    17794, Lr: 0.000300\n",
      "2019-10-30 14:10:07,864 Epoch   6 Step:    56000 Batch Loss:     1.351592 Tokens per Sec:    17532, Lr: 0.000300\n",
      "2019-10-30 14:10:52,688 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:10:52,688 Saving new checkpoint.\n",
      "2019-10-30 14:10:52,872 Example #0\n",
      "2019-10-30 14:10:52,872 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:10:52,872 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:10:52,873 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:10:52,873 Example #1\n",
      "2019-10-30 14:10:52,873 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:10:52,873 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:10:52,873 \tHypothesis: Biblia haisemi ukarimu .\n",
      "2019-10-30 14:10:52,873 Example #2\n",
      "2019-10-30 14:10:52,873 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:10:52,873 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:10:52,873 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 14:10:52,873 Example #3\n",
      "2019-10-30 14:10:52,873 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:10:52,874 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:10:52,874 \tHypothesis: Wanyama wa nyuki na wauaji .\n",
      "2019-10-30 14:10:52,874 Validation result at epoch   6, step    56000: bleu:  33.16, loss: 35905.9570, ppl:   3.6930, duration: 45.0091s\n",
      "2019-10-30 14:11:08,549 Epoch   6 Step:    56100 Batch Loss:     1.427357 Tokens per Sec:    15824, Lr: 0.000300\n",
      "2019-10-30 14:11:23,737 Epoch   6 Step:    56200 Batch Loss:     1.298689 Tokens per Sec:    17076, Lr: 0.000300\n",
      "2019-10-30 14:11:38,539 Epoch   6 Step:    56300 Batch Loss:     1.399272 Tokens per Sec:    17084, Lr: 0.000300\n",
      "2019-10-30 14:11:41,678 Epoch   6: total training loss 14172.03\n",
      "2019-10-30 14:11:41,678 EPOCH 7\n",
      "2019-10-30 14:11:54,268 Epoch   7 Step:    56400 Batch Loss:     1.298042 Tokens per Sec:    16426, Lr: 0.000300\n",
      "2019-10-30 14:12:08,647 Epoch   7 Step:    56500 Batch Loss:     1.248350 Tokens per Sec:    17805, Lr: 0.000300\n",
      "2019-10-30 14:12:22,926 Epoch   7 Step:    56600 Batch Loss:     1.497327 Tokens per Sec:    17976, Lr: 0.000300\n",
      "2019-10-30 14:12:37,021 Epoch   7 Step:    56700 Batch Loss:     1.428274 Tokens per Sec:    18055, Lr: 0.000300\n",
      "2019-10-30 14:12:51,280 Epoch   7 Step:    56800 Batch Loss:     1.625223 Tokens per Sec:    17973, Lr: 0.000300\n",
      "2019-10-30 14:13:05,645 Epoch   7 Step:    56900 Batch Loss:     1.403676 Tokens per Sec:    18201, Lr: 0.000300\n",
      "2019-10-30 14:13:19,933 Epoch   7 Step:    57000 Batch Loss:     1.439426 Tokens per Sec:    18097, Lr: 0.000300\n",
      "2019-10-30 14:14:02,328 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:14:02,328 Saving new checkpoint.\n",
      "2019-10-30 14:14:02,460 Example #0\n",
      "2019-10-30 14:14:02,461 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:14:02,461 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:14:02,461 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:14:02,461 Example #1\n",
      "2019-10-30 14:14:02,461 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:14:02,461 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:14:02,461 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:14:02,461 Example #2\n",
      "2019-10-30 14:14:02,461 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:14:02,461 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:14:02,461 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 14:14:02,461 Example #3\n",
      "2019-10-30 14:14:02,461 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:14:02,461 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:14:02,461 \tHypothesis: Wanyama wa nyuki na wauaji .\n",
      "2019-10-30 14:14:02,462 Validation result at epoch   7, step    57000: bleu:  33.20, loss: 35867.7148, ppl:   3.6878, duration: 42.5277s\n",
      "2019-10-30 14:14:16,834 Epoch   7 Step:    57100 Batch Loss:     1.567499 Tokens per Sec:    17873, Lr: 0.000300\n",
      "2019-10-30 14:14:31,115 Epoch   7 Step:    57200 Batch Loss:     1.510049 Tokens per Sec:    17866, Lr: 0.000300\n",
      "2019-10-30 14:14:45,417 Epoch   7 Step:    57300 Batch Loss:     1.438152 Tokens per Sec:    18199, Lr: 0.000300\n",
      "2019-10-30 14:14:59,900 Epoch   7 Step:    57400 Batch Loss:     1.542856 Tokens per Sec:    18053, Lr: 0.000300\n",
      "2019-10-30 14:15:14,235 Epoch   7 Step:    57500 Batch Loss:     1.349009 Tokens per Sec:    17876, Lr: 0.000300\n",
      "2019-10-30 14:15:28,452 Epoch   7 Step:    57600 Batch Loss:     1.648202 Tokens per Sec:    17602, Lr: 0.000300\n",
      "2019-10-30 14:15:42,794 Epoch   7 Step:    57700 Batch Loss:     1.441609 Tokens per Sec:    17961, Lr: 0.000300\n",
      "2019-10-30 14:15:57,102 Epoch   7 Step:    57800 Batch Loss:     1.661107 Tokens per Sec:    17827, Lr: 0.000300\n",
      "2019-10-30 14:16:11,497 Epoch   7 Step:    57900 Batch Loss:     1.466878 Tokens per Sec:    17867, Lr: 0.000300\n",
      "2019-10-30 14:16:25,723 Epoch   7 Step:    58000 Batch Loss:     1.502746 Tokens per Sec:    17632, Lr: 0.000300\n",
      "2019-10-30 14:17:08,345 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:17:08,345 Saving new checkpoint.\n",
      "2019-10-30 14:17:08,479 Example #0\n",
      "2019-10-30 14:17:08,480 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:17:08,480 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:17:08,480 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:17:08,480 Example #1\n",
      "2019-10-30 14:17:08,480 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:17:08,480 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:17:08,480 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:17:08,480 Example #2\n",
      "2019-10-30 14:17:08,480 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:17:08,480 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:17:08,480 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 14:17:08,480 Example #3\n",
      "2019-10-30 14:17:08,480 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:17:08,480 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:17:08,480 \tHypothesis: Madhabahu na mauaji .\n",
      "2019-10-30 14:17:08,480 Validation result at epoch   7, step    58000: bleu:  32.85, loss: 35726.8008, ppl:   3.6690, duration: 42.7568s\n",
      "2019-10-30 14:17:22,958 Epoch   7 Step:    58100 Batch Loss:     1.364648 Tokens per Sec:    17968, Lr: 0.000300\n",
      "2019-10-30 14:17:37,316 Epoch   7 Step:    58200 Batch Loss:     1.382891 Tokens per Sec:    17756, Lr: 0.000300\n",
      "2019-10-30 14:17:51,771 Epoch   7 Step:    58300 Batch Loss:     1.617212 Tokens per Sec:    17927, Lr: 0.000300\n",
      "2019-10-30 14:18:06,102 Epoch   7 Step:    58400 Batch Loss:     1.474667 Tokens per Sec:    17955, Lr: 0.000300\n",
      "2019-10-30 14:18:20,468 Epoch   7 Step:    58500 Batch Loss:     1.758458 Tokens per Sec:    17865, Lr: 0.000300\n",
      "2019-10-30 14:18:34,772 Epoch   7 Step:    58600 Batch Loss:     1.504095 Tokens per Sec:    17877, Lr: 0.000300\n",
      "2019-10-30 14:18:49,091 Epoch   7 Step:    58700 Batch Loss:     1.418574 Tokens per Sec:    17765, Lr: 0.000300\n",
      "2019-10-30 14:19:03,766 Epoch   7 Step:    58800 Batch Loss:     1.304159 Tokens per Sec:    17906, Lr: 0.000300\n",
      "2019-10-30 14:19:18,158 Epoch   7 Step:    58900 Batch Loss:     1.340327 Tokens per Sec:    17799, Lr: 0.000300\n",
      "2019-10-30 14:19:32,444 Epoch   7 Step:    59000 Batch Loss:     1.407108 Tokens per Sec:    17720, Lr: 0.000300\n",
      "2019-10-30 14:20:15,076 Example #0\n",
      "2019-10-30 14:20:15,077 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:20:15,077 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:20:15,077 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 14:20:15,077 Example #1\n",
      "2019-10-30 14:20:15,077 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:20:15,077 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:20:15,077 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:20:15,077 Example #2\n",
      "2019-10-30 14:20:15,077 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:20:15,077 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:20:15,077 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea kwa nini ?\n",
      "2019-10-30 14:20:15,077 Example #3\n",
      "2019-10-30 14:20:15,077 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:20:15,077 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:20:15,078 \tHypothesis: Wapendao nyuki na wabudu .\n",
      "2019-10-30 14:20:15,078 Validation result at epoch   7, step    59000: bleu:  33.48, loss: 35735.7266, ppl:   3.6702, duration: 42.6328s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 14:20:29,516 Epoch   7 Step:    59100 Batch Loss:     1.428786 Tokens per Sec:    17970, Lr: 0.000300\n",
      "2019-10-30 14:20:44,033 Epoch   7 Step:    59200 Batch Loss:     1.375013 Tokens per Sec:    17853, Lr: 0.000300\n",
      "2019-10-30 14:20:58,444 Epoch   7 Step:    59300 Batch Loss:     1.338412 Tokens per Sec:    17807, Lr: 0.000300\n",
      "2019-10-30 14:21:12,716 Epoch   7 Step:    59400 Batch Loss:     1.765574 Tokens per Sec:    17652, Lr: 0.000300\n",
      "2019-10-30 14:21:27,377 Epoch   7 Step:    59500 Batch Loss:     1.338356 Tokens per Sec:    17925, Lr: 0.000300\n",
      "2019-10-30 14:21:41,794 Epoch   7 Step:    59600 Batch Loss:     1.379952 Tokens per Sec:    17704, Lr: 0.000300\n",
      "2019-10-30 14:21:56,268 Epoch   7 Step:    59700 Batch Loss:     1.586899 Tokens per Sec:    17640, Lr: 0.000300\n",
      "2019-10-30 14:22:10,837 Epoch   7 Step:    59800 Batch Loss:     1.599199 Tokens per Sec:    17399, Lr: 0.000300\n",
      "2019-10-30 14:22:25,409 Epoch   7 Step:    59900 Batch Loss:     1.381836 Tokens per Sec:    17957, Lr: 0.000300\n",
      "2019-10-30 14:22:39,995 Epoch   7 Step:    60000 Batch Loss:     1.509220 Tokens per Sec:    17668, Lr: 0.000300\n",
      "2019-10-30 14:23:22,970 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:23:22,970 Saving new checkpoint.\n",
      "2019-10-30 14:23:23,110 Example #0\n",
      "2019-10-30 14:23:23,110 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:23:23,110 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:23:23,111 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:23:23,111 Example #1\n",
      "2019-10-30 14:23:23,111 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:23:23,111 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:23:23,111 \tHypothesis: Biblia haisemi umashuhuri .\n",
      "2019-10-30 14:23:23,111 Example #2\n",
      "2019-10-30 14:23:23,111 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:23:23,111 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:23:23,111 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri mahangaiko ?\n",
      "2019-10-30 14:23:23,111 Example #3\n",
      "2019-10-30 14:23:23,111 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:23:23,111 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:23:23,111 \tHypothesis: Wanyama wa nyuki na wabudu .\n",
      "2019-10-30 14:23:23,111 Validation result at epoch   7, step    60000: bleu:  33.39, loss: 35672.8906, ppl:   3.6618, duration: 43.1158s\n",
      "2019-10-30 14:23:37,771 Epoch   7 Step:    60100 Batch Loss:     1.262681 Tokens per Sec:    17779, Lr: 0.000300\n",
      "2019-10-30 14:23:52,189 Epoch   7 Step:    60200 Batch Loss:     1.437471 Tokens per Sec:    17722, Lr: 0.000300\n",
      "2019-10-30 14:24:06,614 Epoch   7 Step:    60300 Batch Loss:     1.543776 Tokens per Sec:    17640, Lr: 0.000300\n",
      "2019-10-30 14:24:21,099 Epoch   7 Step:    60400 Batch Loss:     1.481710 Tokens per Sec:    17690, Lr: 0.000300\n",
      "2019-10-30 14:24:35,487 Epoch   7 Step:    60500 Batch Loss:     1.518312 Tokens per Sec:    17343, Lr: 0.000300\n",
      "2019-10-30 14:24:49,974 Epoch   7 Step:    60600 Batch Loss:     1.322987 Tokens per Sec:    17988, Lr: 0.000300\n",
      "2019-10-30 14:25:04,444 Epoch   7 Step:    60700 Batch Loss:     1.375759 Tokens per Sec:    18167, Lr: 0.000300\n",
      "2019-10-30 14:25:18,858 Epoch   7 Step:    60800 Batch Loss:     1.546180 Tokens per Sec:    18028, Lr: 0.000300\n",
      "2019-10-30 14:25:33,541 Epoch   7 Step:    60900 Batch Loss:     1.566452 Tokens per Sec:    18008, Lr: 0.000300\n",
      "2019-10-30 14:25:47,913 Epoch   7 Step:    61000 Batch Loss:     1.496721 Tokens per Sec:    17686, Lr: 0.000300\n",
      "2019-10-30 14:26:30,725 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:26:30,725 Saving new checkpoint.\n",
      "2019-10-30 14:26:30,859 Example #0\n",
      "2019-10-30 14:26:30,859 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:26:30,859 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:26:30,859 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:26:30,859 Example #1\n",
      "2019-10-30 14:26:30,859 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:26:30,859 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:26:30,859 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 14:26:30,859 Example #2\n",
      "2019-10-30 14:26:30,859 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:26:30,859 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:26:30,859 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 14:26:30,860 Example #3\n",
      "2019-10-30 14:26:30,860 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:26:30,860 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:26:30,860 \tHypothesis: Mapenzi ya nyuki na wabusu .\n",
      "2019-10-30 14:26:30,860 Validation result at epoch   7, step    61000: bleu:  33.60, loss: 35411.0312, ppl:   3.6271, duration: 42.9465s\n",
      "2019-10-30 14:26:45,471 Epoch   7 Step:    61100 Batch Loss:     1.511982 Tokens per Sec:    18012, Lr: 0.000300\n",
      "2019-10-30 14:26:59,844 Epoch   7 Step:    61200 Batch Loss:     1.333393 Tokens per Sec:    18018, Lr: 0.000300\n",
      "2019-10-30 14:27:14,236 Epoch   7 Step:    61300 Batch Loss:     1.641596 Tokens per Sec:    17695, Lr: 0.000300\n",
      "2019-10-30 14:27:28,786 Epoch   7 Step:    61400 Batch Loss:     1.498012 Tokens per Sec:    17787, Lr: 0.000300\n",
      "2019-10-30 14:27:43,216 Epoch   7 Step:    61500 Batch Loss:     1.460085 Tokens per Sec:    17972, Lr: 0.000300\n",
      "2019-10-30 14:27:57,353 Epoch   7 Step:    61600 Batch Loss:     1.458960 Tokens per Sec:    17762, Lr: 0.000300\n",
      "2019-10-30 14:28:11,862 Epoch   7 Step:    61700 Batch Loss:     1.424599 Tokens per Sec:    17745, Lr: 0.000300\n",
      "2019-10-30 14:28:26,538 Epoch   7 Step:    61800 Batch Loss:     1.478903 Tokens per Sec:    17674, Lr: 0.000300\n",
      "2019-10-30 14:28:40,978 Epoch   7 Step:    61900 Batch Loss:     1.641393 Tokens per Sec:    17928, Lr: 0.000300\n",
      "2019-10-30 14:28:55,352 Epoch   7 Step:    62000 Batch Loss:     1.500181 Tokens per Sec:    17481, Lr: 0.000300\n",
      "2019-10-30 14:29:38,310 Example #0\n",
      "2019-10-30 14:29:38,310 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:29:38,310 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:29:38,310 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:29:38,310 Example #1\n",
      "2019-10-30 14:29:38,310 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:29:38,310 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:29:38,310 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:29:38,310 Example #2\n",
      "2019-10-30 14:29:38,310 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:29:38,310 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:29:38,310 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 14:29:38,311 Example #3\n",
      "2019-10-30 14:29:38,311 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:29:38,311 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:29:38,311 \tHypothesis: Mapenzi ya nyuki na mauaji .\n",
      "2019-10-30 14:29:38,311 Validation result at epoch   7, step    62000: bleu:  33.24, loss: 35429.2266, ppl:   3.6295, duration: 42.9584s\n",
      "2019-10-30 14:29:52,804 Epoch   7 Step:    62100 Batch Loss:     1.615684 Tokens per Sec:    17570, Lr: 0.000300\n",
      "2019-10-30 14:30:07,288 Epoch   7 Step:    62200 Batch Loss:     1.411690 Tokens per Sec:    17498, Lr: 0.000300\n",
      "2019-10-30 14:30:21,900 Epoch   7 Step:    62300 Batch Loss:     1.441660 Tokens per Sec:    17512, Lr: 0.000300\n",
      "2019-10-30 14:30:36,350 Epoch   7 Step:    62400 Batch Loss:     1.487822 Tokens per Sec:    17662, Lr: 0.000300\n",
      "2019-10-30 14:30:50,853 Epoch   7 Step:    62500 Batch Loss:     1.620019 Tokens per Sec:    17566, Lr: 0.000300\n",
      "2019-10-30 14:31:05,561 Epoch   7 Step:    62600 Batch Loss:     1.274107 Tokens per Sec:    17817, Lr: 0.000300\n",
      "2019-10-30 14:31:20,082 Epoch   7 Step:    62700 Batch Loss:     1.452067 Tokens per Sec:    17796, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 14:31:34,638 Epoch   7 Step:    62800 Batch Loss:     1.527226 Tokens per Sec:    17742, Lr: 0.000300\n",
      "2019-10-30 14:31:49,067 Epoch   7 Step:    62900 Batch Loss:     1.579070 Tokens per Sec:    17343, Lr: 0.000300\n",
      "2019-10-30 14:32:03,510 Epoch   7 Step:    63000 Batch Loss:     1.538301 Tokens per Sec:    17182, Lr: 0.000300\n",
      "2019-10-30 14:32:46,483 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:32:46,483 Saving new checkpoint.\n",
      "2019-10-30 14:32:46,619 Example #0\n",
      "2019-10-30 14:32:46,619 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:32:46,619 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:32:46,619 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:32:46,619 Example #1\n",
      "2019-10-30 14:32:46,619 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:32:46,619 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:32:46,619 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 14:32:46,619 Example #2\n",
      "2019-10-30 14:32:46,619 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:32:46,619 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:32:46,619 \tHypothesis: Ni mambo gani yanayofanya hangaiko , na ni nini kinachoweza kuathiri mahangaiko ?\n",
      "2019-10-30 14:32:46,619 Example #3\n",
      "2019-10-30 14:32:46,619 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:32:46,620 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:32:46,620 \tHypothesis: Mapenzi ya nyuki na wabusu .\n",
      "2019-10-30 14:32:46,620 Validation result at epoch   7, step    63000: bleu:  33.51, loss: 35401.9688, ppl:   3.6259, duration: 43.1089s\n",
      "2019-10-30 14:33:01,287 Epoch   7 Step:    63100 Batch Loss:     1.374836 Tokens per Sec:    17866, Lr: 0.000300\n",
      "2019-10-30 14:33:16,020 Epoch   7 Step:    63200 Batch Loss:     1.453205 Tokens per Sec:    18106, Lr: 0.000300\n",
      "2019-10-30 14:33:30,461 Epoch   7 Step:    63300 Batch Loss:     1.267134 Tokens per Sec:    17537, Lr: 0.000300\n",
      "2019-10-30 14:33:45,098 Epoch   7 Step:    63400 Batch Loss:     1.350035 Tokens per Sec:    17584, Lr: 0.000300\n",
      "2019-10-30 14:33:59,617 Epoch   7 Step:    63500 Batch Loss:     1.471067 Tokens per Sec:    17360, Lr: 0.000300\n",
      "2019-10-30 14:34:14,304 Epoch   7 Step:    63600 Batch Loss:     1.243024 Tokens per Sec:    17865, Lr: 0.000300\n",
      "2019-10-30 14:34:28,957 Epoch   7 Step:    63700 Batch Loss:     1.452374 Tokens per Sec:    17757, Lr: 0.000300\n",
      "2019-10-30 14:34:43,531 Epoch   7 Step:    63800 Batch Loss:     1.520323 Tokens per Sec:    17421, Lr: 0.000300\n",
      "2019-10-30 14:34:58,269 Epoch   7 Step:    63900 Batch Loss:     1.507178 Tokens per Sec:    17680, Lr: 0.000300\n",
      "2019-10-30 14:35:13,083 Epoch   7 Step:    64000 Batch Loss:     1.500874 Tokens per Sec:    17024, Lr: 0.000300\n",
      "2019-10-30 14:35:56,366 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:35:56,366 Saving new checkpoint.\n",
      "2019-10-30 14:35:56,507 Example #0\n",
      "2019-10-30 14:35:56,508 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:35:56,508 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:35:56,508 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:35:56,508 Example #1\n",
      "2019-10-30 14:35:56,508 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:35:56,508 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:35:56,508 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:35:56,508 Example #2\n",
      "2019-10-30 14:35:56,508 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:35:56,508 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:35:56,508 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 14:35:56,508 Example #3\n",
      "2019-10-30 14:35:56,508 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:35:56,508 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:35:56,508 \tHypothesis: Mapenzi ya nyuki na mauaji .\n",
      "2019-10-30 14:35:56,508 Validation result at epoch   7, step    64000: bleu:  33.53, loss: 35357.3594, ppl:   3.6200, duration: 43.4253s\n",
      "2019-10-30 14:36:11,171 Epoch   7 Step:    64100 Batch Loss:     1.534160 Tokens per Sec:    17226, Lr: 0.000300\n",
      "2019-10-30 14:36:25,961 Epoch   7 Step:    64200 Batch Loss:     1.393339 Tokens per Sec:    17513, Lr: 0.000300\n",
      "2019-10-30 14:36:40,747 Epoch   7 Step:    64300 Batch Loss:     1.366787 Tokens per Sec:    17531, Lr: 0.000300\n",
      "2019-10-30 14:36:55,132 Epoch   7 Step:    64400 Batch Loss:     1.345758 Tokens per Sec:    17579, Lr: 0.000300\n",
      "2019-10-30 14:37:09,678 Epoch   7 Step:    64500 Batch Loss:     1.432564 Tokens per Sec:    17807, Lr: 0.000300\n",
      "2019-10-30 14:37:24,391 Epoch   7 Step:    64600 Batch Loss:     1.741451 Tokens per Sec:    17617, Lr: 0.000300\n",
      "2019-10-30 14:37:38,966 Epoch   7 Step:    64700 Batch Loss:     1.524304 Tokens per Sec:    17428, Lr: 0.000300\n",
      "2019-10-30 14:37:53,676 Epoch   7 Step:    64800 Batch Loss:     1.353265 Tokens per Sec:    17587, Lr: 0.000300\n",
      "2019-10-30 14:38:08,299 Epoch   7 Step:    64900 Batch Loss:     1.472197 Tokens per Sec:    17659, Lr: 0.000300\n",
      "2019-10-30 14:38:22,994 Epoch   7 Step:    65000 Batch Loss:     1.518591 Tokens per Sec:    17436, Lr: 0.000300\n",
      "2019-10-30 14:39:06,108 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:39:06,108 Saving new checkpoint.\n",
      "2019-10-30 14:39:06,249 Example #0\n",
      "2019-10-30 14:39:06,250 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:39:06,250 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:39:06,250 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:39:06,250 Example #1\n",
      "2019-10-30 14:39:06,250 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:39:06,250 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:39:06,250 \tHypothesis: Biblia haidhuru uumbaji .\n",
      "2019-10-30 14:39:06,250 Example #2\n",
      "2019-10-30 14:39:06,250 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:39:06,250 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:39:06,250 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 14:39:06,250 Example #3\n",
      "2019-10-30 14:39:06,250 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:39:06,250 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:39:06,250 \tHypothesis: Watu hutenda kwa mashavu na majumba .\n",
      "2019-10-30 14:39:06,250 Validation result at epoch   7, step    65000: bleu:  33.31, loss: 35351.8008, ppl:   3.6193, duration: 43.2563s\n",
      "2019-10-30 14:39:20,989 Epoch   7 Step:    65100 Batch Loss:     1.489587 Tokens per Sec:    17551, Lr: 0.000300\n",
      "2019-10-30 14:39:35,629 Epoch   7 Step:    65200 Batch Loss:     1.560602 Tokens per Sec:    17683, Lr: 0.000300\n",
      "2019-10-30 14:39:50,199 Epoch   7 Step:    65300 Batch Loss:     1.398709 Tokens per Sec:    17623, Lr: 0.000300\n",
      "2019-10-30 14:40:05,025 Epoch   7 Step:    65400 Batch Loss:     1.398295 Tokens per Sec:    17508, Lr: 0.000300\n",
      "2019-10-30 14:40:19,776 Epoch   7 Step:    65500 Batch Loss:     1.472261 Tokens per Sec:    17688, Lr: 0.000300\n",
      "2019-10-30 14:40:34,503 Epoch   7 Step:    65600 Batch Loss:     1.522880 Tokens per Sec:    17417, Lr: 0.000300\n",
      "2019-10-30 14:40:48,983 Epoch   7 Step:    65700 Batch Loss:     1.325106 Tokens per Sec:    17340, Lr: 0.000300\n",
      "2019-10-30 14:40:49,291 Epoch   7: total training loss 13714.24\n",
      "2019-10-30 14:40:49,292 EPOCH 8\n",
      "2019-10-30 14:41:04,745 Epoch   8 Step:    65800 Batch Loss:     1.590087 Tokens per Sec:    16522, Lr: 0.000300\n",
      "2019-10-30 14:41:19,364 Epoch   8 Step:    65900 Batch Loss:     1.288923 Tokens per Sec:    17641, Lr: 0.000300\n",
      "2019-10-30 14:41:33,809 Epoch   8 Step:    66000 Batch Loss:     1.404528 Tokens per Sec:    17672, Lr: 0.000300\n",
      "2019-10-30 14:42:17,043 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:42:17,044 Saving new checkpoint.\n",
      "2019-10-30 14:42:17,229 Example #0\n",
      "2019-10-30 14:42:17,229 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:42:17,229 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:42:17,229 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:42:17,229 Example #1\n",
      "2019-10-30 14:42:17,229 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:42:17,229 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:42:17,229 \tHypothesis: Biblia haidhuru .\n",
      "2019-10-30 14:42:17,229 Example #2\n",
      "2019-10-30 14:42:17,229 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:42:17,229 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:42:17,230 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 14:42:17,230 Example #3\n",
      "2019-10-30 14:42:17,230 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:42:17,230 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:42:17,230 \tHypothesis: Mapenzi ya nyuki na mabusu .\n",
      "2019-10-30 14:42:17,230 Validation result at epoch   8, step    66000: bleu:  33.85, loss: 34965.4102, ppl:   3.5687, duration: 43.4206s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 14:42:32,031 Epoch   8 Step:    66100 Batch Loss:     1.477430 Tokens per Sec:    17608, Lr: 0.000300\n",
      "2019-10-30 14:42:46,505 Epoch   8 Step:    66200 Batch Loss:     1.368944 Tokens per Sec:    17406, Lr: 0.000300\n",
      "2019-10-30 14:43:00,911 Epoch   8 Step:    66300 Batch Loss:     1.261959 Tokens per Sec:    17545, Lr: 0.000300\n",
      "2019-10-30 14:43:15,520 Epoch   8 Step:    66400 Batch Loss:     1.442837 Tokens per Sec:    17791, Lr: 0.000300\n",
      "2019-10-30 14:43:30,183 Epoch   8 Step:    66500 Batch Loss:     1.298256 Tokens per Sec:    17403, Lr: 0.000300\n",
      "2019-10-30 14:43:44,751 Epoch   8 Step:    66600 Batch Loss:     1.385377 Tokens per Sec:    17611, Lr: 0.000300\n",
      "2019-10-30 14:43:59,381 Epoch   8 Step:    66700 Batch Loss:     1.343163 Tokens per Sec:    17889, Lr: 0.000300\n",
      "2019-10-30 14:44:14,022 Epoch   8 Step:    66800 Batch Loss:     1.413364 Tokens per Sec:    17513, Lr: 0.000300\n",
      "2019-10-30 14:44:28,735 Epoch   8 Step:    66900 Batch Loss:     1.330271 Tokens per Sec:    17441, Lr: 0.000300\n",
      "2019-10-30 14:44:43,468 Epoch   8 Step:    67000 Batch Loss:     1.351167 Tokens per Sec:    17377, Lr: 0.000300\n",
      "2019-10-30 14:45:26,849 Example #0\n",
      "2019-10-30 14:45:26,849 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:45:26,849 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:45:26,849 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 14:45:26,849 Example #1\n",
      "2019-10-30 14:45:26,850 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:45:26,850 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:45:26,850 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 14:45:26,850 Example #2\n",
      "2019-10-30 14:45:26,850 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:45:26,850 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:45:26,850 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 14:45:26,850 Example #3\n",
      "2019-10-30 14:45:26,850 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:45:26,850 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:45:26,850 \tHypothesis: Mapenzi ya nyuki na busu .\n",
      "2019-10-30 14:45:26,850 Validation result at epoch   8, step    67000: bleu:  33.88, loss: 35010.4570, ppl:   3.5746, duration: 43.3822s\n",
      "2019-10-30 14:45:41,582 Epoch   8 Step:    67100 Batch Loss:     1.521420 Tokens per Sec:    16896, Lr: 0.000300\n",
      "2019-10-30 14:45:56,504 Epoch   8 Step:    67200 Batch Loss:     1.220242 Tokens per Sec:    17702, Lr: 0.000300\n",
      "2019-10-30 14:46:11,042 Epoch   8 Step:    67300 Batch Loss:     1.387276 Tokens per Sec:    17380, Lr: 0.000300\n",
      "2019-10-30 14:46:25,586 Epoch   8 Step:    67400 Batch Loss:     1.738011 Tokens per Sec:    17652, Lr: 0.000300\n",
      "2019-10-30 14:46:40,088 Epoch   8 Step:    67500 Batch Loss:     1.307685 Tokens per Sec:    17737, Lr: 0.000300\n",
      "2019-10-30 14:46:54,611 Epoch   8 Step:    67600 Batch Loss:     1.503687 Tokens per Sec:    17578, Lr: 0.000300\n",
      "2019-10-30 14:47:09,217 Epoch   8 Step:    67700 Batch Loss:     1.325327 Tokens per Sec:    17363, Lr: 0.000300\n",
      "2019-10-30 14:47:23,847 Epoch   8 Step:    67800 Batch Loss:     1.434350 Tokens per Sec:    17472, Lr: 0.000300\n",
      "2019-10-30 14:47:38,357 Epoch   8 Step:    67900 Batch Loss:     1.451887 Tokens per Sec:    17263, Lr: 0.000300\n",
      "2019-10-30 14:47:52,893 Epoch   8 Step:    68000 Batch Loss:     1.584597 Tokens per Sec:    17441, Lr: 0.000300\n",
      "2019-10-30 14:48:36,247 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 14:48:36,247 Saving new checkpoint.\n",
      "2019-10-30 14:48:36,383 Example #0\n",
      "2019-10-30 14:48:36,383 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:48:36,383 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:48:36,383 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:48:36,383 Example #1\n",
      "2019-10-30 14:48:36,384 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:48:36,384 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:48:36,384 \tHypothesis: Biblia haisemi uumbaji .\n",
      "2019-10-30 14:48:36,384 Example #2\n",
      "2019-10-30 14:48:36,384 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:48:36,384 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:48:36,384 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuathiri nini ?\n",
      "2019-10-30 14:48:36,384 Example #3\n",
      "2019-10-30 14:48:36,384 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:48:36,384 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:48:36,384 \tHypothesis: Mapenzi ya nyuki na makaburi .\n",
      "2019-10-30 14:48:36,384 Validation result at epoch   8, step    68000: bleu:  33.97, loss: 34708.2969, ppl:   3.5355, duration: 43.4905s\n",
      "2019-10-30 14:48:51,111 Epoch   8 Step:    68100 Batch Loss:     1.349465 Tokens per Sec:    17317, Lr: 0.000300\n",
      "2019-10-30 14:49:05,791 Epoch   8 Step:    68200 Batch Loss:     1.290081 Tokens per Sec:    17662, Lr: 0.000300\n",
      "2019-10-30 14:49:20,416 Epoch   8 Step:    68300 Batch Loss:     1.263906 Tokens per Sec:    17691, Lr: 0.000300\n",
      "2019-10-30 14:49:35,243 Epoch   8 Step:    68400 Batch Loss:     1.370213 Tokens per Sec:    17589, Lr: 0.000300\n",
      "2019-10-30 14:49:50,125 Epoch   8 Step:    68500 Batch Loss:     1.738833 Tokens per Sec:    17189, Lr: 0.000300\n",
      "2019-10-30 14:50:04,873 Epoch   8 Step:    68600 Batch Loss:     1.415558 Tokens per Sec:    17615, Lr: 0.000300\n",
      "2019-10-30 14:50:19,483 Epoch   8 Step:    68700 Batch Loss:     1.589280 Tokens per Sec:    17247, Lr: 0.000300\n",
      "2019-10-30 14:50:34,358 Epoch   8 Step:    68800 Batch Loss:     1.554711 Tokens per Sec:    17703, Lr: 0.000300\n",
      "2019-10-30 14:50:49,070 Epoch   8 Step:    68900 Batch Loss:     1.608720 Tokens per Sec:    17602, Lr: 0.000300\n",
      "2019-10-30 14:51:03,718 Epoch   8 Step:    69000 Batch Loss:     1.476094 Tokens per Sec:    17674, Lr: 0.000300\n",
      "2019-10-30 14:51:46,967 Example #0\n",
      "2019-10-30 14:51:46,967 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:51:46,967 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:51:46,967 \tHypothesis: Alikuwa na umri wa miaka 17 .\n",
      "2019-10-30 14:51:46,967 Example #1\n",
      "2019-10-30 14:51:46,967 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:51:46,967 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:51:46,967 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 14:51:46,967 Example #2\n",
      "2019-10-30 14:51:46,968 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:51:46,968 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:51:46,968 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 14:51:46,968 Example #3\n",
      "2019-10-30 14:51:46,968 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:51:46,968 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:51:46,968 \tHypothesis: Watu hutenda kwa majumba na majumba .\n",
      "2019-10-30 14:51:46,968 Validation result at epoch   8, step    69000: bleu:  33.98, loss: 34829.8516, ppl:   3.5512, duration: 43.2497s\n",
      "2019-10-30 14:52:01,757 Epoch   8 Step:    69100 Batch Loss:     1.355072 Tokens per Sec:    17470, Lr: 0.000300\n",
      "2019-10-30 14:52:16,471 Epoch   8 Step:    69200 Batch Loss:     1.511795 Tokens per Sec:    17702, Lr: 0.000300\n",
      "2019-10-30 14:52:31,162 Epoch   8 Step:    69300 Batch Loss:     1.465543 Tokens per Sec:    17374, Lr: 0.000300\n",
      "2019-10-30 14:52:45,737 Epoch   8 Step:    69400 Batch Loss:     1.414832 Tokens per Sec:    17790, Lr: 0.000300\n",
      "2019-10-30 14:53:00,464 Epoch   8 Step:    69500 Batch Loss:     1.661010 Tokens per Sec:    17808, Lr: 0.000300\n",
      "2019-10-30 14:53:15,075 Epoch   8 Step:    69600 Batch Loss:     1.328966 Tokens per Sec:    17526, Lr: 0.000300\n",
      "2019-10-30 14:53:29,659 Epoch   8 Step:    69700 Batch Loss:     1.330781 Tokens per Sec:    17543, Lr: 0.000300\n",
      "2019-10-30 14:53:44,322 Epoch   8 Step:    69800 Batch Loss:     1.520741 Tokens per Sec:    17519, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 14:53:58,907 Epoch   8 Step:    69900 Batch Loss:     1.155964 Tokens per Sec:    17476, Lr: 0.000300\n",
      "2019-10-30 14:54:13,673 Epoch   8 Step:    70000 Batch Loss:     1.308532 Tokens per Sec:    17479, Lr: 0.000300\n",
      "2019-10-30 14:54:57,217 Example #0\n",
      "2019-10-30 14:54:57,217 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:54:57,217 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:54:57,217 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:54:57,217 Example #1\n",
      "2019-10-30 14:54:57,217 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:54:57,217 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:54:57,217 \tHypothesis: Biblia haidhuru .\n",
      "2019-10-30 14:54:57,217 Example #2\n",
      "2019-10-30 14:54:57,218 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:54:57,218 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:54:57,218 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyizwa ?\n",
      "2019-10-30 14:54:57,218 Example #3\n",
      "2019-10-30 14:54:57,218 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:54:57,218 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:54:57,218 \tHypothesis: Upendo wa nyuki na busu .\n",
      "2019-10-30 14:54:57,218 Validation result at epoch   8, step    70000: bleu:  34.02, loss: 34715.0391, ppl:   3.5364, duration: 43.5446s\n",
      "2019-10-30 14:55:12,026 Epoch   8 Step:    70100 Batch Loss:     1.619486 Tokens per Sec:    17370, Lr: 0.000300\n",
      "2019-10-30 14:55:26,809 Epoch   8 Step:    70200 Batch Loss:     1.595089 Tokens per Sec:    17682, Lr: 0.000300\n",
      "2019-10-30 14:55:41,610 Epoch   8 Step:    70300 Batch Loss:     1.560076 Tokens per Sec:    17797, Lr: 0.000300\n",
      "2019-10-30 14:55:56,259 Epoch   8 Step:    70400 Batch Loss:     1.353740 Tokens per Sec:    17715, Lr: 0.000300\n",
      "2019-10-30 14:56:10,693 Epoch   8 Step:    70500 Batch Loss:     1.562477 Tokens per Sec:    17337, Lr: 0.000300\n",
      "2019-10-30 14:56:25,275 Epoch   8 Step:    70600 Batch Loss:     1.387799 Tokens per Sec:    17677, Lr: 0.000300\n",
      "2019-10-30 14:56:39,768 Epoch   8 Step:    70700 Batch Loss:     1.275784 Tokens per Sec:    17738, Lr: 0.000300\n",
      "2019-10-30 14:56:54,544 Epoch   8 Step:    70800 Batch Loss:     1.654707 Tokens per Sec:    17879, Lr: 0.000300\n",
      "2019-10-30 14:57:09,138 Epoch   8 Step:    70900 Batch Loss:     1.541340 Tokens per Sec:    17404, Lr: 0.000300\n",
      "2019-10-30 14:57:23,781 Epoch   8 Step:    71000 Batch Loss:     1.228624 Tokens per Sec:    17451, Lr: 0.000300\n",
      "2019-10-30 14:58:07,377 Example #0\n",
      "2019-10-30 14:58:07,378 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 14:58:07,378 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:58:07,378 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 14:58:07,378 Example #1\n",
      "2019-10-30 14:58:07,378 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 14:58:07,378 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 14:58:07,378 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 14:58:07,378 Example #2\n",
      "2019-10-30 14:58:07,378 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 14:58:07,378 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 14:58:07,378 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kuathiri mahangaiko ?\n",
      "2019-10-30 14:58:07,378 Example #3\n",
      "2019-10-30 14:58:07,378 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 14:58:07,378 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 14:58:07,379 \tHypothesis: Wapenzi wa nyuki na wabusu .\n",
      "2019-10-30 14:58:07,379 Validation result at epoch   8, step    71000: bleu:  33.73, loss: 34794.7695, ppl:   3.5466, duration: 43.5975s\n",
      "2019-10-30 14:58:22,186 Epoch   8 Step:    71100 Batch Loss:     1.366058 Tokens per Sec:    17511, Lr: 0.000300\n",
      "2019-10-30 14:58:36,859 Epoch   8 Step:    71200 Batch Loss:     1.291536 Tokens per Sec:    17547, Lr: 0.000300\n",
      "2019-10-30 14:58:51,373 Epoch   8 Step:    71300 Batch Loss:     1.382409 Tokens per Sec:    17153, Lr: 0.000300\n",
      "2019-10-30 14:59:05,894 Epoch   8 Step:    71400 Batch Loss:     1.332201 Tokens per Sec:    17729, Lr: 0.000300\n",
      "2019-10-30 14:59:20,669 Epoch   8 Step:    71500 Batch Loss:     1.526174 Tokens per Sec:    17552, Lr: 0.000300\n",
      "2019-10-30 14:59:35,427 Epoch   8 Step:    71600 Batch Loss:     1.546427 Tokens per Sec:    17736, Lr: 0.000300\n",
      "2019-10-30 14:59:50,193 Epoch   8 Step:    71700 Batch Loss:     1.301171 Tokens per Sec:    17149, Lr: 0.000300\n",
      "2019-10-30 15:00:04,896 Epoch   8 Step:    71800 Batch Loss:     1.502198 Tokens per Sec:    17204, Lr: 0.000300\n",
      "2019-10-30 15:00:19,691 Epoch   8 Step:    71900 Batch Loss:     1.356700 Tokens per Sec:    17507, Lr: 0.000300\n",
      "2019-10-30 15:00:34,257 Epoch   8 Step:    72000 Batch Loss:     1.243258 Tokens per Sec:    17414, Lr: 0.000300\n",
      "2019-10-30 15:01:17,922 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:01:17,922 Saving new checkpoint.\n",
      "2019-10-30 15:01:18,060 Example #0\n",
      "2019-10-30 15:01:18,061 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:01:18,061 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:01:18,061 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:01:18,061 Example #1\n",
      "2019-10-30 15:01:18,061 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:01:18,061 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:01:18,061 \tHypothesis: Biblia haitoi sifa ya uumbaji .\n",
      "2019-10-30 15:01:18,061 Example #2\n",
      "2019-10-30 15:01:18,061 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:01:18,061 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:01:18,061 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na ni nini kinachoweza kutokea ?\n",
      "2019-10-30 15:01:18,061 Example #3\n",
      "2019-10-30 15:01:18,061 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:01:18,061 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:01:18,061 \tHypothesis: Wapendao na wabuzi .\n",
      "2019-10-30 15:01:18,062 Validation result at epoch   8, step    72000: bleu:  33.77, loss: 34570.5820, ppl:   3.5178, duration: 43.8035s\n",
      "2019-10-30 15:01:32,785 Epoch   8 Step:    72100 Batch Loss:     1.353497 Tokens per Sec:    17648, Lr: 0.000300\n",
      "2019-10-30 15:01:47,317 Epoch   8 Step:    72200 Batch Loss:     1.118407 Tokens per Sec:    17100, Lr: 0.000300\n",
      "2019-10-30 15:02:02,428 Epoch   8 Step:    72300 Batch Loss:     1.333507 Tokens per Sec:    17378, Lr: 0.000300\n",
      "2019-10-30 15:02:17,321 Epoch   8 Step:    72400 Batch Loss:     1.653298 Tokens per Sec:    17261, Lr: 0.000300\n",
      "2019-10-30 15:02:32,222 Epoch   8 Step:    72500 Batch Loss:     1.425116 Tokens per Sec:    17369, Lr: 0.000300\n",
      "2019-10-30 15:02:46,810 Epoch   8 Step:    72600 Batch Loss:     1.245059 Tokens per Sec:    17231, Lr: 0.000300\n",
      "2019-10-30 15:03:01,385 Epoch   8 Step:    72700 Batch Loss:     1.380683 Tokens per Sec:    17309, Lr: 0.000300\n",
      "2019-10-30 15:03:16,115 Epoch   8 Step:    72800 Batch Loss:     1.384424 Tokens per Sec:    17503, Lr: 0.000300\n",
      "2019-10-30 15:03:30,748 Epoch   8 Step:    72900 Batch Loss:     1.403971 Tokens per Sec:    17412, Lr: 0.000300\n",
      "2019-10-30 15:03:45,429 Epoch   8 Step:    73000 Batch Loss:     1.463516 Tokens per Sec:    17627, Lr: 0.000300\n",
      "2019-10-30 15:04:28,803 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:04:28,803 Saving new checkpoint.\n",
      "2019-10-30 15:04:28,942 Example #0\n",
      "2019-10-30 15:04:28,943 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:04:28,943 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:04:28,943 \tHypothesis: Alikuwa mja - mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:04:28,943 Example #1\n",
      "2019-10-30 15:04:28,943 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:04:28,943 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:04:28,943 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:04:28,943 Example #2\n",
      "2019-10-30 15:04:28,943 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:04:28,943 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:04:28,943 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 15:04:28,943 Example #3\n",
      "2019-10-30 15:04:28,943 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:04:28,943 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:04:28,943 \tHypothesis: Watu wanaowapenda na kuwaua .\n",
      "2019-10-30 15:04:28,943 Validation result at epoch   8, step    73000: bleu:  34.03, loss: 34559.2148, ppl:   3.5164, duration: 43.5134s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 15:04:43,564 Epoch   8 Step:    73100 Batch Loss:     1.385044 Tokens per Sec:    17419, Lr: 0.000300\n",
      "2019-10-30 15:04:58,282 Epoch   8 Step:    73200 Batch Loss:     1.471760 Tokens per Sec:    17556, Lr: 0.000300\n",
      "2019-10-30 15:05:12,942 Epoch   8 Step:    73300 Batch Loss:     1.350592 Tokens per Sec:    17504, Lr: 0.000300\n",
      "2019-10-30 15:05:27,738 Epoch   8 Step:    73400 Batch Loss:     1.355379 Tokens per Sec:    17539, Lr: 0.000300\n",
      "2019-10-30 15:05:42,571 Epoch   8 Step:    73500 Batch Loss:     1.502233 Tokens per Sec:    17554, Lr: 0.000300\n",
      "2019-10-30 15:05:57,184 Epoch   8 Step:    73600 Batch Loss:     1.277507 Tokens per Sec:    17427, Lr: 0.000300\n",
      "2019-10-30 15:06:12,006 Epoch   8 Step:    73700 Batch Loss:     1.480201 Tokens per Sec:    17380, Lr: 0.000300\n",
      "2019-10-30 15:06:26,751 Epoch   8 Step:    73800 Batch Loss:     1.599557 Tokens per Sec:    17615, Lr: 0.000300\n",
      "2019-10-30 15:06:41,699 Epoch   8 Step:    73900 Batch Loss:     1.365234 Tokens per Sec:    17385, Lr: 0.000300\n",
      "2019-10-30 15:06:56,551 Epoch   8 Step:    74000 Batch Loss:     1.303959 Tokens per Sec:    16981, Lr: 0.000300\n",
      "2019-10-30 15:07:40,200 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:07:40,200 Saving new checkpoint.\n",
      "2019-10-30 15:07:40,343 Example #0\n",
      "2019-10-30 15:07:40,343 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:07:40,343 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:07:40,343 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:07:40,343 Example #1\n",
      "2019-10-30 15:07:40,343 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:07:40,343 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:07:40,343 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:07:40,343 Example #2\n",
      "2019-10-30 15:07:40,343 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:07:40,343 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:07:40,343 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 15:07:40,343 Example #3\n",
      "2019-10-30 15:07:40,344 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:07:40,344 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:07:40,344 \tHypothesis: Wapendao na wabuzi .\n",
      "2019-10-30 15:07:40,344 Validation result at epoch   8, step    74000: bleu:  34.49, loss: 34489.0078, ppl:   3.5074, duration: 43.7920s\n",
      "2019-10-30 15:07:55,154 Epoch   8 Step:    74100 Batch Loss:     1.309569 Tokens per Sec:    17614, Lr: 0.000300\n",
      "2019-10-30 15:08:09,885 Epoch   8 Step:    74200 Batch Loss:     1.271162 Tokens per Sec:    17262, Lr: 0.000300\n",
      "2019-10-30 15:08:24,554 Epoch   8 Step:    74300 Batch Loss:     1.381868 Tokens per Sec:    17447, Lr: 0.000300\n",
      "2019-10-30 15:08:39,053 Epoch   8 Step:    74400 Batch Loss:     1.305833 Tokens per Sec:    17361, Lr: 0.000300\n",
      "2019-10-30 15:08:53,822 Epoch   8 Step:    74500 Batch Loss:     1.399412 Tokens per Sec:    17622, Lr: 0.000300\n",
      "2019-10-30 15:09:08,709 Epoch   8 Step:    74600 Batch Loss:     1.387230 Tokens per Sec:    17309, Lr: 0.000300\n",
      "2019-10-30 15:09:23,543 Epoch   8 Step:    74700 Batch Loss:     1.559688 Tokens per Sec:    17281, Lr: 0.000300\n",
      "2019-10-30 15:09:38,506 Epoch   8 Step:    74800 Batch Loss:     1.215392 Tokens per Sec:    17371, Lr: 0.000300\n",
      "2019-10-30 15:09:53,352 Epoch   8 Step:    74900 Batch Loss:     1.256550 Tokens per Sec:    17734, Lr: 0.000300\n",
      "2019-10-30 15:10:08,202 Epoch   8 Step:    75000 Batch Loss:     1.370714 Tokens per Sec:    17568, Lr: 0.000300\n",
      "2019-10-30 15:10:51,649 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:10:51,649 Saving new checkpoint.\n",
      "2019-10-30 15:10:51,786 Example #0\n",
      "2019-10-30 15:10:51,786 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:10:51,786 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:10:51,786 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:10:51,786 Example #1\n",
      "2019-10-30 15:10:51,786 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:10:51,786 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:10:51,786 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:10:51,786 Example #2\n",
      "2019-10-30 15:10:51,786 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:10:51,786 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:10:51,786 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea kwa nini ?\n",
      "2019-10-30 15:10:51,787 Example #3\n",
      "2019-10-30 15:10:51,787 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:10:51,787 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:10:51,787 \tHypothesis: Pendo la maua na mauaji .\n",
      "2019-10-30 15:10:51,787 Validation result at epoch   8, step    75000: bleu:  34.04, loss: 34189.3281, ppl:   3.4694, duration: 43.5847s\n",
      "2019-10-30 15:11:03,842 Epoch   8: total training loss 13367.55\n",
      "2019-10-30 15:11:03,842 EPOCH 9\n",
      "2019-10-30 15:11:07,806 Epoch   9 Step:    75100 Batch Loss:     1.366198 Tokens per Sec:    12843, Lr: 0.000300\n",
      "2019-10-30 15:11:22,589 Epoch   9 Step:    75200 Batch Loss:     1.838464 Tokens per Sec:    17601, Lr: 0.000300\n",
      "2019-10-30 15:11:37,220 Epoch   9 Step:    75300 Batch Loss:     1.512972 Tokens per Sec:    17624, Lr: 0.000300\n",
      "2019-10-30 15:11:52,012 Epoch   9 Step:    75400 Batch Loss:     1.165985 Tokens per Sec:    17642, Lr: 0.000300\n",
      "2019-10-30 15:12:06,695 Epoch   9 Step:    75500 Batch Loss:     1.428458 Tokens per Sec:    17547, Lr: 0.000300\n",
      "2019-10-30 15:12:21,307 Epoch   9 Step:    75600 Batch Loss:     1.487396 Tokens per Sec:    17504, Lr: 0.000300\n",
      "2019-10-30 15:12:36,152 Epoch   9 Step:    75700 Batch Loss:     1.344773 Tokens per Sec:    17509, Lr: 0.000300\n",
      "2019-10-30 15:12:51,006 Epoch   9 Step:    75800 Batch Loss:     1.403407 Tokens per Sec:    17344, Lr: 0.000300\n",
      "2019-10-30 15:13:05,895 Epoch   9 Step:    75900 Batch Loss:     1.355379 Tokens per Sec:    17381, Lr: 0.000300\n",
      "2019-10-30 15:13:20,754 Epoch   9 Step:    76000 Batch Loss:     1.397115 Tokens per Sec:    17158, Lr: 0.000300\n",
      "2019-10-30 15:14:04,617 Example #0\n",
      "2019-10-30 15:14:04,617 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:14:04,617 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:14:04,617 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:14:04,617 Example #1\n",
      "2019-10-30 15:14:04,617 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:14:04,618 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:14:04,618 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:14:04,618 Example #2\n",
      "2019-10-30 15:14:04,618 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:14:04,618 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:14:04,618 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 15:14:04,618 Example #3\n",
      "2019-10-30 15:14:04,618 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:14:04,618 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:14:04,618 \tHypothesis: Wapendao nyuki na busu .\n",
      "2019-10-30 15:14:04,618 Validation result at epoch   9, step    76000: bleu:  34.34, loss: 34224.5586, ppl:   3.4738, duration: 43.8641s\n",
      "2019-10-30 15:14:19,363 Epoch   9 Step:    76100 Batch Loss:     1.309205 Tokens per Sec:    17098, Lr: 0.000300\n",
      "2019-10-30 15:14:34,082 Epoch   9 Step:    76200 Batch Loss:     1.262030 Tokens per Sec:    17243, Lr: 0.000300\n",
      "2019-10-30 15:14:48,833 Epoch   9 Step:    76300 Batch Loss:     1.262234 Tokens per Sec:    17609, Lr: 0.000300\n",
      "2019-10-30 15:15:03,605 Epoch   9 Step:    76400 Batch Loss:     1.516769 Tokens per Sec:    17339, Lr: 0.000300\n",
      "2019-10-30 15:15:18,281 Epoch   9 Step:    76500 Batch Loss:     1.334107 Tokens per Sec:    17536, Lr: 0.000300\n",
      "2019-10-30 15:15:33,036 Epoch   9 Step:    76600 Batch Loss:     1.259781 Tokens per Sec:    17473, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 15:15:47,702 Epoch   9 Step:    76700 Batch Loss:     1.243657 Tokens per Sec:    17397, Lr: 0.000300\n",
      "2019-10-30 15:16:02,471 Epoch   9 Step:    76800 Batch Loss:     1.468775 Tokens per Sec:    17521, Lr: 0.000300\n",
      "2019-10-30 15:16:17,553 Epoch   9 Step:    76900 Batch Loss:     1.444001 Tokens per Sec:    17139, Lr: 0.000300\n",
      "2019-10-30 15:16:32,241 Epoch   9 Step:    77000 Batch Loss:     1.341457 Tokens per Sec:    17187, Lr: 0.000300\n",
      "2019-10-30 15:17:15,879 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:17:15,879 Saving new checkpoint.\n",
      "2019-10-30 15:17:16,067 Example #0\n",
      "2019-10-30 15:17:16,068 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:17:16,068 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:17:16,068 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:17:16,068 Example #1\n",
      "2019-10-30 15:17:16,068 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:17:16,068 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:17:16,068 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:17:16,068 Example #2\n",
      "2019-10-30 15:17:16,068 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:17:16,068 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:17:16,068 \tHypothesis: Ni mambo gani yanayofanya mahangaiko , na mahangaiko yanaweza kukukusaidia kufanya nini ?\n",
      "2019-10-30 15:17:16,068 Example #3\n",
      "2019-10-30 15:17:16,068 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:17:16,068 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:17:16,069 \tHypothesis: Mapenzi ya maua na mabuki .\n",
      "2019-10-30 15:17:16,069 Validation result at epoch   9, step    77000: bleu:  34.26, loss: 34033.7930, ppl:   3.4498, duration: 43.8270s\n",
      "2019-10-30 15:17:30,797 Epoch   9 Step:    77100 Batch Loss:     1.266362 Tokens per Sec:    17442, Lr: 0.000300\n",
      "2019-10-30 15:17:45,613 Epoch   9 Step:    77200 Batch Loss:     1.423281 Tokens per Sec:    17617, Lr: 0.000300\n",
      "2019-10-30 15:18:00,218 Epoch   9 Step:    77300 Batch Loss:     1.561037 Tokens per Sec:    17116, Lr: 0.000300\n",
      "2019-10-30 15:18:14,786 Epoch   9 Step:    77400 Batch Loss:     1.531538 Tokens per Sec:    17387, Lr: 0.000300\n",
      "2019-10-30 15:18:29,450 Epoch   9 Step:    77500 Batch Loss:     1.235301 Tokens per Sec:    17721, Lr: 0.000300\n",
      "2019-10-30 15:18:44,259 Epoch   9 Step:    77600 Batch Loss:     1.456343 Tokens per Sec:    17517, Lr: 0.000300\n",
      "2019-10-30 15:18:58,866 Epoch   9 Step:    77700 Batch Loss:     1.414966 Tokens per Sec:    17325, Lr: 0.000300\n",
      "2019-10-30 15:19:13,749 Epoch   9 Step:    77800 Batch Loss:     1.340452 Tokens per Sec:    17491, Lr: 0.000300\n",
      "2019-10-30 15:19:28,479 Epoch   9 Step:    77900 Batch Loss:     1.534397 Tokens per Sec:    17279, Lr: 0.000300\n",
      "2019-10-30 15:19:43,388 Epoch   9 Step:    78000 Batch Loss:     1.439590 Tokens per Sec:    17500, Lr: 0.000300\n",
      "2019-10-30 15:20:26,926 Example #0\n",
      "2019-10-30 15:20:26,926 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:20:26,926 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:20:26,926 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:20:26,926 Example #1\n",
      "2019-10-30 15:20:26,926 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:20:26,926 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:20:26,926 \tHypothesis: Biblia haiendi uthabiti .\n",
      "2019-10-30 15:20:26,926 Example #2\n",
      "2019-10-30 15:20:26,927 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:20:26,927 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:20:26,927 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 15:20:26,927 Example #3\n",
      "2019-10-30 15:20:26,927 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:20:26,927 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:20:26,927 \tHypothesis: Maonyesho ya maua na makumbusho .\n",
      "2019-10-30 15:20:26,927 Validation result at epoch   9, step    78000: bleu:  33.85, loss: 34203.1602, ppl:   3.4711, duration: 43.5384s\n",
      "2019-10-30 15:20:41,592 Epoch   9 Step:    78100 Batch Loss:     1.738191 Tokens per Sec:    17264, Lr: 0.000300\n",
      "2019-10-30 15:20:56,584 Epoch   9 Step:    78200 Batch Loss:     1.328254 Tokens per Sec:    17704, Lr: 0.000300\n",
      "2019-10-30 15:21:11,528 Epoch   9 Step:    78300 Batch Loss:     1.357551 Tokens per Sec:    17712, Lr: 0.000300\n",
      "2019-10-30 15:21:26,258 Epoch   9 Step:    78400 Batch Loss:     1.483402 Tokens per Sec:    17685, Lr: 0.000300\n",
      "2019-10-30 15:21:40,843 Epoch   9 Step:    78500 Batch Loss:     1.223796 Tokens per Sec:    17402, Lr: 0.000300\n",
      "2019-10-30 15:21:55,471 Epoch   9 Step:    78600 Batch Loss:     1.465831 Tokens per Sec:    17354, Lr: 0.000300\n",
      "2019-10-30 15:22:10,239 Epoch   9 Step:    78700 Batch Loss:     1.305147 Tokens per Sec:    17338, Lr: 0.000300\n",
      "2019-10-30 15:22:24,946 Epoch   9 Step:    78800 Batch Loss:     1.405206 Tokens per Sec:    17479, Lr: 0.000300\n",
      "2019-10-30 15:22:39,711 Epoch   9 Step:    78900 Batch Loss:     1.251132 Tokens per Sec:    17474, Lr: 0.000300\n",
      "2019-10-30 15:22:54,511 Epoch   9 Step:    79000 Batch Loss:     1.514172 Tokens per Sec:    17585, Lr: 0.000300\n",
      "2019-10-30 15:23:38,287 Example #0\n",
      "2019-10-30 15:23:38,288 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:23:38,288 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:23:38,288 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:23:38,288 Example #1\n",
      "2019-10-30 15:23:38,288 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:23:38,288 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:23:38,288 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:23:38,288 Example #2\n",
      "2019-10-30 15:23:38,288 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:23:38,288 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:23:38,288 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 15:23:38,288 Example #3\n",
      "2019-10-30 15:23:38,288 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:23:38,288 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:23:38,289 \tHypothesis: Mapenzi ya nyuki na maabusu .\n",
      "2019-10-30 15:23:38,289 Validation result at epoch   9, step    79000: bleu:  33.68, loss: 34139.7930, ppl:   3.4631, duration: 43.7770s\n",
      "2019-10-30 15:23:52,915 Epoch   9 Step:    79100 Batch Loss:     1.544671 Tokens per Sec:    17243, Lr: 0.000300\n",
      "2019-10-30 15:24:07,926 Epoch   9 Step:    79200 Batch Loss:     1.487660 Tokens per Sec:    17453, Lr: 0.000300\n",
      "2019-10-30 15:24:22,845 Epoch   9 Step:    79300 Batch Loss:     1.488370 Tokens per Sec:    17046, Lr: 0.000300\n",
      "2019-10-30 15:24:37,686 Epoch   9 Step:    79400 Batch Loss:     1.212799 Tokens per Sec:    17269, Lr: 0.000300\n",
      "2019-10-30 15:24:52,588 Epoch   9 Step:    79500 Batch Loss:     1.369951 Tokens per Sec:    16984, Lr: 0.000300\n",
      "2019-10-30 15:25:07,551 Epoch   9 Step:    79600 Batch Loss:     1.427011 Tokens per Sec:    17561, Lr: 0.000300\n",
      "2019-10-30 15:25:22,568 Epoch   9 Step:    79700 Batch Loss:     1.381654 Tokens per Sec:    17197, Lr: 0.000300\n",
      "2019-10-30 15:25:37,414 Epoch   9 Step:    79800 Batch Loss:     1.223750 Tokens per Sec:    17128, Lr: 0.000300\n",
      "2019-10-30 15:25:52,289 Epoch   9 Step:    79900 Batch Loss:     1.241816 Tokens per Sec:    17411, Lr: 0.000300\n",
      "2019-10-30 15:26:07,238 Epoch   9 Step:    80000 Batch Loss:     1.412656 Tokens per Sec:    17634, Lr: 0.000300\n",
      "2019-10-30 15:26:51,095 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:26:51,095 Saving new checkpoint.\n",
      "2019-10-30 15:26:51,242 Example #0\n",
      "2019-10-30 15:26:51,242 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:26:51,242 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:26:51,242 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:26:51,242 Example #1\n",
      "2019-10-30 15:26:51,242 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:26:51,242 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:26:51,242 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:26:51,243 Example #2\n",
      "2019-10-30 15:26:51,243 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:26:51,243 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:26:51,243 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 15:26:51,243 Example #3\n",
      "2019-10-30 15:26:51,243 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:26:51,243 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:26:51,243 \tHypothesis: Mapenzi ya nyuki na mabusu .\n",
      "2019-10-30 15:26:51,243 Validation result at epoch   9, step    80000: bleu:  34.26, loss: 33816.4062, ppl:   3.4226, duration: 44.0048s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 15:27:05,924 Epoch   9 Step:    80100 Batch Loss:     1.616957 Tokens per Sec:    17311, Lr: 0.000300\n",
      "2019-10-30 15:27:20,773 Epoch   9 Step:    80200 Batch Loss:     1.391051 Tokens per Sec:    17689, Lr: 0.000300\n",
      "2019-10-30 15:27:35,600 Epoch   9 Step:    80300 Batch Loss:     1.276008 Tokens per Sec:    17222, Lr: 0.000300\n",
      "2019-10-30 15:27:50,238 Epoch   9 Step:    80400 Batch Loss:     1.546536 Tokens per Sec:    17427, Lr: 0.000300\n",
      "2019-10-30 15:28:05,206 Epoch   9 Step:    80500 Batch Loss:     1.433902 Tokens per Sec:    17178, Lr: 0.000300\n",
      "2019-10-30 15:28:20,307 Epoch   9 Step:    80600 Batch Loss:     1.335156 Tokens per Sec:    17146, Lr: 0.000300\n",
      "2019-10-30 15:28:35,059 Epoch   9 Step:    80700 Batch Loss:     1.171184 Tokens per Sec:    17646, Lr: 0.000300\n",
      "2019-10-30 15:28:49,801 Epoch   9 Step:    80800 Batch Loss:     1.348810 Tokens per Sec:    17586, Lr: 0.000300\n",
      "2019-10-30 15:29:04,559 Epoch   9 Step:    80900 Batch Loss:     1.240198 Tokens per Sec:    17425, Lr: 0.000300\n",
      "2019-10-30 15:29:19,470 Epoch   9 Step:    81000 Batch Loss:     1.329023 Tokens per Sec:    17529, Lr: 0.000300\n",
      "2019-10-30 15:30:03,368 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:30:03,369 Saving new checkpoint.\n",
      "2019-10-30 15:30:03,513 Example #0\n",
      "2019-10-30 15:30:03,514 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:30:03,514 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:30:03,514 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:30:03,514 Example #1\n",
      "2019-10-30 15:30:03,514 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:30:03,514 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:30:03,514 \tHypothesis: Biblia haisemi uthabiti .\n",
      "2019-10-30 15:30:03,514 Example #2\n",
      "2019-10-30 15:30:03,514 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:30:03,514 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:30:03,514 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukufanya nini ?\n",
      "2019-10-30 15:30:03,514 Example #3\n",
      "2019-10-30 15:30:03,514 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:30:03,514 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:30:03,514 \tHypothesis: Wapenzi wa nyuki na makaburi .\n",
      "2019-10-30 15:30:03,514 Validation result at epoch   9, step    81000: bleu:  34.40, loss: 33810.9492, ppl:   3.4219, duration: 44.0440s\n",
      "2019-10-30 15:30:18,444 Epoch   9 Step:    81100 Batch Loss:     1.264979 Tokens per Sec:    17414, Lr: 0.000300\n",
      "2019-10-30 15:30:33,317 Epoch   9 Step:    81200 Batch Loss:     1.674268 Tokens per Sec:    17419, Lr: 0.000300\n",
      "2019-10-30 15:30:48,311 Epoch   9 Step:    81300 Batch Loss:     1.521849 Tokens per Sec:    17596, Lr: 0.000300\n",
      "2019-10-30 15:31:03,111 Epoch   9 Step:    81400 Batch Loss:     1.447769 Tokens per Sec:    17183, Lr: 0.000300\n",
      "2019-10-30 15:31:18,053 Epoch   9 Step:    81500 Batch Loss:     1.315326 Tokens per Sec:    17327, Lr: 0.000300\n",
      "2019-10-30 15:31:32,923 Epoch   9 Step:    81600 Batch Loss:     1.225018 Tokens per Sec:    17186, Lr: 0.000300\n",
      "2019-10-30 15:31:47,773 Epoch   9 Step:    81700 Batch Loss:     1.295579 Tokens per Sec:    17513, Lr: 0.000300\n",
      "2019-10-30 15:32:02,485 Epoch   9 Step:    81800 Batch Loss:     1.173829 Tokens per Sec:    17221, Lr: 0.000300\n",
      "2019-10-30 15:32:17,315 Epoch   9 Step:    81900 Batch Loss:     1.490525 Tokens per Sec:    16925, Lr: 0.000300\n",
      "2019-10-30 15:32:32,223 Epoch   9 Step:    82000 Batch Loss:     1.281635 Tokens per Sec:    17266, Lr: 0.000300\n",
      "2019-10-30 15:33:15,759 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:33:15,760 Saving new checkpoint.\n",
      "2019-10-30 15:33:15,898 Example #0\n",
      "2019-10-30 15:33:15,898 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:33:15,899 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:33:15,899 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:33:15,899 Example #1\n",
      "2019-10-30 15:33:15,899 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:33:15,899 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:33:15,899 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:33:15,899 Example #2\n",
      "2019-10-30 15:33:15,899 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:33:15,899 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:33:15,899 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 15:33:15,899 Example #3\n",
      "2019-10-30 15:33:15,899 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:33:15,899 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:33:15,899 \tHypothesis: Miguu ya makaburi na mabusu .\n",
      "2019-10-30 15:33:15,899 Validation result at epoch   9, step    82000: bleu:  34.04, loss: 33773.2578, ppl:   3.4172, duration: 43.6754s\n",
      "2019-10-30 15:33:30,609 Epoch   9 Step:    82100 Batch Loss:     1.360771 Tokens per Sec:    17110, Lr: 0.000300\n",
      "2019-10-30 15:33:45,425 Epoch   9 Step:    82200 Batch Loss:     1.325429 Tokens per Sec:    17685, Lr: 0.000300\n",
      "2019-10-30 15:34:00,115 Epoch   9 Step:    82300 Batch Loss:     1.420725 Tokens per Sec:    17157, Lr: 0.000300\n",
      "2019-10-30 15:34:14,986 Epoch   9 Step:    82400 Batch Loss:     1.376292 Tokens per Sec:    17674, Lr: 0.000300\n",
      "2019-10-30 15:34:29,838 Epoch   9 Step:    82500 Batch Loss:     1.354248 Tokens per Sec:    17416, Lr: 0.000300\n",
      "2019-10-30 15:34:44,775 Epoch   9 Step:    82600 Batch Loss:     1.283264 Tokens per Sec:    16909, Lr: 0.000300\n",
      "2019-10-30 15:34:59,936 Epoch   9 Step:    82700 Batch Loss:     1.304029 Tokens per Sec:    17284, Lr: 0.000300\n",
      "2019-10-30 15:35:14,706 Epoch   9 Step:    82800 Batch Loss:     1.389183 Tokens per Sec:    17039, Lr: 0.000300\n",
      "2019-10-30 15:35:29,485 Epoch   9 Step:    82900 Batch Loss:     1.417917 Tokens per Sec:    17045, Lr: 0.000300\n",
      "2019-10-30 15:35:44,304 Epoch   9 Step:    83000 Batch Loss:     1.189076 Tokens per Sec:    17485, Lr: 0.000300\n",
      "2019-10-30 15:36:28,292 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:36:28,292 Saving new checkpoint.\n",
      "2019-10-30 15:36:28,437 Example #0\n",
      "2019-10-30 15:36:28,437 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:36:28,437 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:36:28,437 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:36:28,437 Example #1\n",
      "2019-10-30 15:36:28,438 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:36:28,438 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:36:28,438 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:36:28,438 Example #2\n",
      "2019-10-30 15:36:28,438 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:36:28,438 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:36:28,438 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea nini ?\n",
      "2019-10-30 15:36:28,438 Example #3\n",
      "2019-10-30 15:36:28,438 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:36:28,438 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:36:28,438 \tHypothesis: Miti ya makaburi na mabusu .\n",
      "2019-10-30 15:36:28,438 Validation result at epoch   9, step    83000: bleu:  34.25, loss: 33684.8789, ppl:   3.4063, duration: 44.1339s\n",
      "2019-10-30 15:36:43,201 Epoch   9 Step:    83100 Batch Loss:     1.344886 Tokens per Sec:    17057, Lr: 0.000300\n",
      "2019-10-30 15:36:58,023 Epoch   9 Step:    83200 Batch Loss:     1.302795 Tokens per Sec:    17373, Lr: 0.000300\n",
      "2019-10-30 15:37:12,906 Epoch   9 Step:    83300 Batch Loss:     1.679052 Tokens per Sec:    17368, Lr: 0.000300\n",
      "2019-10-30 15:37:27,578 Epoch   9 Step:    83400 Batch Loss:     1.235408 Tokens per Sec:    17186, Lr: 0.000300\n",
      "2019-10-30 15:37:42,204 Epoch   9 Step:    83500 Batch Loss:     1.496633 Tokens per Sec:    17662, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 15:37:56,901 Epoch   9 Step:    83600 Batch Loss:     1.538069 Tokens per Sec:    17366, Lr: 0.000300\n",
      "2019-10-30 15:38:11,821 Epoch   9 Step:    83700 Batch Loss:     1.277231 Tokens per Sec:    17086, Lr: 0.000300\n",
      "2019-10-30 15:38:26,587 Epoch   9 Step:    83800 Batch Loss:     1.198104 Tokens per Sec:    17132, Lr: 0.000300\n",
      "2019-10-30 15:38:41,328 Epoch   9 Step:    83900 Batch Loss:     1.403955 Tokens per Sec:    17141, Lr: 0.000300\n",
      "2019-10-30 15:38:56,025 Epoch   9 Step:    84000 Batch Loss:     1.355099 Tokens per Sec:    17115, Lr: 0.000300\n",
      "2019-10-30 15:39:39,625 Example #0\n",
      "2019-10-30 15:39:39,625 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:39:39,625 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:39:39,625 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:39:39,625 Example #1\n",
      "2019-10-30 15:39:39,625 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:39:39,625 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:39:39,625 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:39:39,626 Example #2\n",
      "2019-10-30 15:39:39,626 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:39:39,626 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:39:39,626 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 15:39:39,626 Example #3\n",
      "2019-10-30 15:39:39,626 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:39:39,626 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:39:39,626 \tHypothesis: Miti ya maua na busu .\n",
      "2019-10-30 15:39:39,626 Validation result at epoch   9, step    84000: bleu:  34.57, loss: 33735.3750, ppl:   3.4125, duration: 43.6003s\n",
      "2019-10-30 15:39:54,527 Epoch   9 Step:    84100 Batch Loss:     1.246420 Tokens per Sec:    17624, Lr: 0.000300\n",
      "2019-10-30 15:40:09,254 Epoch   9 Step:    84200 Batch Loss:     1.136039 Tokens per Sec:    17021, Lr: 0.000300\n",
      "2019-10-30 15:40:24,086 Epoch   9 Step:    84300 Batch Loss:     1.378651 Tokens per Sec:    17405, Lr: 0.000300\n",
      "2019-10-30 15:40:38,915 Epoch   9 Step:    84400 Batch Loss:     1.538381 Tokens per Sec:    17366, Lr: 0.000300\n",
      "2019-10-30 15:40:47,854 Epoch   9: total training loss 13100.25\n",
      "2019-10-30 15:40:47,854 EPOCH 10\n",
      "2019-10-30 15:40:55,038 Epoch  10 Step:    84500 Batch Loss:     1.325894 Tokens per Sec:    14733, Lr: 0.000300\n",
      "2019-10-30 15:41:09,839 Epoch  10 Step:    84600 Batch Loss:     1.405308 Tokens per Sec:    17170, Lr: 0.000300\n",
      "2019-10-30 15:41:24,671 Epoch  10 Step:    84700 Batch Loss:     1.390368 Tokens per Sec:    17754, Lr: 0.000300\n",
      "2019-10-30 15:41:39,414 Epoch  10 Step:    84800 Batch Loss:     1.502212 Tokens per Sec:    17469, Lr: 0.000300\n",
      "2019-10-30 15:41:54,053 Epoch  10 Step:    84900 Batch Loss:     1.263424 Tokens per Sec:    16990, Lr: 0.000300\n",
      "2019-10-30 15:42:09,040 Epoch  10 Step:    85000 Batch Loss:     1.638967 Tokens per Sec:    17717, Lr: 0.000300\n",
      "2019-10-30 15:42:53,550 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:42:53,550 Saving new checkpoint.\n",
      "2019-10-30 15:42:53,745 Example #0\n",
      "2019-10-30 15:42:53,745 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:42:53,745 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:42:53,745 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:42:53,745 Example #1\n",
      "2019-10-30 15:42:53,746 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:42:53,746 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:42:53,746 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:42:53,746 Example #2\n",
      "2019-10-30 15:42:53,746 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:42:53,746 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:42:53,746 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 15:42:53,746 Example #3\n",
      "2019-10-30 15:42:53,746 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:42:53,746 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:42:53,746 \tHypothesis: Watu wanaotenda kwa makaburi na mabusu .\n",
      "2019-10-30 15:42:53,746 Validation result at epoch  10, step    85000: bleu:  34.66, loss: 33511.3164, ppl:   3.3848, duration: 44.7054s\n",
      "2019-10-30 15:43:08,881 Epoch  10 Step:    85100 Batch Loss:     1.283467 Tokens per Sec:    17281, Lr: 0.000300\n",
      "2019-10-30 15:43:23,746 Epoch  10 Step:    85200 Batch Loss:     1.376308 Tokens per Sec:    17270, Lr: 0.000300\n",
      "2019-10-30 15:43:38,445 Epoch  10 Step:    85300 Batch Loss:     1.203360 Tokens per Sec:    17235, Lr: 0.000300\n",
      "2019-10-30 15:43:53,693 Epoch  10 Step:    85400 Batch Loss:     1.457374 Tokens per Sec:    16969, Lr: 0.000300\n",
      "2019-10-30 15:44:08,671 Epoch  10 Step:    85500 Batch Loss:     1.203356 Tokens per Sec:    16934, Lr: 0.000300\n",
      "2019-10-30 15:44:23,430 Epoch  10 Step:    85600 Batch Loss:     1.622142 Tokens per Sec:    17209, Lr: 0.000300\n",
      "2019-10-30 15:44:38,230 Epoch  10 Step:    85700 Batch Loss:     1.231284 Tokens per Sec:    17207, Lr: 0.000300\n",
      "2019-10-30 15:44:52,984 Epoch  10 Step:    85800 Batch Loss:     1.326645 Tokens per Sec:    17585, Lr: 0.000300\n",
      "2019-10-30 15:45:07,883 Epoch  10 Step:    85900 Batch Loss:     1.286811 Tokens per Sec:    17305, Lr: 0.000300\n",
      "2019-10-30 15:45:22,762 Epoch  10 Step:    86000 Batch Loss:     1.495724 Tokens per Sec:    17081, Lr: 0.000300\n",
      "2019-10-30 15:46:06,634 Example #0\n",
      "2019-10-30 15:46:06,634 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:46:06,634 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:46:06,634 \tHypothesis: Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:46:06,634 Example #1\n",
      "2019-10-30 15:46:06,634 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:46:06,634 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:46:06,635 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:46:06,635 Example #2\n",
      "2019-10-30 15:46:06,635 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:46:06,635 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:46:06,635 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 15:46:06,635 Example #3\n",
      "2019-10-30 15:46:06,635 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:46:06,635 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:46:06,635 \tHypothesis: Wapendao hula na kubusu .\n",
      "2019-10-30 15:46:06,635 Validation result at epoch  10, step    86000: bleu:  34.34, loss: 33704.4141, ppl:   3.4087, duration: 43.8723s\n",
      "2019-10-30 15:46:21,583 Epoch  10 Step:    86100 Batch Loss:     1.312888 Tokens per Sec:    17200, Lr: 0.000300\n",
      "2019-10-30 15:46:36,370 Epoch  10 Step:    86200 Batch Loss:     1.555307 Tokens per Sec:    17648, Lr: 0.000300\n",
      "2019-10-30 15:46:51,152 Epoch  10 Step:    86300 Batch Loss:     1.556358 Tokens per Sec:    17486, Lr: 0.000300\n",
      "2019-10-30 15:47:06,044 Epoch  10 Step:    86400 Batch Loss:     1.288957 Tokens per Sec:    17248, Lr: 0.000300\n",
      "2019-10-30 15:47:20,820 Epoch  10 Step:    86500 Batch Loss:     1.303393 Tokens per Sec:    17316, Lr: 0.000300\n",
      "2019-10-30 15:47:35,691 Epoch  10 Step:    86600 Batch Loss:     1.428131 Tokens per Sec:    17571, Lr: 0.000300\n",
      "2019-10-30 15:47:50,637 Epoch  10 Step:    86700 Batch Loss:     1.340597 Tokens per Sec:    17314, Lr: 0.000300\n",
      "2019-10-30 15:48:05,517 Epoch  10 Step:    86800 Batch Loss:     1.554366 Tokens per Sec:    17100, Lr: 0.000300\n",
      "2019-10-30 15:48:20,614 Epoch  10 Step:    86900 Batch Loss:     1.368237 Tokens per Sec:    17379, Lr: 0.000300\n",
      "2019-10-30 15:48:35,323 Epoch  10 Step:    87000 Batch Loss:     1.334750 Tokens per Sec:    17286, Lr: 0.000300\n",
      "2019-10-30 15:49:19,374 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:49:19,374 Saving new checkpoint.\n",
      "2019-10-30 15:49:19,519 Example #0\n",
      "2019-10-30 15:49:19,520 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:49:19,520 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:49:19,520 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:49:19,520 Example #1\n",
      "2019-10-30 15:49:19,520 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:49:19,520 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:49:19,520 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:49:19,520 Example #2\n",
      "2019-10-30 15:49:19,520 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:49:19,520 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:49:19,520 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokea ?\n",
      "2019-10-30 15:49:19,520 Example #3\n",
      "2019-10-30 15:49:19,521 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:49:19,521 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:49:19,521 \tHypothesis: Miguu ya nyasi na mabusu .\n",
      "2019-10-30 15:49:19,521 Validation result at epoch  10, step    87000: bleu:  34.54, loss: 33461.0742, ppl:   3.3786, duration: 44.1970s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 15:49:34,472 Epoch  10 Step:    87100 Batch Loss:     1.311441 Tokens per Sec:    16795, Lr: 0.000300\n",
      "2019-10-30 15:49:49,156 Epoch  10 Step:    87200 Batch Loss:     1.348119 Tokens per Sec:    17144, Lr: 0.000300\n",
      "2019-10-30 15:50:03,822 Epoch  10 Step:    87300 Batch Loss:     1.407295 Tokens per Sec:    17419, Lr: 0.000300\n",
      "2019-10-30 15:50:18,688 Epoch  10 Step:    87400 Batch Loss:     1.247401 Tokens per Sec:    17432, Lr: 0.000300\n",
      "2019-10-30 15:50:33,630 Epoch  10 Step:    87500 Batch Loss:     1.329310 Tokens per Sec:    17230, Lr: 0.000300\n",
      "2019-10-30 15:50:48,721 Epoch  10 Step:    87600 Batch Loss:     1.299616 Tokens per Sec:    17466, Lr: 0.000300\n",
      "2019-10-30 15:51:03,652 Epoch  10 Step:    87700 Batch Loss:     1.417706 Tokens per Sec:    17137, Lr: 0.000300\n",
      "2019-10-30 15:51:18,752 Epoch  10 Step:    87800 Batch Loss:     1.199353 Tokens per Sec:    17253, Lr: 0.000300\n",
      "2019-10-30 15:51:33,592 Epoch  10 Step:    87900 Batch Loss:     1.366296 Tokens per Sec:    16981, Lr: 0.000300\n",
      "2019-10-30 15:51:48,330 Epoch  10 Step:    88000 Batch Loss:     1.524254 Tokens per Sec:    17271, Lr: 0.000300\n",
      "2019-10-30 15:52:32,126 Example #0\n",
      "2019-10-30 15:52:32,127 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:52:32,127 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:52:32,127 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:52:32,127 Example #1\n",
      "2019-10-30 15:52:32,127 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:52:32,127 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:52:32,127 \tHypothesis: Biblia haidhuru utukufu .\n",
      "2019-10-30 15:52:32,127 Example #2\n",
      "2019-10-30 15:52:32,127 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:52:32,127 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:52:32,127 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea kwa njia gani ?\n",
      "2019-10-30 15:52:32,127 Example #3\n",
      "2019-10-30 15:52:32,127 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:52:32,127 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:52:32,127 \tHypothesis: Wapendao hukumbatia na kubusu .\n",
      "2019-10-30 15:52:32,128 Validation result at epoch  10, step    88000: bleu:  34.84, loss: 33560.4375, ppl:   3.3909, duration: 43.7970s\n",
      "2019-10-30 15:52:46,976 Epoch  10 Step:    88100 Batch Loss:     1.283007 Tokens per Sec:    17452, Lr: 0.000300\n",
      "2019-10-30 15:53:01,683 Epoch  10 Step:    88200 Batch Loss:     1.261532 Tokens per Sec:    17179, Lr: 0.000300\n",
      "2019-10-30 15:53:16,652 Epoch  10 Step:    88300 Batch Loss:     1.418218 Tokens per Sec:    17338, Lr: 0.000300\n",
      "2019-10-30 15:53:31,548 Epoch  10 Step:    88400 Batch Loss:     1.295044 Tokens per Sec:    17335, Lr: 0.000300\n",
      "2019-10-30 15:53:46,612 Epoch  10 Step:    88500 Batch Loss:     1.365996 Tokens per Sec:    16932, Lr: 0.000300\n",
      "2019-10-30 15:54:01,551 Epoch  10 Step:    88600 Batch Loss:     1.390305 Tokens per Sec:    17403, Lr: 0.000300\n",
      "2019-10-30 15:54:16,346 Epoch  10 Step:    88700 Batch Loss:     1.448873 Tokens per Sec:    17358, Lr: 0.000300\n",
      "2019-10-30 15:54:31,381 Epoch  10 Step:    88800 Batch Loss:     1.215499 Tokens per Sec:    17421, Lr: 0.000300\n",
      "2019-10-30 15:54:46,394 Epoch  10 Step:    88900 Batch Loss:     1.203713 Tokens per Sec:    17386, Lr: 0.000300\n",
      "2019-10-30 15:55:01,238 Epoch  10 Step:    89000 Batch Loss:     1.368253 Tokens per Sec:    17047, Lr: 0.000300\n",
      "2019-10-30 15:55:45,151 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:55:45,151 Saving new checkpoint.\n",
      "2019-10-30 15:55:45,293 Example #0\n",
      "2019-10-30 15:55:45,293 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:55:45,293 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:55:45,293 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:55:45,293 Example #1\n",
      "2019-10-30 15:55:45,293 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:55:45,293 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:55:45,293 \tHypothesis: Biblia haisemi uthabiti .\n",
      "2019-10-30 15:55:45,293 Example #2\n",
      "2019-10-30 15:55:45,294 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:55:45,294 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:55:45,294 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 15:55:45,294 Example #3\n",
      "2019-10-30 15:55:45,294 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:55:45,294 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:55:45,294 \tHypothesis: Mabomu ya maua na makaburi .\n",
      "2019-10-30 15:55:45,294 Validation result at epoch  10, step    89000: bleu:  34.73, loss: 33420.6289, ppl:   3.3737, duration: 44.0557s\n",
      "2019-10-30 15:56:00,181 Epoch  10 Step:    89100 Batch Loss:     1.369700 Tokens per Sec:    17502, Lr: 0.000300\n",
      "2019-10-30 15:56:14,878 Epoch  10 Step:    89200 Batch Loss:     1.503870 Tokens per Sec:    16927, Lr: 0.000300\n",
      "2019-10-30 15:56:29,728 Epoch  10 Step:    89300 Batch Loss:     1.291116 Tokens per Sec:    17252, Lr: 0.000300\n",
      "2019-10-30 15:56:44,570 Epoch  10 Step:    89400 Batch Loss:     1.299114 Tokens per Sec:    17438, Lr: 0.000300\n",
      "2019-10-30 15:56:59,260 Epoch  10 Step:    89500 Batch Loss:     1.302313 Tokens per Sec:    17236, Lr: 0.000300\n",
      "2019-10-30 15:57:13,957 Epoch  10 Step:    89600 Batch Loss:     1.285881 Tokens per Sec:    17340, Lr: 0.000300\n",
      "2019-10-30 15:57:28,776 Epoch  10 Step:    89700 Batch Loss:     1.390226 Tokens per Sec:    17161, Lr: 0.000300\n",
      "2019-10-30 15:57:43,624 Epoch  10 Step:    89800 Batch Loss:     1.340035 Tokens per Sec:    17268, Lr: 0.000300\n",
      "2019-10-30 15:57:58,693 Epoch  10 Step:    89900 Batch Loss:     1.387982 Tokens per Sec:    17417, Lr: 0.000300\n",
      "2019-10-30 15:58:13,580 Epoch  10 Step:    90000 Batch Loss:     1.198132 Tokens per Sec:    17052, Lr: 0.000300\n",
      "2019-10-30 15:58:57,287 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 15:58:57,288 Saving new checkpoint.\n",
      "2019-10-30 15:58:57,427 Example #0\n",
      "2019-10-30 15:58:57,427 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 15:58:57,427 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 15:58:57,427 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 15:58:57,427 Example #1\n",
      "2019-10-30 15:58:57,427 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 15:58:57,427 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 15:58:57,427 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 15:58:57,427 Example #2\n",
      "2019-10-30 15:58:57,428 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 15:58:57,428 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 15:58:57,428 \tHypothesis: Ni mambo gani yanayofanya hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 15:58:57,428 Example #3\n",
      "2019-10-30 15:58:57,428 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 15:58:57,428 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 15:58:57,428 \tHypothesis: Miguu ya makaburi na mabusu .\n",
      "2019-10-30 15:58:57,428 Validation result at epoch  10, step    90000: bleu:  34.03, loss: 33172.3555, ppl:   3.3433, duration: 43.8476s\n",
      "2019-10-30 15:59:12,426 Epoch  10 Step:    90100 Batch Loss:     1.409385 Tokens per Sec:    17094, Lr: 0.000300\n",
      "2019-10-30 15:59:27,296 Epoch  10 Step:    90200 Batch Loss:     1.388977 Tokens per Sec:    17565, Lr: 0.000300\n",
      "2019-10-30 15:59:42,069 Epoch  10 Step:    90300 Batch Loss:     1.214117 Tokens per Sec:    17322, Lr: 0.000300\n",
      "2019-10-30 15:59:56,884 Epoch  10 Step:    90400 Batch Loss:     1.295104 Tokens per Sec:    17199, Lr: 0.000300\n",
      "2019-10-30 16:00:11,660 Epoch  10 Step:    90500 Batch Loss:     1.108548 Tokens per Sec:    17302, Lr: 0.000300\n",
      "2019-10-30 16:00:26,510 Epoch  10 Step:    90600 Batch Loss:     1.151913 Tokens per Sec:    17247, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 16:00:41,392 Epoch  10 Step:    90700 Batch Loss:     1.402385 Tokens per Sec:    17346, Lr: 0.000300\n",
      "2019-10-30 16:00:56,102 Epoch  10 Step:    90800 Batch Loss:     1.305445 Tokens per Sec:    17277, Lr: 0.000300\n",
      "2019-10-30 16:01:11,301 Epoch  10 Step:    90900 Batch Loss:     1.449667 Tokens per Sec:    17324, Lr: 0.000300\n",
      "2019-10-30 16:01:26,149 Epoch  10 Step:    91000 Batch Loss:     1.565037 Tokens per Sec:    17171, Lr: 0.000300\n",
      "2019-10-30 16:02:09,925 Example #0\n",
      "2019-10-30 16:02:09,925 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:02:09,925 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:02:09,925 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:02:09,925 Example #1\n",
      "2019-10-30 16:02:09,926 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:02:09,926 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:02:09,926 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 16:02:09,926 Example #2\n",
      "2019-10-30 16:02:09,926 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:02:09,926 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:02:09,926 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na ni nini kinachoweza kutokeza hangaiko ?\n",
      "2019-10-30 16:02:09,926 Example #3\n",
      "2019-10-30 16:02:09,926 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:02:09,926 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:02:09,926 \tHypothesis: Miguu ya maua na mabusu .\n",
      "2019-10-30 16:02:09,926 Validation result at epoch  10, step    91000: bleu:  34.32, loss: 33278.9062, ppl:   3.3563, duration: 43.7770s\n",
      "2019-10-30 16:02:24,820 Epoch  10 Step:    91100 Batch Loss:     1.191745 Tokens per Sec:    17285, Lr: 0.000300\n",
      "2019-10-30 16:02:39,537 Epoch  10 Step:    91200 Batch Loss:     1.221433 Tokens per Sec:    17310, Lr: 0.000300\n",
      "2019-10-30 16:02:54,434 Epoch  10 Step:    91300 Batch Loss:     1.235889 Tokens per Sec:    17278, Lr: 0.000300\n",
      "2019-10-30 16:03:09,252 Epoch  10 Step:    91400 Batch Loss:     1.357983 Tokens per Sec:    17212, Lr: 0.000300\n",
      "2019-10-30 16:03:24,085 Epoch  10 Step:    91500 Batch Loss:     1.307003 Tokens per Sec:    17364, Lr: 0.000300\n",
      "2019-10-30 16:03:38,800 Epoch  10 Step:    91600 Batch Loss:     1.241288 Tokens per Sec:    17298, Lr: 0.000300\n",
      "2019-10-30 16:03:53,504 Epoch  10 Step:    91700 Batch Loss:     1.547439 Tokens per Sec:    17267, Lr: 0.000300\n",
      "2019-10-30 16:04:08,360 Epoch  10 Step:    91800 Batch Loss:     1.469598 Tokens per Sec:    17366, Lr: 0.000300\n",
      "2019-10-30 16:04:23,297 Epoch  10 Step:    91900 Batch Loss:     1.386333 Tokens per Sec:    17458, Lr: 0.000300\n",
      "2019-10-30 16:04:38,357 Epoch  10 Step:    92000 Batch Loss:     1.403825 Tokens per Sec:    17212, Lr: 0.000300\n",
      "2019-10-30 16:05:22,284 Example #0\n",
      "2019-10-30 16:05:22,285 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:05:22,285 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:05:22,285 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:05:22,285 Example #1\n",
      "2019-10-30 16:05:22,285 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:05:22,285 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:05:22,285 \tHypothesis: Biblia haidhuru utukufu .\n",
      "2019-10-30 16:05:22,285 Example #2\n",
      "2019-10-30 16:05:22,285 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:05:22,285 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:05:22,285 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 16:05:22,285 Example #3\n",
      "2019-10-30 16:05:22,285 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:05:22,285 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:05:22,285 \tHypothesis: Wapenzi wa makaburi na mabusu .\n",
      "2019-10-30 16:05:22,285 Validation result at epoch  10, step    92000: bleu:  35.02, loss: 33262.9609, ppl:   3.3544, duration: 43.9282s\n",
      "2019-10-30 16:05:37,245 Epoch  10 Step:    92100 Batch Loss:     1.491842 Tokens per Sec:    17072, Lr: 0.000300\n",
      "2019-10-30 16:05:52,301 Epoch  10 Step:    92200 Batch Loss:     1.370757 Tokens per Sec:    17078, Lr: 0.000300\n",
      "2019-10-30 16:06:07,248 Epoch  10 Step:    92300 Batch Loss:     1.371876 Tokens per Sec:    17161, Lr: 0.000300\n",
      "2019-10-30 16:06:22,020 Epoch  10 Step:    92400 Batch Loss:     1.431860 Tokens per Sec:    17495, Lr: 0.000300\n",
      "2019-10-30 16:06:36,742 Epoch  10 Step:    92500 Batch Loss:     1.302598 Tokens per Sec:    17499, Lr: 0.000300\n",
      "2019-10-30 16:06:51,578 Epoch  10 Step:    92600 Batch Loss:     1.314567 Tokens per Sec:    17337, Lr: 0.000300\n",
      "2019-10-30 16:07:06,445 Epoch  10 Step:    92700 Batch Loss:     1.328373 Tokens per Sec:    17134, Lr: 0.000300\n",
      "2019-10-30 16:07:21,659 Epoch  10 Step:    92800 Batch Loss:     1.343456 Tokens per Sec:    17394, Lr: 0.000300\n",
      "2019-10-30 16:07:36,634 Epoch  10 Step:    92900 Batch Loss:     1.714190 Tokens per Sec:    17037, Lr: 0.000300\n",
      "2019-10-30 16:07:51,743 Epoch  10 Step:    93000 Batch Loss:     1.475460 Tokens per Sec:    17156, Lr: 0.000300\n",
      "2019-10-30 16:08:35,816 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:08:35,816 Saving new checkpoint.\n",
      "2019-10-30 16:08:35,963 Example #0\n",
      "2019-10-30 16:08:35,963 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:08:35,963 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:08:35,963 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:08:35,963 Example #1\n",
      "2019-10-30 16:08:35,963 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:08:35,963 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:08:35,963 \tHypothesis: Biblia haidhuru sifa ya urembo .\n",
      "2019-10-30 16:08:35,963 Example #2\n",
      "2019-10-30 16:08:35,963 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:08:35,963 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:08:35,963 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:08:35,964 Example #3\n",
      "2019-10-30 16:08:35,964 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:08:35,964 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:08:35,964 \tHypothesis: Miguu ya makaburi na mabusu .\n",
      "2019-10-30 16:08:35,964 Validation result at epoch  10, step    93000: bleu:  34.94, loss: 33005.4844, ppl:   3.3231, duration: 44.2202s\n",
      "2019-10-30 16:08:50,994 Epoch  10 Step:    93100 Batch Loss:     1.383578 Tokens per Sec:    17313, Lr: 0.000300\n",
      "2019-10-30 16:09:05,927 Epoch  10 Step:    93200 Batch Loss:     1.380709 Tokens per Sec:    17052, Lr: 0.000300\n",
      "2019-10-30 16:09:20,860 Epoch  10 Step:    93300 Batch Loss:     1.350309 Tokens per Sec:    17095, Lr: 0.000300\n",
      "2019-10-30 16:09:35,819 Epoch  10 Step:    93400 Batch Loss:     1.416344 Tokens per Sec:    17321, Lr: 0.000300\n",
      "2019-10-30 16:09:50,628 Epoch  10 Step:    93500 Batch Loss:     1.323323 Tokens per Sec:    17361, Lr: 0.000300\n",
      "2019-10-30 16:10:05,634 Epoch  10 Step:    93600 Batch Loss:     1.313738 Tokens per Sec:    17343, Lr: 0.000300\n",
      "2019-10-30 16:10:20,720 Epoch  10 Step:    93700 Batch Loss:     1.493611 Tokens per Sec:    17606, Lr: 0.000300\n",
      "2019-10-30 16:10:35,635 Epoch  10 Step:    93800 Batch Loss:     1.229618 Tokens per Sec:    17118, Lr: 0.000300\n",
      "2019-10-30 16:10:41,104 Epoch  10: total training loss 12853.83\n",
      "2019-10-30 16:10:41,104 EPOCH 11\n",
      "2019-10-30 16:10:51,541 Epoch  11 Step:    93900 Batch Loss:     1.641189 Tokens per Sec:    15131, Lr: 0.000300\n",
      "2019-10-30 16:11:06,232 Epoch  11 Step:    94000 Batch Loss:     1.421051 Tokens per Sec:    17113, Lr: 0.000300\n",
      "2019-10-30 16:11:50,005 Example #0\n",
      "2019-10-30 16:11:50,005 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:11:50,005 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:11:50,005 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:11:50,005 Example #1\n",
      "2019-10-30 16:11:50,006 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:11:50,006 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:11:50,006 \tHypothesis: Biblia haidhuru utukufu .\n",
      "2019-10-30 16:11:50,006 Example #2\n",
      "2019-10-30 16:11:50,006 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:11:50,006 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:11:50,006 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 16:11:50,006 Example #3\n",
      "2019-10-30 16:11:50,006 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:11:50,006 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:11:50,006 \tHypothesis: Wanyama wa makaburi na mabusu .\n",
      "2019-10-30 16:11:50,006 Validation result at epoch  11, step    94000: bleu:  34.91, loss: 33049.3984, ppl:   3.3284, duration: 43.7741s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 16:12:04,885 Epoch  11 Step:    94100 Batch Loss:     1.191529 Tokens per Sec:    17603, Lr: 0.000300\n",
      "2019-10-30 16:12:19,785 Epoch  11 Step:    94200 Batch Loss:     1.528227 Tokens per Sec:    17086, Lr: 0.000300\n",
      "2019-10-30 16:12:34,772 Epoch  11 Step:    94300 Batch Loss:     1.318947 Tokens per Sec:    17344, Lr: 0.000300\n",
      "2019-10-30 16:12:49,850 Epoch  11 Step:    94400 Batch Loss:     1.420350 Tokens per Sec:    17550, Lr: 0.000300\n",
      "2019-10-30 16:13:04,720 Epoch  11 Step:    94500 Batch Loss:     1.328180 Tokens per Sec:    17227, Lr: 0.000300\n",
      "2019-10-30 16:13:19,821 Epoch  11 Step:    94600 Batch Loss:     1.350707 Tokens per Sec:    17024, Lr: 0.000300\n",
      "2019-10-30 16:13:34,694 Epoch  11 Step:    94700 Batch Loss:     1.475865 Tokens per Sec:    16838, Lr: 0.000300\n",
      "2019-10-30 16:13:49,469 Epoch  11 Step:    94800 Batch Loss:     1.340694 Tokens per Sec:    17472, Lr: 0.000300\n",
      "2019-10-30 16:14:04,561 Epoch  11 Step:    94900 Batch Loss:     1.565766 Tokens per Sec:    17184, Lr: 0.000300\n",
      "2019-10-30 16:14:19,622 Epoch  11 Step:    95000 Batch Loss:     1.442489 Tokens per Sec:    17064, Lr: 0.000300\n",
      "2019-10-30 16:15:03,444 Example #0\n",
      "2019-10-30 16:15:03,444 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:15:03,444 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:15:03,444 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:15:03,444 Example #1\n",
      "2019-10-30 16:15:03,444 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:15:03,444 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:15:03,444 \tHypothesis: Biblia haidhuru utukufu .\n",
      "2019-10-30 16:15:03,444 Example #2\n",
      "2019-10-30 16:15:03,444 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:15:03,444 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:15:03,444 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukuleta nini ?\n",
      "2019-10-30 16:15:03,444 Example #3\n",
      "2019-10-30 16:15:03,445 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:15:03,445 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:15:03,445 \tHypothesis: Miguu ya makaburi na mabusu .\n",
      "2019-10-30 16:15:03,445 Validation result at epoch  11, step    95000: bleu:  35.47, loss: 33007.1328, ppl:   3.3233, duration: 43.8223s\n",
      "2019-10-30 16:15:18,484 Epoch  11 Step:    95100 Batch Loss:     1.516530 Tokens per Sec:    17225, Lr: 0.000300\n",
      "2019-10-30 16:15:33,319 Epoch  11 Step:    95200 Batch Loss:     1.489432 Tokens per Sec:    17297, Lr: 0.000300\n",
      "2019-10-30 16:15:48,189 Epoch  11 Step:    95300 Batch Loss:     1.538916 Tokens per Sec:    17272, Lr: 0.000300\n",
      "2019-10-30 16:16:03,077 Epoch  11 Step:    95400 Batch Loss:     1.295851 Tokens per Sec:    17581, Lr: 0.000300\n",
      "2019-10-30 16:16:18,073 Epoch  11 Step:    95500 Batch Loss:     1.416139 Tokens per Sec:    17155, Lr: 0.000300\n",
      "2019-10-30 16:16:32,952 Epoch  11 Step:    95600 Batch Loss:     1.467642 Tokens per Sec:    17312, Lr: 0.000300\n",
      "2019-10-30 16:16:47,726 Epoch  11 Step:    95700 Batch Loss:     1.340930 Tokens per Sec:    17444, Lr: 0.000300\n",
      "2019-10-30 16:17:02,672 Epoch  11 Step:    95800 Batch Loss:     1.246075 Tokens per Sec:    17247, Lr: 0.000300\n",
      "2019-10-30 16:17:17,718 Epoch  11 Step:    95900 Batch Loss:     1.265657 Tokens per Sec:    17081, Lr: 0.000300\n",
      "2019-10-30 16:17:32,592 Epoch  11 Step:    96000 Batch Loss:     1.440078 Tokens per Sec:    17441, Lr: 0.000300\n",
      "2019-10-30 16:18:16,423 Example #0\n",
      "2019-10-30 16:18:16,424 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:18:16,424 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:18:16,424 \tHypothesis: Alikuwa mja - mzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:18:16,424 Example #1\n",
      "2019-10-30 16:18:16,424 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:18:16,424 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:18:16,424 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 16:18:16,424 Example #2\n",
      "2019-10-30 16:18:16,424 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:18:16,424 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:18:16,424 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:18:16,424 Example #3\n",
      "2019-10-30 16:18:16,424 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:18:16,425 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:18:16,425 \tHypothesis: Watu hupenda makumbatio na mabusu .\n",
      "2019-10-30 16:18:16,425 Validation result at epoch  11, step    96000: bleu:  34.93, loss: 33020.6445, ppl:   3.3249, duration: 43.8320s\n",
      "2019-10-30 16:18:31,217 Epoch  11 Step:    96100 Batch Loss:     1.349506 Tokens per Sec:    16964, Lr: 0.000300\n",
      "2019-10-30 16:18:46,085 Epoch  11 Step:    96200 Batch Loss:     1.296295 Tokens per Sec:    17408, Lr: 0.000300\n",
      "2019-10-30 16:19:00,872 Epoch  11 Step:    96300 Batch Loss:     1.274385 Tokens per Sec:    17415, Lr: 0.000300\n",
      "2019-10-30 16:19:15,855 Epoch  11 Step:    96400 Batch Loss:     1.267052 Tokens per Sec:    17010, Lr: 0.000300\n",
      "2019-10-30 16:19:30,687 Epoch  11 Step:    96500 Batch Loss:     1.354231 Tokens per Sec:    17013, Lr: 0.000300\n",
      "2019-10-30 16:19:45,762 Epoch  11 Step:    96600 Batch Loss:     1.407585 Tokens per Sec:    16978, Lr: 0.000300\n",
      "2019-10-30 16:20:00,781 Epoch  11 Step:    96700 Batch Loss:     1.259688 Tokens per Sec:    16785, Lr: 0.000300\n",
      "2019-10-30 16:20:15,575 Epoch  11 Step:    96800 Batch Loss:     1.258698 Tokens per Sec:    17309, Lr: 0.000300\n",
      "2019-10-30 16:20:30,493 Epoch  11 Step:    96900 Batch Loss:     1.258187 Tokens per Sec:    17389, Lr: 0.000300\n",
      "2019-10-30 16:20:45,172 Epoch  11 Step:    97000 Batch Loss:     1.391330 Tokens per Sec:    17029, Lr: 0.000300\n",
      "2019-10-30 16:21:28,999 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:21:29,000 Saving new checkpoint.\n",
      "2019-10-30 16:21:29,145 Example #0\n",
      "2019-10-30 16:21:29,145 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:21:29,145 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:21:29,145 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:21:29,145 Example #1\n",
      "2019-10-30 16:21:29,146 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:21:29,146 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:21:29,146 \tHypothesis: Biblia haisemi umashuhuri .\n",
      "2019-10-30 16:21:29,146 Example #2\n",
      "2019-10-30 16:21:29,146 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:21:29,146 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:21:29,146 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:21:29,146 Example #3\n",
      "2019-10-30 16:21:29,146 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:21:29,146 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:21:29,146 \tHypothesis: Mapenzi ya makumbatio na mabusu .\n",
      "2019-10-30 16:21:29,146 Validation result at epoch  11, step    97000: bleu:  34.74, loss: 32866.2344, ppl:   3.3063, duration: 43.9733s\n",
      "2019-10-30 16:21:43,955 Epoch  11 Step:    97100 Batch Loss:     1.213243 Tokens per Sec:    17184, Lr: 0.000300\n",
      "2019-10-30 16:21:58,931 Epoch  11 Step:    97200 Batch Loss:     1.436308 Tokens per Sec:    17341, Lr: 0.000300\n",
      "2019-10-30 16:22:13,855 Epoch  11 Step:    97300 Batch Loss:     1.426064 Tokens per Sec:    16701, Lr: 0.000300\n",
      "2019-10-30 16:22:28,852 Epoch  11 Step:    97400 Batch Loss:     1.371626 Tokens per Sec:    17047, Lr: 0.000300\n",
      "2019-10-30 16:22:44,041 Epoch  11 Step:    97500 Batch Loss:     1.465967 Tokens per Sec:    17116, Lr: 0.000300\n",
      "2019-10-30 16:22:59,240 Epoch  11 Step:    97600 Batch Loss:     1.253742 Tokens per Sec:    17311, Lr: 0.000300\n",
      "2019-10-30 16:23:14,210 Epoch  11 Step:    97700 Batch Loss:     1.161166 Tokens per Sec:    17096, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 16:23:29,080 Epoch  11 Step:    97800 Batch Loss:     1.233443 Tokens per Sec:    17365, Lr: 0.000300\n",
      "2019-10-30 16:23:44,241 Epoch  11 Step:    97900 Batch Loss:     1.277148 Tokens per Sec:    17162, Lr: 0.000300\n",
      "2019-10-30 16:23:59,546 Epoch  11 Step:    98000 Batch Loss:     1.332991 Tokens per Sec:    17104, Lr: 0.000300\n",
      "2019-10-30 16:24:43,461 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:24:43,462 Saving new checkpoint.\n",
      "2019-10-30 16:24:43,604 Example #0\n",
      "2019-10-30 16:24:43,605 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:24:43,605 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:24:43,605 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:24:43,605 Example #1\n",
      "2019-10-30 16:24:43,605 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:24:43,605 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:24:43,605 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 16:24:43,605 Example #2\n",
      "2019-10-30 16:24:43,605 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:24:43,605 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:24:43,605 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:24:43,605 Example #3\n",
      "2019-10-30 16:24:43,605 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:24:43,605 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:24:43,605 \tHypothesis: Miti ya maua na mabusu .\n",
      "2019-10-30 16:24:43,606 Validation result at epoch  11, step    98000: bleu:  35.09, loss: 32813.8438, ppl:   3.3000, duration: 44.0591s\n",
      "2019-10-30 16:24:58,607 Epoch  11 Step:    98100 Batch Loss:     1.403624 Tokens per Sec:    17168, Lr: 0.000300\n",
      "2019-10-30 16:25:13,652 Epoch  11 Step:    98200 Batch Loss:     1.226211 Tokens per Sec:    16948, Lr: 0.000300\n",
      "2019-10-30 16:25:28,675 Epoch  11 Step:    98300 Batch Loss:     1.428766 Tokens per Sec:    17055, Lr: 0.000300\n",
      "2019-10-30 16:25:43,621 Epoch  11 Step:    98400 Batch Loss:     1.356020 Tokens per Sec:    17008, Lr: 0.000300\n",
      "2019-10-30 16:25:58,633 Epoch  11 Step:    98500 Batch Loss:     1.263955 Tokens per Sec:    17422, Lr: 0.000300\n",
      "2019-10-30 16:26:13,504 Epoch  11 Step:    98600 Batch Loss:     1.218480 Tokens per Sec:    17335, Lr: 0.000300\n",
      "2019-10-30 16:26:28,327 Epoch  11 Step:    98700 Batch Loss:     1.380581 Tokens per Sec:    17478, Lr: 0.000300\n",
      "2019-10-30 16:26:43,349 Epoch  11 Step:    98800 Batch Loss:     1.430318 Tokens per Sec:    17026, Lr: 0.000300\n",
      "2019-10-30 16:26:58,399 Epoch  11 Step:    98900 Batch Loss:     1.287821 Tokens per Sec:    17134, Lr: 0.000300\n",
      "2019-10-30 16:27:13,486 Epoch  11 Step:    99000 Batch Loss:     1.356292 Tokens per Sec:    17228, Lr: 0.000300\n",
      "2019-10-30 16:27:57,483 Example #0\n",
      "2019-10-30 16:27:57,483 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:27:57,483 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:27:57,483 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:27:57,483 Example #1\n",
      "2019-10-30 16:27:57,483 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:27:57,483 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:27:57,483 \tHypothesis: Biblia haidhuru uthabiti .\n",
      "2019-10-30 16:27:57,483 Example #2\n",
      "2019-10-30 16:27:57,483 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:27:57,483 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:27:57,483 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:27:57,484 Example #3\n",
      "2019-10-30 16:27:57,484 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:27:57,484 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:27:57,484 \tHypothesis: Mapigo ya makumbatio na mabusu .\n",
      "2019-10-30 16:27:57,484 Validation result at epoch  11, step    99000: bleu:  35.17, loss: 32815.8516, ppl:   3.3003, duration: 43.9973s\n",
      "2019-10-30 16:28:12,467 Epoch  11 Step:    99100 Batch Loss:     1.311918 Tokens per Sec:    16812, Lr: 0.000300\n",
      "2019-10-30 16:28:27,287 Epoch  11 Step:    99200 Batch Loss:     1.424480 Tokens per Sec:    17080, Lr: 0.000300\n",
      "2019-10-30 16:28:42,149 Epoch  11 Step:    99300 Batch Loss:     1.392737 Tokens per Sec:    17027, Lr: 0.000300\n",
      "2019-10-30 16:28:56,970 Epoch  11 Step:    99400 Batch Loss:     1.324794 Tokens per Sec:    17306, Lr: 0.000300\n",
      "2019-10-30 16:29:11,930 Epoch  11 Step:    99500 Batch Loss:     1.326437 Tokens per Sec:    17492, Lr: 0.000300\n",
      "2019-10-30 16:29:26,719 Epoch  11 Step:    99600 Batch Loss:     1.309648 Tokens per Sec:    17141, Lr: 0.000300\n",
      "2019-10-30 16:29:41,409 Epoch  11 Step:    99700 Batch Loss:     1.365254 Tokens per Sec:    16832, Lr: 0.000300\n",
      "2019-10-30 16:29:56,442 Epoch  11 Step:    99800 Batch Loss:     1.340744 Tokens per Sec:    17485, Lr: 0.000300\n",
      "2019-10-30 16:30:11,459 Epoch  11 Step:    99900 Batch Loss:     1.261574 Tokens per Sec:    17355, Lr: 0.000300\n",
      "2019-10-30 16:30:26,464 Epoch  11 Step:   100000 Batch Loss:     1.153374 Tokens per Sec:    17189, Lr: 0.000300\n",
      "2019-10-30 16:31:10,675 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:31:10,675 Saving new checkpoint.\n",
      "2019-10-30 16:31:10,822 Example #0\n",
      "2019-10-30 16:31:10,822 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:31:10,822 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:31:10,822 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:31:10,822 Example #1\n",
      "2019-10-30 16:31:10,822 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:31:10,822 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:31:10,822 \tHypothesis: Biblia haisemi ukweli .\n",
      "2019-10-30 16:31:10,822 Example #2\n",
      "2019-10-30 16:31:10,822 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:31:10,822 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:31:10,822 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:31:10,822 Example #3\n",
      "2019-10-30 16:31:10,823 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:31:10,823 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:31:10,823 \tHypothesis: Mapigo na mabusu .\n",
      "2019-10-30 16:31:10,823 Validation result at epoch  11, step   100000: bleu:  34.93, loss: 32683.2773, ppl:   3.2844, duration: 44.3585s\n",
      "2019-10-30 16:31:25,780 Epoch  11 Step:   100100 Batch Loss:     1.601825 Tokens per Sec:    17074, Lr: 0.000300\n",
      "2019-10-30 16:31:40,521 Epoch  11 Step:   100200 Batch Loss:     1.390234 Tokens per Sec:    17319, Lr: 0.000300\n",
      "2019-10-30 16:31:55,352 Epoch  11 Step:   100300 Batch Loss:     1.405399 Tokens per Sec:    16988, Lr: 0.000300\n",
      "2019-10-30 16:32:10,328 Epoch  11 Step:   100400 Batch Loss:     1.338304 Tokens per Sec:    17156, Lr: 0.000300\n",
      "2019-10-30 16:32:25,290 Epoch  11 Step:   100500 Batch Loss:     1.264371 Tokens per Sec:    17367, Lr: 0.000300\n",
      "2019-10-30 16:32:40,288 Epoch  11 Step:   100600 Batch Loss:     1.461347 Tokens per Sec:    17045, Lr: 0.000300\n",
      "2019-10-30 16:32:55,283 Epoch  11 Step:   100700 Batch Loss:     1.317091 Tokens per Sec:    17390, Lr: 0.000300\n",
      "2019-10-30 16:33:10,445 Epoch  11 Step:   100800 Batch Loss:     1.228321 Tokens per Sec:    17171, Lr: 0.000300\n",
      "2019-10-30 16:33:25,716 Epoch  11 Step:   100900 Batch Loss:     1.239890 Tokens per Sec:    16785, Lr: 0.000300\n",
      "2019-10-30 16:33:40,989 Epoch  11 Step:   101000 Batch Loss:     1.521468 Tokens per Sec:    16801, Lr: 0.000300\n",
      "2019-10-30 16:34:25,129 Example #0\n",
      "2019-10-30 16:34:25,129 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:34:25,129 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:34:25,129 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:34:25,129 Example #1\n",
      "2019-10-30 16:34:25,130 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:34:25,130 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:34:25,130 \tHypothesis: Biblia haisemi ukweli .\n",
      "2019-10-30 16:34:25,130 Example #2\n",
      "2019-10-30 16:34:25,130 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:34:25,130 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:34:25,130 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 16:34:25,130 Example #3\n",
      "2019-10-30 16:34:25,130 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:34:25,130 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:34:25,130 \tHypothesis: Wanyama wa makumbatio na mabusu .\n",
      "2019-10-30 16:34:25,130 Validation result at epoch  11, step   101000: bleu:  35.49, loss: 32718.9316, ppl:   3.2886, duration: 44.1402s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 16:34:40,216 Epoch  11 Step:   101100 Batch Loss:     1.477701 Tokens per Sec:    16908, Lr: 0.000300\n",
      "2019-10-30 16:34:55,059 Epoch  11 Step:   101200 Batch Loss:     1.199437 Tokens per Sec:    17299, Lr: 0.000300\n",
      "2019-10-30 16:35:09,843 Epoch  11 Step:   101300 Batch Loss:     1.175637 Tokens per Sec:    17568, Lr: 0.000300\n",
      "2019-10-30 16:35:24,761 Epoch  11 Step:   101400 Batch Loss:     1.230750 Tokens per Sec:    17386, Lr: 0.000300\n",
      "2019-10-30 16:35:40,015 Epoch  11 Step:   101500 Batch Loss:     1.453219 Tokens per Sec:    16909, Lr: 0.000300\n",
      "2019-10-30 16:35:55,075 Epoch  11 Step:   101600 Batch Loss:     1.507329 Tokens per Sec:    17261, Lr: 0.000300\n",
      "2019-10-30 16:36:10,200 Epoch  11 Step:   101700 Batch Loss:     1.395085 Tokens per Sec:    17182, Lr: 0.000300\n",
      "2019-10-30 16:36:25,251 Epoch  11 Step:   101800 Batch Loss:     1.315895 Tokens per Sec:    16976, Lr: 0.000300\n",
      "2019-10-30 16:36:40,307 Epoch  11 Step:   101900 Batch Loss:     1.496288 Tokens per Sec:    16892, Lr: 0.000300\n",
      "2019-10-30 16:36:55,214 Epoch  11 Step:   102000 Batch Loss:     1.498945 Tokens per Sec:    17187, Lr: 0.000300\n",
      "2019-10-30 16:37:39,320 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:37:39,320 Saving new checkpoint.\n",
      "2019-10-30 16:37:39,467 Example #0\n",
      "2019-10-30 16:37:39,467 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:37:39,467 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:37:39,467 \tHypothesis: Alikuwa mjamzito alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:37:39,467 Example #1\n",
      "2019-10-30 16:37:39,468 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:37:39,468 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:37:39,468 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 16:37:39,468 Example #2\n",
      "2019-10-30 16:37:39,468 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:37:39,468 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:37:39,468 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:37:39,468 Example #3\n",
      "2019-10-30 16:37:39,468 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:37:39,468 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:37:39,468 \tHypothesis: Wanyama wa makumbatio na mabusu .\n",
      "2019-10-30 16:37:39,468 Validation result at epoch  11, step   102000: bleu:  35.17, loss: 32614.1562, ppl:   3.2761, duration: 44.2532s\n",
      "2019-10-30 16:37:54,756 Epoch  11 Step:   102100 Batch Loss:     1.306539 Tokens per Sec:    17194, Lr: 0.000300\n",
      "2019-10-30 16:38:09,594 Epoch  11 Step:   102200 Batch Loss:     1.200989 Tokens per Sec:    17227, Lr: 0.000300\n",
      "2019-10-30 16:38:24,410 Epoch  11 Step:   102300 Batch Loss:     1.280920 Tokens per Sec:    17106, Lr: 0.000300\n",
      "2019-10-30 16:38:39,593 Epoch  11 Step:   102400 Batch Loss:     1.359547 Tokens per Sec:    17262, Lr: 0.000300\n",
      "2019-10-30 16:38:54,773 Epoch  11 Step:   102500 Batch Loss:     1.553583 Tokens per Sec:    17043, Lr: 0.000300\n",
      "2019-10-30 16:39:09,981 Epoch  11 Step:   102600 Batch Loss:     1.484060 Tokens per Sec:    16938, Lr: 0.000300\n",
      "2019-10-30 16:39:25,075 Epoch  11 Step:   102700 Batch Loss:     1.325256 Tokens per Sec:    16788, Lr: 0.000300\n",
      "2019-10-30 16:39:40,140 Epoch  11 Step:   102800 Batch Loss:     1.264477 Tokens per Sec:    16972, Lr: 0.000300\n",
      "2019-10-30 16:39:55,084 Epoch  11 Step:   102900 Batch Loss:     1.487790 Tokens per Sec:    17068, Lr: 0.000300\n",
      "2019-10-30 16:40:10,058 Epoch  11 Step:   103000 Batch Loss:     1.242520 Tokens per Sec:    17459, Lr: 0.000300\n",
      "2019-10-30 16:40:54,004 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:40:54,005 Saving new checkpoint.\n",
      "2019-10-30 16:40:54,149 Example #0\n",
      "2019-10-30 16:40:54,149 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:40:54,149 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:40:54,149 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:40:54,149 Example #1\n",
      "2019-10-30 16:40:54,149 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:40:54,149 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:40:54,149 \tHypothesis: Biblia haidhihirishi utukufu .\n",
      "2019-10-30 16:40:54,149 Example #2\n",
      "2019-10-30 16:40:54,149 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:40:54,149 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:40:54,149 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 16:40:54,149 Example #3\n",
      "2019-10-30 16:40:54,150 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:40:54,150 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:40:54,150 \tHypothesis: Miguu ya makumbusho na mabusu .\n",
      "2019-10-30 16:40:54,150 Validation result at epoch  11, step   103000: bleu:  34.72, loss: 32521.3711, ppl:   3.2651, duration: 44.0910s\n",
      "2019-10-30 16:41:09,045 Epoch  11 Step:   103100 Batch Loss:     1.366599 Tokens per Sec:    17182, Lr: 0.000300\n",
      "2019-10-30 16:41:23,881 Epoch  11 Step:   103200 Batch Loss:     1.424035 Tokens per Sec:    17247, Lr: 0.000300\n",
      "2019-10-30 16:41:26,982 Epoch  11: total training loss 12661.86\n",
      "2019-10-30 16:41:26,982 EPOCH 12\n",
      "2019-10-30 16:41:39,792 Epoch  12 Step:   103300 Batch Loss:     1.394114 Tokens per Sec:    15560, Lr: 0.000300\n",
      "2019-10-30 16:41:54,849 Epoch  12 Step:   103400 Batch Loss:     1.325628 Tokens per Sec:    17458, Lr: 0.000300\n",
      "2019-10-30 16:42:10,070 Epoch  12 Step:   103500 Batch Loss:     1.300135 Tokens per Sec:    16971, Lr: 0.000300\n",
      "2019-10-30 16:42:25,034 Epoch  12 Step:   103600 Batch Loss:     1.269475 Tokens per Sec:    16788, Lr: 0.000300\n",
      "2019-10-30 16:42:40,144 Epoch  12 Step:   103700 Batch Loss:     1.437438 Tokens per Sec:    17364, Lr: 0.000300\n",
      "2019-10-30 16:42:55,073 Epoch  12 Step:   103800 Batch Loss:     1.426135 Tokens per Sec:    17061, Lr: 0.000300\n",
      "2019-10-30 16:43:09,878 Epoch  12 Step:   103900 Batch Loss:     1.455143 Tokens per Sec:    17422, Lr: 0.000300\n",
      "2019-10-30 16:43:24,832 Epoch  12 Step:   104000 Batch Loss:     1.192068 Tokens per Sec:    17300, Lr: 0.000300\n",
      "2019-10-30 16:44:09,487 Example #0\n",
      "2019-10-30 16:44:09,487 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:44:09,487 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:44:09,487 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:44:09,487 Example #1\n",
      "2019-10-30 16:44:09,487 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:44:09,488 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:44:09,488 \tHypothesis: Biblia haidhuru utoaji - sifa .\n",
      "2019-10-30 16:44:09,488 Example #2\n",
      "2019-10-30 16:44:09,488 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:44:09,488 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:44:09,488 \tHypothesis: Ni mambo gani yanayofanya mahangaiko , na mahangaiko yanaweza kuwa na matokeo gani ?\n",
      "2019-10-30 16:44:09,488 Example #3\n",
      "2019-10-30 16:44:09,488 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:44:09,488 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:44:09,488 \tHypothesis: Mapigo ya makumbusho na mabusu .\n",
      "2019-10-30 16:44:09,488 Validation result at epoch  12, step   104000: bleu:  35.24, loss: 32619.6953, ppl:   3.2768, duration: 44.6551s\n",
      "2019-10-30 16:44:24,512 Epoch  12 Step:   104100 Batch Loss:     1.274446 Tokens per Sec:    17124, Lr: 0.000300\n",
      "2019-10-30 16:44:39,393 Epoch  12 Step:   104200 Batch Loss:     1.378191 Tokens per Sec:    17388, Lr: 0.000300\n",
      "2019-10-30 16:44:54,162 Epoch  12 Step:   104300 Batch Loss:     1.181774 Tokens per Sec:    17189, Lr: 0.000300\n",
      "2019-10-30 16:45:09,087 Epoch  12 Step:   104400 Batch Loss:     1.357125 Tokens per Sec:    17359, Lr: 0.000300\n",
      "2019-10-30 16:45:23,916 Epoch  12 Step:   104500 Batch Loss:     1.354706 Tokens per Sec:    17141, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 16:45:38,778 Epoch  12 Step:   104600 Batch Loss:     1.309068 Tokens per Sec:    16929, Lr: 0.000300\n",
      "2019-10-30 16:45:53,616 Epoch  12 Step:   104700 Batch Loss:     1.180131 Tokens per Sec:    17434, Lr: 0.000300\n",
      "2019-10-30 16:46:08,715 Epoch  12 Step:   104800 Batch Loss:     1.406518 Tokens per Sec:    17247, Lr: 0.000300\n",
      "2019-10-30 16:46:23,635 Epoch  12 Step:   104900 Batch Loss:     1.495262 Tokens per Sec:    16977, Lr: 0.000300\n",
      "2019-10-30 16:46:38,620 Epoch  12 Step:   105000 Batch Loss:     1.254391 Tokens per Sec:    17004, Lr: 0.000300\n",
      "2019-10-30 16:47:22,557 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 16:47:22,557 Saving new checkpoint.\n",
      "2019-10-30 16:47:22,698 Example #0\n",
      "2019-10-30 16:47:22,699 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:47:22,699 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:47:22,699 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:47:22,699 Example #1\n",
      "2019-10-30 16:47:22,699 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:47:22,699 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:47:22,699 \tHypothesis: Biblia haidhihirishi utukufu .\n",
      "2019-10-30 16:47:22,699 Example #2\n",
      "2019-10-30 16:47:22,699 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:47:22,699 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:47:22,699 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyiza nini ?\n",
      "2019-10-30 16:47:22,699 Example #3\n",
      "2019-10-30 16:47:22,699 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:47:22,699 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:47:22,700 \tHypothesis: Miguu ya nyati na mabusu .\n",
      "2019-10-30 16:47:22,700 Validation result at epoch  12, step   105000: bleu:  35.00, loss: 32434.0957, ppl:   3.2547, duration: 44.0794s\n",
      "2019-10-30 16:47:37,918 Epoch  12 Step:   105100 Batch Loss:     1.186605 Tokens per Sec:    17229, Lr: 0.000300\n",
      "2019-10-30 16:47:53,042 Epoch  12 Step:   105200 Batch Loss:     1.387057 Tokens per Sec:    17016, Lr: 0.000300\n",
      "2019-10-30 16:48:08,106 Epoch  12 Step:   105300 Batch Loss:     1.389285 Tokens per Sec:    17079, Lr: 0.000300\n",
      "2019-10-30 16:48:23,075 Epoch  12 Step:   105400 Batch Loss:     1.158874 Tokens per Sec:    16990, Lr: 0.000300\n",
      "2019-10-30 16:48:38,197 Epoch  12 Step:   105500 Batch Loss:     1.205258 Tokens per Sec:    17281, Lr: 0.000300\n",
      "2019-10-30 16:48:53,257 Epoch  12 Step:   105600 Batch Loss:     1.590065 Tokens per Sec:    17019, Lr: 0.000300\n",
      "2019-10-30 16:49:08,108 Epoch  12 Step:   105700 Batch Loss:     1.062003 Tokens per Sec:    16525, Lr: 0.000300\n",
      "2019-10-30 16:49:23,142 Epoch  12 Step:   105800 Batch Loss:     1.393815 Tokens per Sec:    16578, Lr: 0.000300\n",
      "2019-10-30 16:49:38,246 Epoch  12 Step:   105900 Batch Loss:     1.253051 Tokens per Sec:    17182, Lr: 0.000300\n",
      "2019-10-30 16:49:53,268 Epoch  12 Step:   106000 Batch Loss:     1.230266 Tokens per Sec:    17265, Lr: 0.000300\n",
      "2019-10-30 16:50:37,177 Example #0\n",
      "2019-10-30 16:50:37,177 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:50:37,177 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:50:37,177 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:50:37,177 Example #1\n",
      "2019-10-30 16:50:37,177 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:50:37,178 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:50:37,178 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 16:50:37,178 Example #2\n",
      "2019-10-30 16:50:37,178 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:50:37,178 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:50:37,178 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuchochea nini ?\n",
      "2019-10-30 16:50:37,178 Example #3\n",
      "2019-10-30 16:50:37,178 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:50:37,178 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:50:37,178 \tHypothesis: Mapigo ya makumbatio na mabusu .\n",
      "2019-10-30 16:50:37,178 Validation result at epoch  12, step   106000: bleu:  35.49, loss: 32509.0352, ppl:   3.2636, duration: 43.9092s\n",
      "2019-10-30 16:50:52,255 Epoch  12 Step:   106100 Batch Loss:     1.192293 Tokens per Sec:    17193, Lr: 0.000300\n",
      "2019-10-30 16:51:07,288 Epoch  12 Step:   106200 Batch Loss:     1.310944 Tokens per Sec:    16976, Lr: 0.000300\n",
      "2019-10-30 16:51:22,350 Epoch  12 Step:   106300 Batch Loss:     1.437109 Tokens per Sec:    16780, Lr: 0.000300\n",
      "2019-10-30 16:51:37,310 Epoch  12 Step:   106400 Batch Loss:     1.256850 Tokens per Sec:    16971, Lr: 0.000300\n",
      "2019-10-30 16:51:52,434 Epoch  12 Step:   106500 Batch Loss:     1.584865 Tokens per Sec:    17166, Lr: 0.000300\n",
      "2019-10-30 16:52:07,598 Epoch  12 Step:   106600 Batch Loss:     1.380042 Tokens per Sec:    16798, Lr: 0.000300\n",
      "2019-10-30 16:52:22,898 Epoch  12 Step:   106700 Batch Loss:     1.405191 Tokens per Sec:    17377, Lr: 0.000300\n",
      "2019-10-30 16:52:38,044 Epoch  12 Step:   106800 Batch Loss:     1.197884 Tokens per Sec:    16919, Lr: 0.000300\n",
      "2019-10-30 16:52:53,174 Epoch  12 Step:   106900 Batch Loss:     1.390163 Tokens per Sec:    17065, Lr: 0.000300\n",
      "2019-10-30 16:53:08,183 Epoch  12 Step:   107000 Batch Loss:     1.377176 Tokens per Sec:    17158, Lr: 0.000300\n",
      "2019-10-30 16:53:52,260 Example #0\n",
      "2019-10-30 16:53:52,260 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:53:52,260 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:53:52,260 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 16:53:52,260 Example #1\n",
      "2019-10-30 16:53:52,260 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:53:52,261 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:53:52,261 \tHypothesis: Biblia haisemi sifa ya uthabiti .\n",
      "2019-10-30 16:53:52,261 Example #2\n",
      "2019-10-30 16:53:52,261 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:53:52,261 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:53:52,261 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 16:53:52,261 Example #3\n",
      "2019-10-30 16:53:52,261 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:53:52,261 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:53:52,261 \tHypothesis: Mapigo ya maua na mabusu .\n",
      "2019-10-30 16:53:52,261 Validation result at epoch  12, step   107000: bleu:  35.39, loss: 32454.4102, ppl:   3.2571, duration: 44.0772s\n",
      "2019-10-30 16:54:07,104 Epoch  12 Step:   107100 Batch Loss:     1.216520 Tokens per Sec:    16993, Lr: 0.000300\n",
      "2019-10-30 16:54:22,090 Epoch  12 Step:   107200 Batch Loss:     1.279639 Tokens per Sec:    17610, Lr: 0.000300\n",
      "2019-10-30 16:54:37,438 Epoch  12 Step:   107300 Batch Loss:     1.283575 Tokens per Sec:    16878, Lr: 0.000300\n",
      "2019-10-30 16:54:52,642 Epoch  12 Step:   107400 Batch Loss:     1.351085 Tokens per Sec:    17015, Lr: 0.000300\n",
      "2019-10-30 16:55:07,700 Epoch  12 Step:   107500 Batch Loss:     1.383982 Tokens per Sec:    17040, Lr: 0.000300\n",
      "2019-10-30 16:55:22,875 Epoch  12 Step:   107600 Batch Loss:     1.546627 Tokens per Sec:    17085, Lr: 0.000300\n",
      "2019-10-30 16:55:37,618 Epoch  12 Step:   107700 Batch Loss:     2.125069 Tokens per Sec:    17234, Lr: 0.000300\n",
      "2019-10-30 16:55:52,414 Epoch  12 Step:   107800 Batch Loss:     1.257461 Tokens per Sec:    17034, Lr: 0.000300\n",
      "2019-10-30 16:56:07,744 Epoch  12 Step:   107900 Batch Loss:     1.233565 Tokens per Sec:    16752, Lr: 0.000300\n",
      "2019-10-30 16:56:22,849 Epoch  12 Step:   108000 Batch Loss:     1.332915 Tokens per Sec:    17014, Lr: 0.000300\n",
      "2019-10-30 16:57:06,809 Example #0\n",
      "2019-10-30 16:57:06,809 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 16:57:06,809 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:57:06,809 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 16:57:06,809 Example #1\n",
      "2019-10-30 16:57:06,809 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 16:57:06,809 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 16:57:06,809 \tHypothesis: Biblia haisemi sifa ya uthabiti .\n",
      "2019-10-30 16:57:06,809 Example #2\n",
      "2019-10-30 16:57:06,810 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 16:57:06,810 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 16:57:06,810 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 16:57:06,810 Example #3\n",
      "2019-10-30 16:57:06,810 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 16:57:06,810 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 16:57:06,810 \tHypothesis: Wanyama wa makaburi na mabusu .\n",
      "2019-10-30 16:57:06,810 Validation result at epoch  12, step   108000: bleu:  35.68, loss: 32438.5410, ppl:   3.2553, duration: 43.9604s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 16:57:21,895 Epoch  12 Step:   108100 Batch Loss:     1.475471 Tokens per Sec:    16757, Lr: 0.000300\n",
      "2019-10-30 16:57:36,836 Epoch  12 Step:   108200 Batch Loss:     1.256675 Tokens per Sec:    17268, Lr: 0.000300\n",
      "2019-10-30 16:57:51,972 Epoch  12 Step:   108300 Batch Loss:     1.232620 Tokens per Sec:    17345, Lr: 0.000300\n",
      "2019-10-30 16:58:07,006 Epoch  12 Step:   108400 Batch Loss:     1.233303 Tokens per Sec:    17399, Lr: 0.000300\n",
      "2019-10-30 16:58:22,294 Epoch  12 Step:   108500 Batch Loss:     1.345724 Tokens per Sec:    16993, Lr: 0.000300\n",
      "2019-10-30 16:58:37,638 Epoch  12 Step:   108600 Batch Loss:     1.330140 Tokens per Sec:    17191, Lr: 0.000300\n",
      "2019-10-30 16:58:52,711 Epoch  12 Step:   108700 Batch Loss:     1.338794 Tokens per Sec:    16838, Lr: 0.000300\n",
      "2019-10-30 16:59:07,642 Epoch  12 Step:   108800 Batch Loss:     1.291398 Tokens per Sec:    17001, Lr: 0.000300\n",
      "2019-10-30 16:59:22,650 Epoch  12 Step:   108900 Batch Loss:     1.347979 Tokens per Sec:    17270, Lr: 0.000300\n",
      "2019-10-30 16:59:37,638 Epoch  12 Step:   109000 Batch Loss:     1.358806 Tokens per Sec:    16928, Lr: 0.000300\n",
      "2019-10-30 17:00:22,066 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:00:22,066 Saving new checkpoint.\n",
      "2019-10-30 17:00:22,213 Example #0\n",
      "2019-10-30 17:00:22,213 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:00:22,213 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:00:22,214 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:00:22,214 Example #1\n",
      "2019-10-30 17:00:22,214 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:00:22,214 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:00:22,214 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 17:00:22,214 Example #2\n",
      "2019-10-30 17:00:22,214 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:00:22,214 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:00:22,214 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 17:00:22,214 Example #3\n",
      "2019-10-30 17:00:22,214 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:00:22,214 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:00:22,214 \tHypothesis: Mabusu ya makaburi na mabusu .\n",
      "2019-10-30 17:00:22,214 Validation result at epoch  12, step   109000: bleu:  35.19, loss: 32308.5801, ppl:   3.2399, duration: 44.5757s\n",
      "2019-10-30 17:00:37,228 Epoch  12 Step:   109100 Batch Loss:     1.288289 Tokens per Sec:    17302, Lr: 0.000300\n",
      "2019-10-30 17:00:52,334 Epoch  12 Step:   109200 Batch Loss:     1.463105 Tokens per Sec:    16648, Lr: 0.000300\n",
      "2019-10-30 17:01:07,537 Epoch  12 Step:   109300 Batch Loss:     1.415742 Tokens per Sec:    16994, Lr: 0.000300\n",
      "2019-10-30 17:01:23,150 Epoch  12 Step:   109400 Batch Loss:     1.445693 Tokens per Sec:    16666, Lr: 0.000300\n",
      "2019-10-30 17:01:38,334 Epoch  12 Step:   109500 Batch Loss:     1.268400 Tokens per Sec:    16987, Lr: 0.000300\n",
      "2019-10-30 17:01:53,349 Epoch  12 Step:   109600 Batch Loss:     1.353449 Tokens per Sec:    17015, Lr: 0.000300\n",
      "2019-10-30 17:02:08,268 Epoch  12 Step:   109700 Batch Loss:     1.338812 Tokens per Sec:    17117, Lr: 0.000300\n",
      "2019-10-30 17:02:23,444 Epoch  12 Step:   109800 Batch Loss:     1.141561 Tokens per Sec:    17169, Lr: 0.000300\n",
      "2019-10-30 17:02:38,413 Epoch  12 Step:   109900 Batch Loss:     1.118439 Tokens per Sec:    16909, Lr: 0.000300\n",
      "2019-10-30 17:02:53,524 Epoch  12 Step:   110000 Batch Loss:     1.073659 Tokens per Sec:    16901, Lr: 0.000300\n",
      "2019-10-30 17:03:37,633 Example #0\n",
      "2019-10-30 17:03:37,634 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:03:37,634 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:03:37,634 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:03:37,634 Example #1\n",
      "2019-10-30 17:03:37,634 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:03:37,634 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:03:37,634 \tHypothesis: Biblia haidhihirishi sifa ya utoaji - nyara .\n",
      "2019-10-30 17:03:37,634 Example #2\n",
      "2019-10-30 17:03:37,634 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:03:37,634 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:03:37,634 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 17:03:37,634 Example #3\n",
      "2019-10-30 17:03:37,634 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:03:37,634 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:03:37,634 \tHypothesis: Mapigo ya makaburi na mabusu .\n",
      "2019-10-30 17:03:37,634 Validation result at epoch  12, step   110000: bleu:  35.64, loss: 32375.7129, ppl:   3.2478, duration: 44.1100s\n",
      "2019-10-30 17:03:52,730 Epoch  12 Step:   110100 Batch Loss:     1.386534 Tokens per Sec:    16988, Lr: 0.000300\n",
      "2019-10-30 17:04:08,080 Epoch  12 Step:   110200 Batch Loss:     1.257811 Tokens per Sec:    16765, Lr: 0.000300\n",
      "2019-10-30 17:04:23,139 Epoch  12 Step:   110300 Batch Loss:     1.258221 Tokens per Sec:    17190, Lr: 0.000300\n",
      "2019-10-30 17:04:38,257 Epoch  12 Step:   110400 Batch Loss:     1.311409 Tokens per Sec:    17211, Lr: 0.000300\n",
      "2019-10-30 17:04:53,148 Epoch  12 Step:   110500 Batch Loss:     1.320721 Tokens per Sec:    17229, Lr: 0.000300\n",
      "2019-10-30 17:05:08,231 Epoch  12 Step:   110600 Batch Loss:     1.243394 Tokens per Sec:    17028, Lr: 0.000300\n",
      "2019-10-30 17:05:23,290 Epoch  12 Step:   110700 Batch Loss:     1.261618 Tokens per Sec:    16683, Lr: 0.000300\n",
      "2019-10-30 17:05:38,487 Epoch  12 Step:   110800 Batch Loss:     1.368608 Tokens per Sec:    17152, Lr: 0.000300\n",
      "2019-10-30 17:05:53,687 Epoch  12 Step:   110900 Batch Loss:     1.325748 Tokens per Sec:    17262, Lr: 0.000300\n",
      "2019-10-30 17:06:08,708 Epoch  12 Step:   111000 Batch Loss:     1.507208 Tokens per Sec:    16614, Lr: 0.000300\n",
      "2019-10-30 17:06:53,015 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:06:53,016 Saving new checkpoint.\n",
      "2019-10-30 17:06:53,165 Example #0\n",
      "2019-10-30 17:06:53,165 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:06:53,165 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:06:53,165 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:06:53,165 Example #1\n",
      "2019-10-30 17:06:53,165 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:06:53,165 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:06:53,165 \tHypothesis: Biblia haisemi ukweli .\n",
      "2019-10-30 17:06:53,165 Example #2\n",
      "2019-10-30 17:06:53,165 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:06:53,165 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:06:53,165 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukusaidia kufanya nini ?\n",
      "2019-10-30 17:06:53,166 Example #3\n",
      "2019-10-30 17:06:53,166 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:06:53,166 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:06:53,166 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 17:06:53,166 Validation result at epoch  12, step   111000: bleu:  35.54, loss: 32240.7598, ppl:   3.2319, duration: 44.4573s\n",
      "2019-10-30 17:07:08,216 Epoch  12 Step:   111100 Batch Loss:     1.291779 Tokens per Sec:    16690, Lr: 0.000300\n",
      "2019-10-30 17:07:23,432 Epoch  12 Step:   111200 Batch Loss:     1.288524 Tokens per Sec:    17126, Lr: 0.000300\n",
      "2019-10-30 17:07:38,364 Epoch  12 Step:   111300 Batch Loss:     1.364749 Tokens per Sec:    17447, Lr: 0.000300\n",
      "2019-10-30 17:07:53,328 Epoch  12 Step:   111400 Batch Loss:     1.328744 Tokens per Sec:    17104, Lr: 0.000300\n",
      "2019-10-30 17:08:08,301 Epoch  12 Step:   111500 Batch Loss:     1.297610 Tokens per Sec:    17314, Lr: 0.000300\n",
      "2019-10-30 17:08:23,235 Epoch  12 Step:   111600 Batch Loss:     1.282902 Tokens per Sec:    17368, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 17:08:38,503 Epoch  12 Step:   111700 Batch Loss:     1.306700 Tokens per Sec:    17206, Lr: 0.000300\n",
      "2019-10-30 17:08:53,442 Epoch  12 Step:   111800 Batch Loss:     1.252646 Tokens per Sec:    16883, Lr: 0.000300\n",
      "2019-10-30 17:09:08,485 Epoch  12 Step:   111900 Batch Loss:     1.285327 Tokens per Sec:    17046, Lr: 0.000300\n",
      "2019-10-30 17:09:23,459 Epoch  12 Step:   112000 Batch Loss:     1.404303 Tokens per Sec:    17032, Lr: 0.000300\n",
      "2019-10-30 17:10:07,772 Example #0\n",
      "2019-10-30 17:10:07,772 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:10:07,772 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:10:07,772 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:10:07,772 Example #1\n",
      "2019-10-30 17:10:07,772 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:10:07,772 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:10:07,772 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kustahili .\n",
      "2019-10-30 17:10:07,772 Example #2\n",
      "2019-10-30 17:10:07,772 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:10:07,773 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:10:07,773 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 17:10:07,773 Example #3\n",
      "2019-10-30 17:10:07,773 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:10:07,773 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:10:07,773 \tHypothesis: Mapigo ya maua na busu .\n",
      "2019-10-30 17:10:07,773 Validation result at epoch  12, step   112000: bleu:  35.59, loss: 32313.7363, ppl:   3.2405, duration: 44.3132s\n",
      "2019-10-30 17:10:22,700 Epoch  12 Step:   112100 Batch Loss:     1.380175 Tokens per Sec:    17029, Lr: 0.000300\n",
      "2019-10-30 17:10:37,581 Epoch  12 Step:   112200 Batch Loss:     1.075159 Tokens per Sec:    16673, Lr: 0.000300\n",
      "2019-10-30 17:10:52,408 Epoch  12 Step:   112300 Batch Loss:     1.391107 Tokens per Sec:    16832, Lr: 0.000300\n",
      "2019-10-30 17:11:07,392 Epoch  12 Step:   112400 Batch Loss:     1.379524 Tokens per Sec:    17005, Lr: 0.000300\n",
      "2019-10-30 17:11:22,499 Epoch  12 Step:   112500 Batch Loss:     1.744204 Tokens per Sec:    17028, Lr: 0.000300\n",
      "2019-10-30 17:11:37,519 Epoch  12 Step:   112600 Batch Loss:     1.334989 Tokens per Sec:    16932, Lr: 0.000300\n",
      "2019-10-30 17:11:39,528 Epoch  12: total training loss 12506.78\n",
      "2019-10-30 17:11:39,528 EPOCH 13\n",
      "2019-10-30 17:11:53,889 Epoch  13 Step:   112700 Batch Loss:     1.374600 Tokens per Sec:    15669, Lr: 0.000300\n",
      "2019-10-30 17:12:09,100 Epoch  13 Step:   112800 Batch Loss:     1.399010 Tokens per Sec:    16743, Lr: 0.000300\n",
      "2019-10-30 17:12:23,970 Epoch  13 Step:   112900 Batch Loss:     1.357598 Tokens per Sec:    17011, Lr: 0.000300\n",
      "2019-10-30 17:12:39,325 Epoch  13 Step:   113000 Batch Loss:     1.554083 Tokens per Sec:    16890, Lr: 0.000300\n",
      "2019-10-30 17:13:23,546 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:13:23,546 Saving new checkpoint.\n",
      "2019-10-30 17:13:23,742 Example #0\n",
      "2019-10-30 17:13:23,742 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:13:23,742 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:13:23,742 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:13:23,742 Example #1\n",
      "2019-10-30 17:13:23,742 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:13:23,742 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:13:23,742 \tHypothesis: Biblia haisemi utukufu .\n",
      "2019-10-30 17:13:23,742 Example #2\n",
      "2019-10-30 17:13:23,742 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:13:23,742 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:13:23,743 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 17:13:23,743 Example #3\n",
      "2019-10-30 17:13:23,743 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:13:23,743 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:13:23,743 \tHypothesis: Wapenzi wa makumbatio na mabusu .\n",
      "2019-10-30 17:13:23,743 Validation result at epoch  13, step   113000: bleu:  35.47, loss: 32175.1055, ppl:   3.2242, duration: 44.4178s\n",
      "2019-10-30 17:13:38,865 Epoch  13 Step:   113100 Batch Loss:     1.208541 Tokens per Sec:    17016, Lr: 0.000300\n",
      "2019-10-30 17:13:53,770 Epoch  13 Step:   113200 Batch Loss:     1.288707 Tokens per Sec:    17146, Lr: 0.000300\n",
      "2019-10-30 17:14:08,789 Epoch  13 Step:   113300 Batch Loss:     1.077242 Tokens per Sec:    16887, Lr: 0.000300\n",
      "2019-10-30 17:14:24,054 Epoch  13 Step:   113400 Batch Loss:     1.208218 Tokens per Sec:    17278, Lr: 0.000300\n",
      "2019-10-30 17:14:39,209 Epoch  13 Step:   113500 Batch Loss:     1.364233 Tokens per Sec:    17061, Lr: 0.000300\n",
      "2019-10-30 17:14:54,491 Epoch  13 Step:   113600 Batch Loss:     1.343386 Tokens per Sec:    16843, Lr: 0.000300\n",
      "2019-10-30 17:15:09,544 Epoch  13 Step:   113700 Batch Loss:     1.380540 Tokens per Sec:    17118, Lr: 0.000300\n",
      "2019-10-30 17:15:24,433 Epoch  13 Step:   113800 Batch Loss:     1.256028 Tokens per Sec:    17159, Lr: 0.000300\n",
      "2019-10-30 17:15:39,742 Epoch  13 Step:   113900 Batch Loss:     1.337033 Tokens per Sec:    17140, Lr: 0.000300\n",
      "2019-10-30 17:15:54,925 Epoch  13 Step:   114000 Batch Loss:     1.098175 Tokens per Sec:    16829, Lr: 0.000300\n",
      "2019-10-30 17:16:39,177 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:16:39,177 Saving new checkpoint.\n",
      "2019-10-30 17:16:39,327 Example #0\n",
      "2019-10-30 17:16:39,327 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:16:39,327 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:16:39,327 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:16:39,327 Example #1\n",
      "2019-10-30 17:16:39,327 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:16:39,327 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:16:39,327 \tHypothesis: Biblia haisemi kwamba watu hawana sifa ya kustahili .\n",
      "2019-10-30 17:16:39,327 Example #2\n",
      "2019-10-30 17:16:39,327 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:16:39,327 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:16:39,327 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 17:16:39,328 Example #3\n",
      "2019-10-30 17:16:39,328 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:16:39,328 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:16:39,328 \tHypothesis: Mapigo na busu .\n",
      "2019-10-30 17:16:39,328 Validation result at epoch  13, step   114000: bleu:  35.23, loss: 32020.2246, ppl:   3.2061, duration: 44.4019s\n",
      "2019-10-30 17:16:54,507 Epoch  13 Step:   114100 Batch Loss:     1.226511 Tokens per Sec:    16622, Lr: 0.000300\n",
      "2019-10-30 17:17:09,691 Epoch  13 Step:   114200 Batch Loss:     1.322062 Tokens per Sec:    17114, Lr: 0.000300\n",
      "2019-10-30 17:17:24,652 Epoch  13 Step:   114300 Batch Loss:     1.248587 Tokens per Sec:    17164, Lr: 0.000300\n",
      "2019-10-30 17:17:39,477 Epoch  13 Step:   114400 Batch Loss:     1.359211 Tokens per Sec:    16972, Lr: 0.000300\n",
      "2019-10-30 17:17:54,626 Epoch  13 Step:   114500 Batch Loss:     1.311956 Tokens per Sec:    17192, Lr: 0.000300\n",
      "2019-10-30 17:18:09,711 Epoch  13 Step:   114600 Batch Loss:     1.490162 Tokens per Sec:    17132, Lr: 0.000300\n",
      "2019-10-30 17:18:24,622 Epoch  13 Step:   114700 Batch Loss:     1.367454 Tokens per Sec:    17110, Lr: 0.000300\n",
      "2019-10-30 17:18:40,012 Epoch  13 Step:   114800 Batch Loss:     1.291623 Tokens per Sec:    16772, Lr: 0.000300\n",
      "2019-10-30 17:18:55,330 Epoch  13 Step:   114900 Batch Loss:     1.258110 Tokens per Sec:    17155, Lr: 0.000300\n",
      "2019-10-30 17:19:10,228 Epoch  13 Step:   115000 Batch Loss:     1.300854 Tokens per Sec:    17082, Lr: 0.000300\n",
      "2019-10-30 17:19:54,648 Example #0\n",
      "2019-10-30 17:19:54,648 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:19:54,648 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:19:54,648 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:19:54,648 Example #1\n",
      "2019-10-30 17:19:54,649 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:19:54,649 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:19:54,649 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 17:19:54,649 Example #2\n",
      "2019-10-30 17:19:54,649 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:19:54,649 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:19:54,649 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukuchochea nini ?\n",
      "2019-10-30 17:19:54,649 Example #3\n",
      "2019-10-30 17:19:54,649 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:19:54,649 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:19:54,649 \tHypothesis: Nyangumi na busu .\n",
      "2019-10-30 17:19:54,649 Validation result at epoch  13, step   115000: bleu:  35.49, loss: 32154.1641, ppl:   3.2217, duration: 44.4210s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 17:20:09,908 Epoch  13 Step:   115100 Batch Loss:     1.482404 Tokens per Sec:    16874, Lr: 0.000300\n",
      "2019-10-30 17:20:25,051 Epoch  13 Step:   115200 Batch Loss:     1.347613 Tokens per Sec:    16817, Lr: 0.000300\n",
      "2019-10-30 17:20:39,949 Epoch  13 Step:   115300 Batch Loss:     1.458443 Tokens per Sec:    17016, Lr: 0.000300\n",
      "2019-10-30 17:20:55,036 Epoch  13 Step:   115400 Batch Loss:     1.156127 Tokens per Sec:    17274, Lr: 0.000300\n",
      "2019-10-30 17:21:10,042 Epoch  13 Step:   115500 Batch Loss:     1.413334 Tokens per Sec:    17261, Lr: 0.000300\n",
      "2019-10-30 17:21:25,212 Epoch  13 Step:   115600 Batch Loss:     1.340074 Tokens per Sec:    16971, Lr: 0.000300\n",
      "2019-10-30 17:21:40,445 Epoch  13 Step:   115700 Batch Loss:     1.253044 Tokens per Sec:    16832, Lr: 0.000300\n",
      "2019-10-30 17:21:55,508 Epoch  13 Step:   115800 Batch Loss:     1.392222 Tokens per Sec:    17085, Lr: 0.000300\n",
      "2019-10-30 17:22:10,753 Epoch  13 Step:   115900 Batch Loss:     1.524449 Tokens per Sec:    16952, Lr: 0.000300\n",
      "2019-10-30 17:22:25,839 Epoch  13 Step:   116000 Batch Loss:     1.386536 Tokens per Sec:    16913, Lr: 0.000300\n",
      "2019-10-30 17:23:09,972 Example #0\n",
      "2019-10-30 17:23:09,972 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:23:09,972 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:23:09,972 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:23:09,972 Example #1\n",
      "2019-10-30 17:23:09,972 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:23:09,972 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:23:09,972 \tHypothesis: Biblia haidhuru sifa ya uthabiti .\n",
      "2019-10-30 17:23:09,972 Example #2\n",
      "2019-10-30 17:23:09,972 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:23:09,972 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:23:09,972 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 17:23:09,972 Example #3\n",
      "2019-10-30 17:23:09,973 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:23:09,973 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:23:09,973 \tHypothesis: Upendo wa makumbatio na busu .\n",
      "2019-10-30 17:23:09,973 Validation result at epoch  13, step   116000: bleu:  35.53, loss: 32157.8750, ppl:   3.2222, duration: 44.1334s\n",
      "2019-10-30 17:23:24,988 Epoch  13 Step:   116100 Batch Loss:     1.381131 Tokens per Sec:    17197, Lr: 0.000300\n",
      "2019-10-30 17:23:39,954 Epoch  13 Step:   116200 Batch Loss:     1.406007 Tokens per Sec:    17107, Lr: 0.000300\n",
      "2019-10-30 17:23:55,099 Epoch  13 Step:   116300 Batch Loss:     1.338724 Tokens per Sec:    17118, Lr: 0.000300\n",
      "2019-10-30 17:24:10,282 Epoch  13 Step:   116400 Batch Loss:     1.395381 Tokens per Sec:    16739, Lr: 0.000300\n",
      "2019-10-30 17:24:25,533 Epoch  13 Step:   116500 Batch Loss:     1.183536 Tokens per Sec:    17373, Lr: 0.000300\n",
      "2019-10-30 17:24:40,623 Epoch  13 Step:   116600 Batch Loss:     1.277345 Tokens per Sec:    16770, Lr: 0.000300\n",
      "2019-10-30 17:24:55,740 Epoch  13 Step:   116700 Batch Loss:     1.333676 Tokens per Sec:    16851, Lr: 0.000300\n",
      "2019-10-30 17:25:10,949 Epoch  13 Step:   116800 Batch Loss:     1.271517 Tokens per Sec:    16836, Lr: 0.000300\n",
      "2019-10-30 17:25:25,992 Epoch  13 Step:   116900 Batch Loss:     1.328204 Tokens per Sec:    17038, Lr: 0.000300\n",
      "2019-10-30 17:25:41,056 Epoch  13 Step:   117000 Batch Loss:     1.118400 Tokens per Sec:    16778, Lr: 0.000300\n",
      "2019-10-30 17:26:25,475 Example #0\n",
      "2019-10-30 17:26:25,475 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:26:25,475 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:26:25,475 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:26:25,475 Example #1\n",
      "2019-10-30 17:26:25,475 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:26:25,475 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:26:25,475 \tHypothesis: Biblia haidhoofishi utukufu .\n",
      "2019-10-30 17:26:25,475 Example #2\n",
      "2019-10-30 17:26:25,475 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:26:25,475 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:26:25,475 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 17:26:25,475 Example #3\n",
      "2019-10-30 17:26:25,476 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:26:25,476 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:26:25,476 \tHypothesis: Wapenzi wa makumbatio na mabusu .\n",
      "2019-10-30 17:26:25,476 Validation result at epoch  13, step   117000: bleu:  35.68, loss: 32123.2109, ppl:   3.2181, duration: 44.4192s\n",
      "2019-10-30 17:26:40,499 Epoch  13 Step:   117100 Batch Loss:     1.287542 Tokens per Sec:    17094, Lr: 0.000300\n",
      "2019-10-30 17:26:55,682 Epoch  13 Step:   117200 Batch Loss:     1.555461 Tokens per Sec:    17134, Lr: 0.000300\n",
      "2019-10-30 17:27:10,856 Epoch  13 Step:   117300 Batch Loss:     1.194169 Tokens per Sec:    16853, Lr: 0.000300\n",
      "2019-10-30 17:27:26,113 Epoch  13 Step:   117400 Batch Loss:     1.298811 Tokens per Sec:    17126, Lr: 0.000300\n",
      "2019-10-30 17:27:41,398 Epoch  13 Step:   117500 Batch Loss:     1.202428 Tokens per Sec:    16970, Lr: 0.000300\n",
      "2019-10-30 17:27:56,490 Epoch  13 Step:   117600 Batch Loss:     1.307809 Tokens per Sec:    17064, Lr: 0.000300\n",
      "2019-10-30 17:28:11,509 Epoch  13 Step:   117700 Batch Loss:     1.917832 Tokens per Sec:    16740, Lr: 0.000300\n",
      "2019-10-30 17:28:26,425 Epoch  13 Step:   117800 Batch Loss:     1.368488 Tokens per Sec:    16897, Lr: 0.000300\n",
      "2019-10-30 17:28:41,506 Epoch  13 Step:   117900 Batch Loss:     1.419657 Tokens per Sec:    17076, Lr: 0.000300\n",
      "2019-10-30 17:28:56,774 Epoch  13 Step:   118000 Batch Loss:     1.278239 Tokens per Sec:    17070, Lr: 0.000300\n",
      "2019-10-30 17:29:40,943 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:29:40,943 Saving new checkpoint.\n",
      "2019-10-30 17:29:41,091 Example #0\n",
      "2019-10-30 17:29:41,091 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:29:41,091 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:29:41,091 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:29:41,091 Example #1\n",
      "2019-10-30 17:29:41,091 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:29:41,091 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:29:41,091 \tHypothesis: Biblia haisemi kwamba mtu hawezi kuendeleza sifa ya utoaji - mimba .\n",
      "2019-10-30 17:29:41,091 Example #2\n",
      "2019-10-30 17:29:41,091 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:29:41,091 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:29:41,091 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na hangaiko laweza kukukufanyaje ?\n",
      "2019-10-30 17:29:41,091 Example #3\n",
      "2019-10-30 17:29:41,092 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:29:41,092 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:29:41,092 \tHypothesis: Wapenzi wa makumbatio na mabusu .\n",
      "2019-10-30 17:29:41,092 Validation result at epoch  13, step   118000: bleu:  35.36, loss: 31925.0820, ppl:   3.1950, duration: 44.3169s\n",
      "2019-10-30 17:29:55,946 Epoch  13 Step:   118100 Batch Loss:     1.268355 Tokens per Sec:    17006, Lr: 0.000300\n",
      "2019-10-30 17:30:11,076 Epoch  13 Step:   118200 Batch Loss:     1.294538 Tokens per Sec:    17195, Lr: 0.000300\n",
      "2019-10-30 17:30:26,361 Epoch  13 Step:   118300 Batch Loss:     1.441674 Tokens per Sec:    17052, Lr: 0.000300\n",
      "2019-10-30 17:30:41,565 Epoch  13 Step:   118400 Batch Loss:     1.315348 Tokens per Sec:    16755, Lr: 0.000300\n",
      "2019-10-30 17:30:56,666 Epoch  13 Step:   118500 Batch Loss:     1.327962 Tokens per Sec:    16955, Lr: 0.000300\n",
      "2019-10-30 17:31:11,854 Epoch  13 Step:   118600 Batch Loss:     1.614837 Tokens per Sec:    17282, Lr: 0.000300\n",
      "2019-10-30 17:31:26,894 Epoch  13 Step:   118700 Batch Loss:     1.324627 Tokens per Sec:    17051, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 17:31:41,791 Epoch  13 Step:   118800 Batch Loss:     1.495188 Tokens per Sec:    16923, Lr: 0.000300\n",
      "2019-10-30 17:31:56,842 Epoch  13 Step:   118900 Batch Loss:     1.498140 Tokens per Sec:    17141, Lr: 0.000300\n",
      "2019-10-30 17:32:11,812 Epoch  13 Step:   119000 Batch Loss:     1.276235 Tokens per Sec:    17027, Lr: 0.000300\n",
      "2019-10-30 17:32:55,845 Example #0\n",
      "2019-10-30 17:32:55,845 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:32:55,845 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:32:55,845 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:32:55,845 Example #1\n",
      "2019-10-30 17:32:55,846 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:32:55,846 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:32:55,846 \tHypothesis: Biblia haidhoofishi sifa ya uthabiti .\n",
      "2019-10-30 17:32:55,846 Example #2\n",
      "2019-10-30 17:32:55,846 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:32:55,846 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:32:55,846 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 17:32:55,846 Example #3\n",
      "2019-10-30 17:32:55,846 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:32:55,846 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:32:55,846 \tHypothesis: Wapenzi wa makumbatio na mabusu .\n",
      "2019-10-30 17:32:55,846 Validation result at epoch  13, step   119000: bleu:  35.59, loss: 32032.6211, ppl:   3.2075, duration: 44.0338s\n",
      "2019-10-30 17:33:10,909 Epoch  13 Step:   119100 Batch Loss:     1.281529 Tokens per Sec:    17234, Lr: 0.000300\n",
      "2019-10-30 17:33:26,152 Epoch  13 Step:   119200 Batch Loss:     1.258335 Tokens per Sec:    16851, Lr: 0.000300\n",
      "2019-10-30 17:33:41,521 Epoch  13 Step:   119300 Batch Loss:     1.392522 Tokens per Sec:    16763, Lr: 0.000300\n",
      "2019-10-30 17:33:56,779 Epoch  13 Step:   119400 Batch Loss:     1.473634 Tokens per Sec:    16944, Lr: 0.000300\n",
      "2019-10-30 17:34:12,144 Epoch  13 Step:   119500 Batch Loss:     1.307290 Tokens per Sec:    17108, Lr: 0.000300\n",
      "2019-10-30 17:34:27,021 Epoch  13 Step:   119600 Batch Loss:     1.371415 Tokens per Sec:    16735, Lr: 0.000300\n",
      "2019-10-30 17:34:42,211 Epoch  13 Step:   119700 Batch Loss:     1.196831 Tokens per Sec:    16965, Lr: 0.000300\n",
      "2019-10-30 17:34:57,269 Epoch  13 Step:   119800 Batch Loss:     1.358566 Tokens per Sec:    16799, Lr: 0.000300\n",
      "2019-10-30 17:35:12,382 Epoch  13 Step:   119900 Batch Loss:     1.412742 Tokens per Sec:    16846, Lr: 0.000300\n",
      "2019-10-30 17:35:27,438 Epoch  13 Step:   120000 Batch Loss:     1.375274 Tokens per Sec:    17095, Lr: 0.000300\n",
      "2019-10-30 17:36:11,871 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:36:11,872 Saving new checkpoint.\n",
      "2019-10-30 17:36:12,013 Example #0\n",
      "2019-10-30 17:36:12,013 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:36:12,013 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:36:12,013 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:36:12,013 Example #1\n",
      "2019-10-30 17:36:12,014 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:36:12,014 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:36:12,014 \tHypothesis: Biblia haidhoofishi sifa ya utoaji - sifa .\n",
      "2019-10-30 17:36:12,014 Example #2\n",
      "2019-10-30 17:36:12,014 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:36:12,014 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:36:12,014 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na hangaiko laweza kutokea ?\n",
      "2019-10-30 17:36:12,014 Example #3\n",
      "2019-10-30 17:36:12,014 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:36:12,014 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:36:12,014 \tHypothesis: Mapigo ya maua na mabusu .\n",
      "2019-10-30 17:36:12,014 Validation result at epoch  13, step   120000: bleu:  35.47, loss: 31874.0664, ppl:   3.1891, duration: 44.5761s\n",
      "2019-10-30 17:36:27,174 Epoch  13 Step:   120100 Batch Loss:     1.198982 Tokens per Sec:    17362, Lr: 0.000300\n",
      "2019-10-30 17:36:42,289 Epoch  13 Step:   120200 Batch Loss:     1.230417 Tokens per Sec:    17145, Lr: 0.000300\n",
      "2019-10-30 17:36:57,404 Epoch  13 Step:   120300 Batch Loss:     1.219208 Tokens per Sec:    16984, Lr: 0.000300\n",
      "2019-10-30 17:37:12,372 Epoch  13 Step:   120400 Batch Loss:     1.351052 Tokens per Sec:    17182, Lr: 0.000300\n",
      "2019-10-30 17:37:27,359 Epoch  13 Step:   120500 Batch Loss:     1.418652 Tokens per Sec:    16748, Lr: 0.000300\n",
      "2019-10-30 17:37:42,333 Epoch  13 Step:   120600 Batch Loss:     1.194305 Tokens per Sec:    17000, Lr: 0.000300\n",
      "2019-10-30 17:37:57,677 Epoch  13 Step:   120700 Batch Loss:     1.287249 Tokens per Sec:    16658, Lr: 0.000300\n",
      "2019-10-30 17:38:12,810 Epoch  13 Step:   120800 Batch Loss:     1.529994 Tokens per Sec:    16615, Lr: 0.000300\n",
      "2019-10-30 17:38:28,011 Epoch  13 Step:   120900 Batch Loss:     1.528515 Tokens per Sec:    17121, Lr: 0.000300\n",
      "2019-10-30 17:38:43,214 Epoch  13 Step:   121000 Batch Loss:     1.398746 Tokens per Sec:    16525, Lr: 0.000300\n",
      "2019-10-30 17:39:27,424 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:39:27,425 Saving new checkpoint.\n",
      "2019-10-30 17:39:27,572 Example #0\n",
      "2019-10-30 17:39:27,572 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:39:27,572 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:39:27,572 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:39:27,573 Example #1\n",
      "2019-10-30 17:39:27,573 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:39:27,573 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:39:27,573 \tHypothesis: Biblia haidhihirishi uthabiti .\n",
      "2019-10-30 17:39:27,573 Example #2\n",
      "2019-10-30 17:39:27,573 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:39:27,573 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:39:27,573 \tHypothesis: Ni baadhi ya mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 17:39:27,573 Example #3\n",
      "2019-10-30 17:39:27,573 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:39:27,573 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:39:27,573 \tHypothesis: Mapenzi ya makumbatio na mabusu .\n",
      "2019-10-30 17:39:27,573 Validation result at epoch  13, step   121000: bleu:  35.70, loss: 31844.5703, ppl:   3.1857, duration: 44.3588s\n",
      "2019-10-30 17:39:42,710 Epoch  13 Step:   121100 Batch Loss:     1.328076 Tokens per Sec:    16993, Lr: 0.000300\n",
      "2019-10-30 17:39:57,581 Epoch  13 Step:   121200 Batch Loss:     1.245093 Tokens per Sec:    17026, Lr: 0.000300\n",
      "2019-10-30 17:40:12,584 Epoch  13 Step:   121300 Batch Loss:     1.550025 Tokens per Sec:    17178, Lr: 0.000300\n",
      "2019-10-30 17:40:27,849 Epoch  13 Step:   121400 Batch Loss:     1.239316 Tokens per Sec:    17077, Lr: 0.000300\n",
      "2019-10-30 17:40:42,889 Epoch  13 Step:   121500 Batch Loss:     1.483409 Tokens per Sec:    16893, Lr: 0.000300\n",
      "2019-10-30 17:40:57,752 Epoch  13 Step:   121600 Batch Loss:     1.214568 Tokens per Sec:    17229, Lr: 0.000300\n",
      "2019-10-30 17:41:12,976 Epoch  13 Step:   121700 Batch Loss:     1.242532 Tokens per Sec:    16926, Lr: 0.000300\n",
      "2019-10-30 17:41:28,050 Epoch  13 Step:   121800 Batch Loss:     1.288401 Tokens per Sec:    17144, Lr: 0.000300\n",
      "2019-10-30 17:41:43,151 Epoch  13 Step:   121900 Batch Loss:     1.207478 Tokens per Sec:    16869, Lr: 0.000300\n",
      "2019-10-30 17:41:58,337 Epoch  13 Step:   122000 Batch Loss:     1.284312 Tokens per Sec:    16998, Lr: 0.000300\n",
      "2019-10-30 17:42:42,453 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:42:42,454 Saving new checkpoint.\n",
      "2019-10-30 17:42:42,597 Example #0\n",
      "2019-10-30 17:42:42,597 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:42:42,597 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:42:42,597 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:42:42,597 Example #1\n",
      "2019-10-30 17:42:42,597 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:42:42,597 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:42:42,597 \tHypothesis: Biblia haidhuru sifa ya upole .\n",
      "2019-10-30 17:42:42,597 Example #2\n",
      "2019-10-30 17:42:42,597 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:42:42,597 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:42:42,597 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 17:42:42,597 Example #3\n",
      "2019-10-30 17:42:42,598 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:42:42,598 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:42:42,598 \tHypothesis: Mapigo ya maua na busu .\n",
      "2019-10-30 17:42:42,598 Validation result at epoch  13, step   122000: bleu:  35.55, loss: 31838.4434, ppl:   3.1849, duration: 44.2607s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 17:42:42,777 Epoch  13: total training loss 12349.15\n",
      "2019-10-30 17:42:42,777 EPOCH 14\n",
      "2019-10-30 17:42:59,059 Epoch  14 Step:   122100 Batch Loss:     1.532502 Tokens per Sec:    15574, Lr: 0.000300\n",
      "2019-10-30 17:43:14,137 Epoch  14 Step:   122200 Batch Loss:     1.230187 Tokens per Sec:    17069, Lr: 0.000300\n",
      "2019-10-30 17:43:29,346 Epoch  14 Step:   122300 Batch Loss:     1.464883 Tokens per Sec:    17117, Lr: 0.000300\n",
      "2019-10-30 17:43:44,226 Epoch  14 Step:   122400 Batch Loss:     1.220448 Tokens per Sec:    17029, Lr: 0.000300\n",
      "2019-10-30 17:43:59,270 Epoch  14 Step:   122500 Batch Loss:     1.543728 Tokens per Sec:    17142, Lr: 0.000300\n",
      "2019-10-30 17:44:14,352 Epoch  14 Step:   122600 Batch Loss:     1.422091 Tokens per Sec:    16955, Lr: 0.000300\n",
      "2019-10-30 17:44:29,316 Epoch  14 Step:   122700 Batch Loss:     1.248504 Tokens per Sec:    16757, Lr: 0.000300\n",
      "2019-10-30 17:44:44,569 Epoch  14 Step:   122800 Batch Loss:     1.303248 Tokens per Sec:    16892, Lr: 0.000300\n",
      "2019-10-30 17:44:59,746 Epoch  14 Step:   122900 Batch Loss:     1.427370 Tokens per Sec:    16858, Lr: 0.000300\n",
      "2019-10-30 17:45:14,911 Epoch  14 Step:   123000 Batch Loss:     1.197733 Tokens per Sec:    17077, Lr: 0.000300\n",
      "2019-10-30 17:45:59,262 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:45:59,262 Saving new checkpoint.\n",
      "2019-10-30 17:45:59,406 Example #0\n",
      "2019-10-30 17:45:59,406 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:45:59,406 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:45:59,406 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:45:59,406 Example #1\n",
      "2019-10-30 17:45:59,406 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:45:59,406 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:45:59,406 \tHypothesis: Biblia haidhoofishi sifa .\n",
      "2019-10-30 17:45:59,406 Example #2\n",
      "2019-10-30 17:45:59,406 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:45:59,406 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:45:59,406 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 17:45:59,406 Example #3\n",
      "2019-10-30 17:45:59,407 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:45:59,407 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:45:59,407 \tHypothesis: Mapigo ya maua na mabusu .\n",
      "2019-10-30 17:45:59,407 Validation result at epoch  14, step   123000: bleu:  35.50, loss: 31738.6523, ppl:   3.1734, duration: 44.4948s\n",
      "2019-10-30 17:46:14,680 Epoch  14 Step:   123100 Batch Loss:     1.565000 Tokens per Sec:    17139, Lr: 0.000300\n",
      "2019-10-30 17:46:29,693 Epoch  14 Step:   123200 Batch Loss:     1.296771 Tokens per Sec:    16727, Lr: 0.000300\n",
      "2019-10-30 17:46:44,860 Epoch  14 Step:   123300 Batch Loss:     1.499552 Tokens per Sec:    16832, Lr: 0.000300\n",
      "2019-10-30 17:47:00,155 Epoch  14 Step:   123400 Batch Loss:     1.123274 Tokens per Sec:    17102, Lr: 0.000300\n",
      "2019-10-30 17:47:15,284 Epoch  14 Step:   123500 Batch Loss:     1.185047 Tokens per Sec:    16794, Lr: 0.000300\n",
      "2019-10-30 17:47:30,443 Epoch  14 Step:   123600 Batch Loss:     1.356540 Tokens per Sec:    17323, Lr: 0.000300\n",
      "2019-10-30 17:47:45,560 Epoch  14 Step:   123700 Batch Loss:     1.446604 Tokens per Sec:    16706, Lr: 0.000300\n",
      "2019-10-30 17:48:00,727 Epoch  14 Step:   123800 Batch Loss:     1.396621 Tokens per Sec:    16768, Lr: 0.000300\n",
      "2019-10-30 17:48:16,072 Epoch  14 Step:   123900 Batch Loss:     1.322527 Tokens per Sec:    16814, Lr: 0.000300\n",
      "2019-10-30 17:48:31,312 Epoch  14 Step:   124000 Batch Loss:     1.413499 Tokens per Sec:    16560, Lr: 0.000300\n",
      "2019-10-30 17:49:15,668 Example #0\n",
      "2019-10-30 17:49:15,669 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:49:15,669 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:49:15,669 \tHypothesis: Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:49:15,669 Example #1\n",
      "2019-10-30 17:49:15,669 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:49:15,669 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:49:15,669 \tHypothesis: Biblia haidhihirishi sifa ya kustahili .\n",
      "2019-10-30 17:49:15,669 Example #2\n",
      "2019-10-30 17:49:15,669 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:49:15,669 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:49:15,669 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukuchochea nini ?\n",
      "2019-10-30 17:49:15,669 Example #3\n",
      "2019-10-30 17:49:15,669 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:49:15,669 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:49:15,669 \tHypothesis: Miguu ya maua na mabusu .\n",
      "2019-10-30 17:49:15,670 Validation result at epoch  14, step   124000: bleu:  35.33, loss: 31784.0723, ppl:   3.1787, duration: 44.3572s\n",
      "2019-10-30 17:49:31,150 Epoch  14 Step:   124100 Batch Loss:     1.184392 Tokens per Sec:    17081, Lr: 0.000300\n",
      "2019-10-30 17:49:46,277 Epoch  14 Step:   124200 Batch Loss:     1.233084 Tokens per Sec:    17050, Lr: 0.000300\n",
      "2019-10-30 17:50:01,199 Epoch  14 Step:   124300 Batch Loss:     1.397820 Tokens per Sec:    17049, Lr: 0.000300\n",
      "2019-10-30 17:50:16,642 Epoch  14 Step:   124400 Batch Loss:     1.249109 Tokens per Sec:    16999, Lr: 0.000300\n",
      "2019-10-30 17:50:31,641 Epoch  14 Step:   124500 Batch Loss:     1.286895 Tokens per Sec:    16511, Lr: 0.000300\n",
      "2019-10-30 17:50:46,870 Epoch  14 Step:   124600 Batch Loss:     1.168995 Tokens per Sec:    16997, Lr: 0.000300\n",
      "2019-10-30 17:51:02,054 Epoch  14 Step:   124700 Batch Loss:     1.323032 Tokens per Sec:    16875, Lr: 0.000300\n",
      "2019-10-30 17:51:17,377 Epoch  14 Step:   124800 Batch Loss:     1.282068 Tokens per Sec:    17003, Lr: 0.000300\n",
      "2019-10-30 17:51:32,587 Epoch  14 Step:   124900 Batch Loss:     1.346549 Tokens per Sec:    16937, Lr: 0.000300\n",
      "2019-10-30 17:51:47,507 Epoch  14 Step:   125000 Batch Loss:     1.504567 Tokens per Sec:    16893, Lr: 0.000300\n",
      "2019-10-30 17:52:31,930 Example #0\n",
      "2019-10-30 17:52:31,931 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:52:31,931 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:52:31,931 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:52:31,931 Example #1\n",
      "2019-10-30 17:52:31,931 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:52:31,931 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:52:31,931 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 17:52:31,931 Example #2\n",
      "2019-10-30 17:52:31,931 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:52:31,931 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:52:31,931 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 17:52:31,931 Example #3\n",
      "2019-10-30 17:52:31,931 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:52:31,931 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:52:31,932 \tHypothesis: Miti ya maua na busu .\n",
      "2019-10-30 17:52:31,932 Validation result at epoch  14, step   125000: bleu:  35.69, loss: 31742.4473, ppl:   3.1738, duration: 44.4246s\n",
      "2019-10-30 17:52:47,090 Epoch  14 Step:   125100 Batch Loss:     1.051620 Tokens per Sec:    17020, Lr: 0.000300\n",
      "2019-10-30 17:53:02,255 Epoch  14 Step:   125200 Batch Loss:     1.171691 Tokens per Sec:    16826, Lr: 0.000300\n",
      "2019-10-30 17:53:17,734 Epoch  14 Step:   125300 Batch Loss:     1.158235 Tokens per Sec:    16663, Lr: 0.000300\n",
      "2019-10-30 17:53:32,937 Epoch  14 Step:   125400 Batch Loss:     1.407431 Tokens per Sec:    17221, Lr: 0.000300\n",
      "2019-10-30 17:53:48,127 Epoch  14 Step:   125500 Batch Loss:     1.451397 Tokens per Sec:    16835, Lr: 0.000300\n",
      "2019-10-30 17:54:03,366 Epoch  14 Step:   125600 Batch Loss:     1.159057 Tokens per Sec:    17288, Lr: 0.000300\n",
      "2019-10-30 17:54:18,489 Epoch  14 Step:   125700 Batch Loss:     1.346475 Tokens per Sec:    16823, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 17:54:33,423 Epoch  14 Step:   125800 Batch Loss:     1.410442 Tokens per Sec:    17027, Lr: 0.000300\n",
      "2019-10-30 17:54:48,517 Epoch  14 Step:   125900 Batch Loss:     1.782229 Tokens per Sec:    17027, Lr: 0.000300\n",
      "2019-10-30 17:55:03,670 Epoch  14 Step:   126000 Batch Loss:     1.236800 Tokens per Sec:    16966, Lr: 0.000300\n",
      "2019-10-30 17:55:48,127 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 17:55:48,127 Saving new checkpoint.\n",
      "2019-10-30 17:55:48,277 Example #0\n",
      "2019-10-30 17:55:48,277 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:55:48,277 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:55:48,277 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:55:48,277 Example #1\n",
      "2019-10-30 17:55:48,278 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:55:48,278 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:55:48,278 \tHypothesis: Biblia haidhihirishi sifa ya sifa .\n",
      "2019-10-30 17:55:48,278 Example #2\n",
      "2019-10-30 17:55:48,278 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:55:48,278 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:55:48,278 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 17:55:48,278 Example #3\n",
      "2019-10-30 17:55:48,278 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:55:48,278 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:55:48,278 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 17:55:48,278 Validation result at epoch  14, step   126000: bleu:  35.82, loss: 31644.5703, ppl:   3.1626, duration: 44.6078s\n",
      "2019-10-30 17:56:03,336 Epoch  14 Step:   126100 Batch Loss:     1.641344 Tokens per Sec:    16918, Lr: 0.000300\n",
      "2019-10-30 17:56:18,459 Epoch  14 Step:   126200 Batch Loss:     1.500421 Tokens per Sec:    17167, Lr: 0.000300\n",
      "2019-10-30 17:56:33,563 Epoch  14 Step:   126300 Batch Loss:     1.483075 Tokens per Sec:    16973, Lr: 0.000300\n",
      "2019-10-30 17:56:48,599 Epoch  14 Step:   126400 Batch Loss:     1.429587 Tokens per Sec:    16463, Lr: 0.000300\n",
      "2019-10-30 17:57:03,926 Epoch  14 Step:   126500 Batch Loss:     1.158798 Tokens per Sec:    16996, Lr: 0.000300\n",
      "2019-10-30 17:57:19,010 Epoch  14 Step:   126600 Batch Loss:     1.276918 Tokens per Sec:    17071, Lr: 0.000300\n",
      "2019-10-30 17:57:34,042 Epoch  14 Step:   126700 Batch Loss:     1.165095 Tokens per Sec:    17157, Lr: 0.000300\n",
      "2019-10-30 17:57:49,300 Epoch  14 Step:   126800 Batch Loss:     1.198633 Tokens per Sec:    17132, Lr: 0.000300\n",
      "2019-10-30 17:58:04,270 Epoch  14 Step:   126900 Batch Loss:     1.204591 Tokens per Sec:    17007, Lr: 0.000300\n",
      "2019-10-30 17:58:19,255 Epoch  14 Step:   127000 Batch Loss:     1.326641 Tokens per Sec:    16667, Lr: 0.000300\n",
      "2019-10-30 17:59:03,302 Example #0\n",
      "2019-10-30 17:59:03,303 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 17:59:03,303 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 17:59:03,303 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 17:59:03,303 Example #1\n",
      "2019-10-30 17:59:03,303 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 17:59:03,303 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 17:59:03,303 \tHypothesis: Biblia haisemi kwamba watu wana sifa nzuri .\n",
      "2019-10-30 17:59:03,303 Example #2\n",
      "2019-10-30 17:59:03,303 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 17:59:03,303 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 17:59:03,303 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 17:59:03,303 Example #3\n",
      "2019-10-30 17:59:03,303 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 17:59:03,303 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 17:59:03,304 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 17:59:03,304 Validation result at epoch  14, step   127000: bleu:  36.15, loss: 31720.3320, ppl:   3.1713, duration: 44.0480s\n",
      "2019-10-30 17:59:18,582 Epoch  14 Step:   127100 Batch Loss:     1.303728 Tokens per Sec:    16905, Lr: 0.000300\n",
      "2019-10-30 17:59:33,798 Epoch  14 Step:   127200 Batch Loss:     1.335613 Tokens per Sec:    17142, Lr: 0.000300\n",
      "2019-10-30 17:59:48,972 Epoch  14 Step:   127300 Batch Loss:     1.434633 Tokens per Sec:    17303, Lr: 0.000300\n",
      "2019-10-30 18:00:04,220 Epoch  14 Step:   127400 Batch Loss:     1.232390 Tokens per Sec:    16794, Lr: 0.000300\n",
      "2019-10-30 18:00:19,114 Epoch  14 Step:   127500 Batch Loss:     1.471709 Tokens per Sec:    17122, Lr: 0.000300\n",
      "2019-10-30 18:00:34,085 Epoch  14 Step:   127600 Batch Loss:     1.330404 Tokens per Sec:    16620, Lr: 0.000300\n",
      "2019-10-30 18:00:49,243 Epoch  14 Step:   127700 Batch Loss:     1.506830 Tokens per Sec:    16992, Lr: 0.000300\n",
      "2019-10-30 18:01:04,803 Epoch  14 Step:   127800 Batch Loss:     1.539775 Tokens per Sec:    16518, Lr: 0.000300\n",
      "2019-10-30 18:01:19,881 Epoch  14 Step:   127900 Batch Loss:     1.299128 Tokens per Sec:    16556, Lr: 0.000300\n",
      "2019-10-30 18:01:35,122 Epoch  14 Step:   128000 Batch Loss:     1.398485 Tokens per Sec:    16686, Lr: 0.000300\n",
      "2019-10-30 18:02:19,453 Example #0\n",
      "2019-10-30 18:02:19,453 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:02:19,453 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:02:19,453 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:02:19,454 Example #1\n",
      "2019-10-30 18:02:19,454 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:02:19,454 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:02:19,454 \tHypothesis: Biblia haidhihirishi sifa ya ustahili .\n",
      "2019-10-30 18:02:19,454 Example #2\n",
      "2019-10-30 18:02:19,454 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:02:19,454 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:02:19,454 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 18:02:19,454 Example #3\n",
      "2019-10-30 18:02:19,454 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:02:19,454 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:02:19,454 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 18:02:19,454 Validation result at epoch  14, step   128000: bleu:  35.92, loss: 31720.5684, ppl:   3.1713, duration: 44.3320s\n",
      "2019-10-30 18:02:34,705 Epoch  14 Step:   128100 Batch Loss:     1.258367 Tokens per Sec:    17007, Lr: 0.000300\n",
      "2019-10-30 18:02:50,033 Epoch  14 Step:   128200 Batch Loss:     1.456009 Tokens per Sec:    17092, Lr: 0.000300\n",
      "2019-10-30 18:03:05,255 Epoch  14 Step:   128300 Batch Loss:     1.311596 Tokens per Sec:    16586, Lr: 0.000300\n",
      "2019-10-30 18:03:20,774 Epoch  14 Step:   128400 Batch Loss:     1.404663 Tokens per Sec:    16885, Lr: 0.000300\n",
      "2019-10-30 18:03:35,621 Epoch  14 Step:   128500 Batch Loss:     1.608064 Tokens per Sec:    16661, Lr: 0.000300\n",
      "2019-10-30 18:03:50,730 Epoch  14 Step:   128600 Batch Loss:     1.200250 Tokens per Sec:    17229, Lr: 0.000300\n",
      "2019-10-30 18:04:05,910 Epoch  14 Step:   128700 Batch Loss:     1.396808 Tokens per Sec:    16840, Lr: 0.000300\n",
      "2019-10-30 18:04:21,028 Epoch  14 Step:   128800 Batch Loss:     1.234158 Tokens per Sec:    16995, Lr: 0.000300\n",
      "2019-10-30 18:04:36,350 Epoch  14 Step:   128900 Batch Loss:     1.392890 Tokens per Sec:    16914, Lr: 0.000300\n",
      "2019-10-30 18:04:51,399 Epoch  14 Step:   129000 Batch Loss:     1.414642 Tokens per Sec:    17110, Lr: 0.000300\n",
      "2019-10-30 18:05:35,720 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:05:35,720 Saving new checkpoint.\n",
      "2019-10-30 18:05:35,862 Example #0\n",
      "2019-10-30 18:05:35,863 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:05:35,863 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:05:35,863 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:05:35,863 Example #1\n",
      "2019-10-30 18:05:35,863 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:05:35,863 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:05:35,863 \tHypothesis: Biblia haishutumu sifa .\n",
      "2019-10-30 18:05:35,863 Example #2\n",
      "2019-10-30 18:05:35,863 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:05:35,863 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:05:35,863 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 18:05:35,863 Example #3\n",
      "2019-10-30 18:05:35,863 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:05:35,863 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:05:35,863 \tHypothesis: Nyasi za maua na busu .\n",
      "2019-10-30 18:05:35,863 Validation result at epoch  14, step   129000: bleu:  35.65, loss: 31583.3398, ppl:   3.1555, duration: 44.4643s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 18:05:50,931 Epoch  14 Step:   129100 Batch Loss:     1.319658 Tokens per Sec:    16819, Lr: 0.000300\n",
      "2019-10-30 18:06:05,884 Epoch  14 Step:   129200 Batch Loss:     1.333558 Tokens per Sec:    17179, Lr: 0.000300\n",
      "2019-10-30 18:06:21,042 Epoch  14 Step:   129300 Batch Loss:     1.208661 Tokens per Sec:    16820, Lr: 0.000300\n",
      "2019-10-30 18:06:36,318 Epoch  14 Step:   129400 Batch Loss:     1.429147 Tokens per Sec:    16910, Lr: 0.000300\n",
      "2019-10-30 18:06:51,449 Epoch  14 Step:   129500 Batch Loss:     1.072049 Tokens per Sec:    17024, Lr: 0.000300\n",
      "2019-10-30 18:07:06,740 Epoch  14 Step:   129600 Batch Loss:     1.547908 Tokens per Sec:    17086, Lr: 0.000300\n",
      "2019-10-30 18:07:22,116 Epoch  14 Step:   129700 Batch Loss:     1.659095 Tokens per Sec:    17169, Lr: 0.000300\n",
      "2019-10-30 18:07:37,015 Epoch  14 Step:   129800 Batch Loss:     1.404494 Tokens per Sec:    16991, Lr: 0.000300\n",
      "2019-10-30 18:07:52,346 Epoch  14 Step:   129900 Batch Loss:     1.386403 Tokens per Sec:    16566, Lr: 0.000300\n",
      "2019-10-30 18:08:07,373 Epoch  14 Step:   130000 Batch Loss:     1.198428 Tokens per Sec:    16812, Lr: 0.000300\n",
      "2019-10-30 18:08:51,665 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:08:51,665 Saving new checkpoint.\n",
      "2019-10-30 18:08:51,806 Example #0\n",
      "2019-10-30 18:08:51,806 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:08:51,806 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:08:51,806 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:08:51,806 Example #1\n",
      "2019-10-30 18:08:51,806 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:08:51,806 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:08:51,806 \tHypothesis: Biblia haidhihirishi sifa ya uthabiti .\n",
      "2019-10-30 18:08:51,806 Example #2\n",
      "2019-10-30 18:08:51,807 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:08:51,807 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:08:51,807 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 18:08:51,807 Example #3\n",
      "2019-10-30 18:08:51,807 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:08:51,807 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:08:51,807 \tHypothesis: Nyuki na mabusu .\n",
      "2019-10-30 18:08:51,807 Validation result at epoch  14, step   130000: bleu:  35.91, loss: 31557.4414, ppl:   3.1525, duration: 44.4329s\n",
      "2019-10-30 18:09:06,814 Epoch  14 Step:   130100 Batch Loss:     1.249543 Tokens per Sec:    16970, Lr: 0.000300\n",
      "2019-10-30 18:09:22,146 Epoch  14 Step:   130200 Batch Loss:     1.254049 Tokens per Sec:    16790, Lr: 0.000300\n",
      "2019-10-30 18:09:37,222 Epoch  14 Step:   130300 Batch Loss:     1.398398 Tokens per Sec:    16775, Lr: 0.000300\n",
      "2019-10-30 18:09:52,432 Epoch  14 Step:   130400 Batch Loss:     1.216218 Tokens per Sec:    17107, Lr: 0.000300\n",
      "2019-10-30 18:10:07,656 Epoch  14 Step:   130500 Batch Loss:     1.253473 Tokens per Sec:    17027, Lr: 0.000300\n",
      "2019-10-30 18:10:22,872 Epoch  14 Step:   130600 Batch Loss:     1.512534 Tokens per Sec:    16400, Lr: 0.000300\n",
      "2019-10-30 18:10:38,205 Epoch  14 Step:   130700 Batch Loss:     1.229877 Tokens per Sec:    17077, Lr: 0.000300\n",
      "2019-10-30 18:10:53,367 Epoch  14 Step:   130800 Batch Loss:     1.164653 Tokens per Sec:    16641, Lr: 0.000300\n",
      "2019-10-30 18:11:08,727 Epoch  14 Step:   130900 Batch Loss:     1.333302 Tokens per Sec:    17286, Lr: 0.000300\n",
      "2019-10-30 18:11:24,158 Epoch  14 Step:   131000 Batch Loss:     1.425178 Tokens per Sec:    16897, Lr: 0.000300\n",
      "2019-10-30 18:12:08,609 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:12:08,609 Saving new checkpoint.\n",
      "2019-10-30 18:12:08,751 Example #0\n",
      "2019-10-30 18:12:08,752 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:12:08,752 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:12:08,752 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:12:08,752 Example #1\n",
      "2019-10-30 18:12:08,752 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:12:08,752 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:12:08,752 \tHypothesis: Biblia haidhuru unyenyekevu .\n",
      "2019-10-30 18:12:08,752 Example #2\n",
      "2019-10-30 18:12:08,752 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:12:08,752 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:12:08,752 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokea nini ?\n",
      "2019-10-30 18:12:08,752 Example #3\n",
      "2019-10-30 18:12:08,752 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:12:08,752 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:12:08,753 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 18:12:08,753 Validation result at epoch  14, step   131000: bleu:  35.75, loss: 31527.2617, ppl:   3.1491, duration: 44.5937s\n",
      "2019-10-30 18:12:24,119 Epoch  14 Step:   131100 Batch Loss:     1.285885 Tokens per Sec:    16890, Lr: 0.000300\n",
      "2019-10-30 18:12:38,991 Epoch  14 Step:   131200 Batch Loss:     1.339574 Tokens per Sec:    17049, Lr: 0.000300\n",
      "2019-10-30 18:12:54,242 Epoch  14 Step:   131300 Batch Loss:     1.406360 Tokens per Sec:    16983, Lr: 0.000300\n",
      "2019-10-30 18:13:08,135 Epoch  14: total training loss 12215.52\n",
      "2019-10-30 18:13:08,136 EPOCH 15\n",
      "2019-10-30 18:13:10,715 Epoch  15 Step:   131400 Batch Loss:     1.260232 Tokens per Sec:     9282, Lr: 0.000300\n",
      "2019-10-30 18:13:26,092 Epoch  15 Step:   131500 Batch Loss:     1.294175 Tokens per Sec:    17012, Lr: 0.000300\n",
      "2019-10-30 18:13:41,231 Epoch  15 Step:   131600 Batch Loss:     1.388931 Tokens per Sec:    16702, Lr: 0.000300\n",
      "2019-10-30 18:13:56,481 Epoch  15 Step:   131700 Batch Loss:     1.349469 Tokens per Sec:    16892, Lr: 0.000300\n",
      "2019-10-30 18:14:11,804 Epoch  15 Step:   131800 Batch Loss:     1.267317 Tokens per Sec:    17186, Lr: 0.000300\n",
      "2019-10-30 18:14:27,063 Epoch  15 Step:   131900 Batch Loss:     1.192484 Tokens per Sec:    17130, Lr: 0.000300\n",
      "2019-10-30 18:14:42,388 Epoch  15 Step:   132000 Batch Loss:     1.318073 Tokens per Sec:    16917, Lr: 0.000300\n",
      "2019-10-30 18:15:27,103 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:15:27,103 Saving new checkpoint.\n",
      "2019-10-30 18:15:27,250 Example #0\n",
      "2019-10-30 18:15:27,250 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:15:27,250 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:15:27,250 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:15:27,250 Example #1\n",
      "2019-10-30 18:15:27,250 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:15:27,250 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:15:27,250 \tHypothesis: Biblia haidhihirishi uthibitisho .\n",
      "2019-10-30 18:15:27,250 Example #2\n",
      "2019-10-30 18:15:27,250 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:15:27,250 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:15:27,250 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:15:27,250 Example #3\n",
      "2019-10-30 18:15:27,251 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:15:27,251 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:15:27,251 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 18:15:27,251 Validation result at epoch  15, step   132000: bleu:  36.36, loss: 31488.2090, ppl:   3.1446, duration: 44.8623s\n",
      "2019-10-30 18:15:42,441 Epoch  15 Step:   132100 Batch Loss:     1.175014 Tokens per Sec:    17034, Lr: 0.000300\n",
      "2019-10-30 18:15:57,787 Epoch  15 Step:   132200 Batch Loss:     1.292079 Tokens per Sec:    16532, Lr: 0.000300\n",
      "2019-10-30 18:16:13,117 Epoch  15 Step:   132300 Batch Loss:     1.253967 Tokens per Sec:    16949, Lr: 0.000300\n",
      "2019-10-30 18:16:28,286 Epoch  15 Step:   132400 Batch Loss:     1.376677 Tokens per Sec:    16816, Lr: 0.000300\n",
      "2019-10-30 18:16:43,381 Epoch  15 Step:   132500 Batch Loss:     1.141666 Tokens per Sec:    16767, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 18:16:58,445 Epoch  15 Step:   132600 Batch Loss:     1.226578 Tokens per Sec:    16843, Lr: 0.000300\n",
      "2019-10-30 18:17:13,798 Epoch  15 Step:   132700 Batch Loss:     1.022012 Tokens per Sec:    16862, Lr: 0.000300\n",
      "2019-10-30 18:17:28,673 Epoch  15 Step:   132800 Batch Loss:     1.383803 Tokens per Sec:    16972, Lr: 0.000300\n",
      "2019-10-30 18:17:43,827 Epoch  15 Step:   132900 Batch Loss:     1.421976 Tokens per Sec:    17320, Lr: 0.000300\n",
      "2019-10-30 18:17:58,963 Epoch  15 Step:   133000 Batch Loss:     1.351618 Tokens per Sec:    16712, Lr: 0.000300\n",
      "2019-10-30 18:18:43,988 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:18:43,988 Saving new checkpoint.\n",
      "2019-10-30 18:18:44,138 Example #0\n",
      "2019-10-30 18:18:44,139 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:18:44,139 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:18:44,139 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:18:44,139 Example #1\n",
      "2019-10-30 18:18:44,139 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:18:44,139 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:18:44,139 \tHypothesis: Biblia haidhoofishi sifa ya ustahili .\n",
      "2019-10-30 18:18:44,139 Example #2\n",
      "2019-10-30 18:18:44,139 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:18:44,139 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:18:44,139 \tHypothesis: Ni mambo gani mengine yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 18:18:44,139 Example #3\n",
      "2019-10-30 18:18:44,139 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:18:44,140 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:18:44,140 \tHypothesis: Miguu ya maua na mabusu .\n",
      "2019-10-30 18:18:44,140 Validation result at epoch  15, step   133000: bleu:  36.09, loss: 31434.6992, ppl:   3.1385, duration: 45.1766s\n",
      "2019-10-30 18:18:59,257 Epoch  15 Step:   133100 Batch Loss:     1.300531 Tokens per Sec:    16701, Lr: 0.000300\n",
      "2019-10-30 18:19:14,373 Epoch  15 Step:   133200 Batch Loss:     1.223165 Tokens per Sec:    17209, Lr: 0.000300\n",
      "2019-10-30 18:19:29,520 Epoch  15 Step:   133300 Batch Loss:     1.282138 Tokens per Sec:    16926, Lr: 0.000300\n",
      "2019-10-30 18:19:44,825 Epoch  15 Step:   133400 Batch Loss:     1.266791 Tokens per Sec:    17100, Lr: 0.000300\n",
      "2019-10-30 18:19:59,944 Epoch  15 Step:   133500 Batch Loss:     1.467545 Tokens per Sec:    16902, Lr: 0.000300\n",
      "2019-10-30 18:20:15,279 Epoch  15 Step:   133600 Batch Loss:     1.249543 Tokens per Sec:    16710, Lr: 0.000300\n",
      "2019-10-30 18:20:30,612 Epoch  15 Step:   133700 Batch Loss:     1.304482 Tokens per Sec:    16822, Lr: 0.000300\n",
      "2019-10-30 18:20:46,041 Epoch  15 Step:   133800 Batch Loss:     1.249830 Tokens per Sec:    16492, Lr: 0.000300\n",
      "2019-10-30 18:21:01,189 Epoch  15 Step:   133900 Batch Loss:     1.265324 Tokens per Sec:    16968, Lr: 0.000300\n",
      "2019-10-30 18:21:16,549 Epoch  15 Step:   134000 Batch Loss:     1.201224 Tokens per Sec:    16736, Lr: 0.000300\n",
      "2019-10-30 18:22:00,983 Example #0\n",
      "2019-10-30 18:22:00,984 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:22:00,984 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:22:00,984 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:22:00,984 Example #1\n",
      "2019-10-30 18:22:00,984 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:22:00,984 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:22:00,984 \tHypothesis: Biblia haidhihirishi uthabiti .\n",
      "2019-10-30 18:22:00,984 Example #2\n",
      "2019-10-30 18:22:00,984 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:22:00,984 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:22:00,984 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 18:22:00,984 Example #3\n",
      "2019-10-30 18:22:00,984 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:22:00,984 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:22:00,984 \tHypothesis: Wapenzi wa maua na mabusu .\n",
      "2019-10-30 18:22:00,984 Validation result at epoch  15, step   134000: bleu:  35.54, loss: 31535.3770, ppl:   3.1500, duration: 44.4345s\n",
      "2019-10-30 18:22:16,388 Epoch  15 Step:   134100 Batch Loss:     1.384937 Tokens per Sec:    16851, Lr: 0.000300\n",
      "2019-10-30 18:22:31,735 Epoch  15 Step:   134200 Batch Loss:     1.178881 Tokens per Sec:    16790, Lr: 0.000300\n",
      "2019-10-30 18:22:46,820 Epoch  15 Step:   134300 Batch Loss:     1.083133 Tokens per Sec:    16544, Lr: 0.000300\n",
      "2019-10-30 18:23:02,043 Epoch  15 Step:   134400 Batch Loss:     1.304722 Tokens per Sec:    16768, Lr: 0.000300\n",
      "2019-10-30 18:23:17,241 Epoch  15 Step:   134500 Batch Loss:     1.245246 Tokens per Sec:    16819, Lr: 0.000300\n",
      "2019-10-30 18:23:32,550 Epoch  15 Step:   134600 Batch Loss:     1.437443 Tokens per Sec:    16821, Lr: 0.000300\n",
      "2019-10-30 18:23:47,872 Epoch  15 Step:   134700 Batch Loss:     1.456070 Tokens per Sec:    17000, Lr: 0.000300\n",
      "2019-10-30 18:24:03,133 Epoch  15 Step:   134800 Batch Loss:     1.304901 Tokens per Sec:    16519, Lr: 0.000300\n",
      "2019-10-30 18:24:18,250 Epoch  15 Step:   134900 Batch Loss:     1.175016 Tokens per Sec:    16546, Lr: 0.000300\n",
      "2019-10-30 18:24:33,385 Epoch  15 Step:   135000 Batch Loss:     1.142667 Tokens per Sec:    16870, Lr: 0.000300\n",
      "2019-10-30 18:25:18,415 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:25:18,415 Saving new checkpoint.\n",
      "2019-10-30 18:25:18,565 Example #0\n",
      "2019-10-30 18:25:18,565 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:25:18,565 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:25:18,566 \tHypothesis: Alikuwa mjamzito akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:25:18,566 Example #1\n",
      "2019-10-30 18:25:18,566 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:25:18,566 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:25:18,566 \tHypothesis: Biblia haisemi kwamba mtu anapaswa kuwa na sifa nzuri .\n",
      "2019-10-30 18:25:18,566 Example #2\n",
      "2019-10-30 18:25:18,566 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:25:18,566 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:25:18,566 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 18:25:18,566 Example #3\n",
      "2019-10-30 18:25:18,566 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:25:18,566 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:25:18,566 \tHypothesis: Mapigo ya maua na mabusu .\n",
      "2019-10-30 18:25:18,566 Validation result at epoch  15, step   135000: bleu:  36.36, loss: 31373.8242, ppl:   3.1316, duration: 45.1811s\n",
      "2019-10-30 18:25:33,806 Epoch  15 Step:   135100 Batch Loss:     1.153123 Tokens per Sec:    16870, Lr: 0.000300\n",
      "2019-10-30 18:25:48,699 Epoch  15 Step:   135200 Batch Loss:     1.300873 Tokens per Sec:    17101, Lr: 0.000300\n",
      "2019-10-30 18:26:03,737 Epoch  15 Step:   135300 Batch Loss:     1.310160 Tokens per Sec:    16848, Lr: 0.000300\n",
      "2019-10-30 18:26:18,975 Epoch  15 Step:   135400 Batch Loss:     1.295559 Tokens per Sec:    16993, Lr: 0.000300\n",
      "2019-10-30 18:26:34,329 Epoch  15 Step:   135500 Batch Loss:     1.287220 Tokens per Sec:    16721, Lr: 0.000300\n",
      "2019-10-30 18:26:49,500 Epoch  15 Step:   135600 Batch Loss:     1.330351 Tokens per Sec:    16966, Lr: 0.000300\n",
      "2019-10-30 18:27:04,742 Epoch  15 Step:   135700 Batch Loss:     1.169084 Tokens per Sec:    17042, Lr: 0.000300\n",
      "2019-10-30 18:27:20,116 Epoch  15 Step:   135800 Batch Loss:     1.074217 Tokens per Sec:    16876, Lr: 0.000300\n",
      "2019-10-30 18:27:35,245 Epoch  15 Step:   135900 Batch Loss:     1.237857 Tokens per Sec:    16589, Lr: 0.000300\n",
      "2019-10-30 18:27:50,491 Epoch  15 Step:   136000 Batch Loss:     1.335158 Tokens per Sec:    16889, Lr: 0.000300\n",
      "2019-10-30 18:28:35,235 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:28:35,236 Saving new checkpoint.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 18:28:35,379 Example #0\n",
      "2019-10-30 18:28:35,379 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:28:35,379 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:28:35,379 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:28:35,380 Example #1\n",
      "2019-10-30 18:28:35,380 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:28:35,380 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:28:35,380 \tHypothesis: Biblia haidhuru sifa ya uthabiti .\n",
      "2019-10-30 18:28:35,380 Example #2\n",
      "2019-10-30 18:28:35,380 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:28:35,380 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:28:35,380 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 18:28:35,380 Example #3\n",
      "2019-10-30 18:28:35,380 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:28:35,380 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:28:35,380 \tHypothesis: Mapigo na busu .\n",
      "2019-10-30 18:28:35,380 Validation result at epoch  15, step   136000: bleu:  35.97, loss: 31369.9180, ppl:   3.1311, duration: 44.8882s\n",
      "2019-10-30 18:28:50,611 Epoch  15 Step:   136100 Batch Loss:     1.247669 Tokens per Sec:    16758, Lr: 0.000300\n",
      "2019-10-30 18:29:06,013 Epoch  15 Step:   136200 Batch Loss:     1.361514 Tokens per Sec:    16970, Lr: 0.000300\n",
      "2019-10-30 18:29:21,178 Epoch  15 Step:   136300 Batch Loss:     1.212581 Tokens per Sec:    16767, Lr: 0.000300\n",
      "2019-10-30 18:29:36,614 Epoch  15 Step:   136400 Batch Loss:     1.273953 Tokens per Sec:    16781, Lr: 0.000300\n",
      "2019-10-30 18:29:52,240 Epoch  15 Step:   136500 Batch Loss:     1.186684 Tokens per Sec:    16687, Lr: 0.000300\n",
      "2019-10-30 18:30:07,646 Epoch  15 Step:   136600 Batch Loss:     1.116263 Tokens per Sec:    16758, Lr: 0.000300\n",
      "2019-10-30 18:30:22,962 Epoch  15 Step:   136700 Batch Loss:     1.209254 Tokens per Sec:    16719, Lr: 0.000300\n",
      "2019-10-30 18:30:38,142 Epoch  15 Step:   136800 Batch Loss:     1.636846 Tokens per Sec:    16516, Lr: 0.000300\n",
      "2019-10-30 18:30:53,714 Epoch  15 Step:   136900 Batch Loss:     1.215137 Tokens per Sec:    16507, Lr: 0.000300\n",
      "2019-10-30 18:31:09,021 Epoch  15 Step:   137000 Batch Loss:     1.204937 Tokens per Sec:    16839, Lr: 0.000300\n",
      "2019-10-30 18:31:53,429 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:31:53,430 Saving new checkpoint.\n",
      "2019-10-30 18:31:53,574 Example #0\n",
      "2019-10-30 18:31:53,575 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:31:53,575 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:31:53,575 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:31:53,575 Example #1\n",
      "2019-10-30 18:31:53,575 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:31:53,575 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:31:53,575 \tHypothesis: Biblia haidhihirishi sifa ya uthabiti .\n",
      "2019-10-30 18:31:53,575 Example #2\n",
      "2019-10-30 18:31:53,575 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:31:53,575 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:31:53,575 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 18:31:53,575 Example #3\n",
      "2019-10-30 18:31:53,575 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:31:53,575 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:31:53,576 \tHypothesis: Vikumbusha na busu .\n",
      "2019-10-30 18:31:53,576 Validation result at epoch  15, step   137000: bleu:  35.82, loss: 31290.9375, ppl:   3.1221, duration: 44.5537s\n",
      "2019-10-30 18:32:09,028 Epoch  15 Step:   137100 Batch Loss:     1.441136 Tokens per Sec:    16898, Lr: 0.000300\n",
      "2019-10-30 18:32:24,175 Epoch  15 Step:   137200 Batch Loss:     1.209270 Tokens per Sec:    16947, Lr: 0.000300\n",
      "2019-10-30 18:32:39,468 Epoch  15 Step:   137300 Batch Loss:     1.191454 Tokens per Sec:    17251, Lr: 0.000300\n",
      "2019-10-30 18:32:54,788 Epoch  15 Step:   137400 Batch Loss:     1.893786 Tokens per Sec:    16821, Lr: 0.000300\n",
      "2019-10-30 18:33:10,293 Epoch  15 Step:   137500 Batch Loss:     1.145040 Tokens per Sec:    17258, Lr: 0.000300\n",
      "2019-10-30 18:33:25,542 Epoch  15 Step:   137600 Batch Loss:     1.330248 Tokens per Sec:    16566, Lr: 0.000300\n",
      "2019-10-30 18:33:40,843 Epoch  15 Step:   137700 Batch Loss:     1.308347 Tokens per Sec:    16818, Lr: 0.000300\n",
      "2019-10-30 18:33:56,295 Epoch  15 Step:   137800 Batch Loss:     1.321378 Tokens per Sec:    16755, Lr: 0.000300\n",
      "2019-10-30 18:34:11,812 Epoch  15 Step:   137900 Batch Loss:     1.317493 Tokens per Sec:    16721, Lr: 0.000300\n",
      "2019-10-30 18:34:27,221 Epoch  15 Step:   138000 Batch Loss:     1.254171 Tokens per Sec:    16859, Lr: 0.000300\n",
      "2019-10-30 18:35:11,981 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:35:11,982 Saving new checkpoint.\n",
      "2019-10-30 18:35:12,125 Example #0\n",
      "2019-10-30 18:35:12,125 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:35:12,125 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:35:12,125 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:35:12,125 Example #1\n",
      "2019-10-30 18:35:12,125 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:35:12,125 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:35:12,125 \tHypothesis: Biblia haidhuru sifa ya utukufu .\n",
      "2019-10-30 18:35:12,125 Example #2\n",
      "2019-10-30 18:35:12,125 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:35:12,125 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:35:12,125 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 18:35:12,126 Example #3\n",
      "2019-10-30 18:35:12,126 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:35:12,126 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:35:12,126 \tHypothesis: Mabusu ya vikumbusho na busu .\n",
      "2019-10-30 18:35:12,126 Validation result at epoch  15, step   138000: bleu:  35.94, loss: 31235.0312, ppl:   3.1158, duration: 44.9041s\n",
      "2019-10-30 18:35:27,290 Epoch  15 Step:   138100 Batch Loss:     1.331795 Tokens per Sec:    17019, Lr: 0.000300\n",
      "2019-10-30 18:35:42,368 Epoch  15 Step:   138200 Batch Loss:     1.276215 Tokens per Sec:    16599, Lr: 0.000300\n",
      "2019-10-30 18:35:57,627 Epoch  15 Step:   138300 Batch Loss:     1.420874 Tokens per Sec:    16852, Lr: 0.000300\n",
      "2019-10-30 18:36:12,927 Epoch  15 Step:   138400 Batch Loss:     1.216025 Tokens per Sec:    16868, Lr: 0.000300\n",
      "2019-10-30 18:36:28,107 Epoch  15 Step:   138500 Batch Loss:     1.277577 Tokens per Sec:    16990, Lr: 0.000300\n",
      "2019-10-30 18:36:43,682 Epoch  15 Step:   138600 Batch Loss:     1.292931 Tokens per Sec:    16859, Lr: 0.000300\n",
      "2019-10-30 18:36:59,083 Epoch  15 Step:   138700 Batch Loss:     1.257852 Tokens per Sec:    16646, Lr: 0.000300\n",
      "2019-10-30 18:37:14,283 Epoch  15 Step:   138800 Batch Loss:     1.207330 Tokens per Sec:    16915, Lr: 0.000300\n",
      "2019-10-30 18:37:29,625 Epoch  15 Step:   138900 Batch Loss:     1.347576 Tokens per Sec:    16315, Lr: 0.000300\n",
      "2019-10-30 18:37:44,773 Epoch  15 Step:   139000 Batch Loss:     1.545108 Tokens per Sec:    16767, Lr: 0.000300\n",
      "2019-10-30 18:38:29,430 Example #0\n",
      "2019-10-30 18:38:29,430 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:38:29,430 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:38:29,431 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:38:29,431 Example #1\n",
      "2019-10-30 18:38:29,431 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:38:29,431 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:38:29,431 \tHypothesis: Biblia haidumu sifa .\n",
      "2019-10-30 18:38:29,431 Example #2\n",
      "2019-10-30 18:38:29,431 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:38:29,431 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:38:29,431 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 18:38:29,431 Example #3\n",
      "2019-10-30 18:38:29,431 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:38:29,431 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:38:29,431 \tHypothesis: Loti za maua na busu .\n",
      "2019-10-30 18:38:29,431 Validation result at epoch  15, step   139000: bleu:  35.81, loss: 31268.1230, ppl:   3.1195, duration: 44.6580s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 18:38:44,850 Epoch  15 Step:   139100 Batch Loss:     1.174313 Tokens per Sec:    16453, Lr: 0.000300\n",
      "2019-10-30 18:39:00,220 Epoch  15 Step:   139200 Batch Loss:     1.285943 Tokens per Sec:    16429, Lr: 0.000300\n",
      "2019-10-30 18:39:15,588 Epoch  15 Step:   139300 Batch Loss:     1.185293 Tokens per Sec:    16876, Lr: 0.000300\n",
      "2019-10-30 18:39:30,619 Epoch  15 Step:   139400 Batch Loss:     1.290485 Tokens per Sec:    16691, Lr: 0.000300\n",
      "2019-10-30 18:39:45,985 Epoch  15 Step:   139500 Batch Loss:     1.377419 Tokens per Sec:    16930, Lr: 0.000300\n",
      "2019-10-30 18:40:01,456 Epoch  15 Step:   139600 Batch Loss:     1.286143 Tokens per Sec:    16755, Lr: 0.000300\n",
      "2019-10-30 18:40:16,678 Epoch  15 Step:   139700 Batch Loss:     1.191706 Tokens per Sec:    16916, Lr: 0.000300\n",
      "2019-10-30 18:40:32,083 Epoch  15 Step:   139800 Batch Loss:     1.402220 Tokens per Sec:    16939, Lr: 0.000300\n",
      "2019-10-30 18:40:47,416 Epoch  15 Step:   139900 Batch Loss:     1.370015 Tokens per Sec:    16437, Lr: 0.000300\n",
      "2019-10-30 18:41:02,838 Epoch  15 Step:   140000 Batch Loss:     1.416574 Tokens per Sec:    16762, Lr: 0.000300\n",
      "2019-10-30 18:41:47,715 Example #0\n",
      "2019-10-30 18:41:47,715 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:41:47,716 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:41:47,716 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:41:47,716 Example #1\n",
      "2019-10-30 18:41:47,716 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:41:47,716 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:41:47,716 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 18:41:47,716 Example #2\n",
      "2019-10-30 18:41:47,716 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:41:47,716 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:41:47,716 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea nini ?\n",
      "2019-10-30 18:41:47,716 Example #3\n",
      "2019-10-30 18:41:47,716 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:41:47,716 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:41:47,716 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 18:41:47,716 Validation result at epoch  15, step   140000: bleu:  36.15, loss: 31296.9570, ppl:   3.1228, duration: 44.8778s\n",
      "2019-10-30 18:42:03,165 Epoch  15 Step:   140100 Batch Loss:     1.301632 Tokens per Sec:    17021, Lr: 0.000300\n",
      "2019-10-30 18:42:18,351 Epoch  15 Step:   140200 Batch Loss:     1.315696 Tokens per Sec:    16740, Lr: 0.000300\n",
      "2019-10-30 18:42:33,651 Epoch  15 Step:   140300 Batch Loss:     1.539450 Tokens per Sec:    17044, Lr: 0.000300\n",
      "2019-10-30 18:42:49,048 Epoch  15 Step:   140400 Batch Loss:     1.209575 Tokens per Sec:    16685, Lr: 0.000300\n",
      "2019-10-30 18:43:04,360 Epoch  15 Step:   140500 Batch Loss:     1.162516 Tokens per Sec:    16913, Lr: 0.000300\n",
      "2019-10-30 18:43:19,848 Epoch  15 Step:   140600 Batch Loss:     1.261189 Tokens per Sec:    16545, Lr: 0.000300\n",
      "2019-10-30 18:43:35,137 Epoch  15 Step:   140700 Batch Loss:     1.408452 Tokens per Sec:    16726, Lr: 0.000300\n",
      "2019-10-30 18:43:46,357 Epoch  15: total training loss 12087.38\n",
      "2019-10-30 18:43:46,357 EPOCH 16\n",
      "2019-10-30 18:43:51,519 Epoch  16 Step:   140800 Batch Loss:     1.206576 Tokens per Sec:    13196, Lr: 0.000300\n",
      "2019-10-30 18:44:06,911 Epoch  16 Step:   140900 Batch Loss:     1.638583 Tokens per Sec:    16457, Lr: 0.000300\n",
      "2019-10-30 18:44:22,074 Epoch  16 Step:   141000 Batch Loss:     1.460813 Tokens per Sec:    16541, Lr: 0.000300\n",
      "2019-10-30 18:45:07,173 Example #0\n",
      "2019-10-30 18:45:07,173 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:45:07,174 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:45:07,174 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 18:45:07,174 Example #1\n",
      "2019-10-30 18:45:07,174 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:45:07,174 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:45:07,174 \tHypothesis: Biblia haisemi kwamba mtu anastahili sifa .\n",
      "2019-10-30 18:45:07,174 Example #2\n",
      "2019-10-30 18:45:07,174 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:45:07,174 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:45:07,174 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 18:45:07,174 Example #3\n",
      "2019-10-30 18:45:07,174 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:45:07,174 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:45:07,174 \tHypothesis: Miguu ya maua na busu .\n",
      "2019-10-30 18:45:07,174 Validation result at epoch  16, step   141000: bleu:  36.42, loss: 31286.5781, ppl:   3.1216, duration: 45.0994s\n",
      "2019-10-30 18:45:22,414 Epoch  16 Step:   141100 Batch Loss:     1.176355 Tokens per Sec:    16562, Lr: 0.000300\n",
      "2019-10-30 18:45:37,875 Epoch  16 Step:   141200 Batch Loss:     1.159231 Tokens per Sec:    16573, Lr: 0.000300\n",
      "2019-10-30 18:45:53,248 Epoch  16 Step:   141300 Batch Loss:     1.188052 Tokens per Sec:    16864, Lr: 0.000300\n",
      "2019-10-30 18:46:08,559 Epoch  16 Step:   141400 Batch Loss:     1.291242 Tokens per Sec:    16463, Lr: 0.000300\n",
      "2019-10-30 18:46:23,946 Epoch  16 Step:   141500 Batch Loss:     1.431778 Tokens per Sec:    16759, Lr: 0.000300\n",
      "2019-10-30 18:46:39,353 Epoch  16 Step:   141600 Batch Loss:     1.240370 Tokens per Sec:    16561, Lr: 0.000300\n",
      "2019-10-30 18:46:54,450 Epoch  16 Step:   141700 Batch Loss:     1.136743 Tokens per Sec:    16858, Lr: 0.000300\n",
      "2019-10-30 18:47:09,509 Epoch  16 Step:   141800 Batch Loss:     1.335451 Tokens per Sec:    17217, Lr: 0.000300\n",
      "2019-10-30 18:47:24,822 Epoch  16 Step:   141900 Batch Loss:     1.243581 Tokens per Sec:    16677, Lr: 0.000300\n",
      "2019-10-30 18:47:40,075 Epoch  16 Step:   142000 Batch Loss:     1.401030 Tokens per Sec:    17071, Lr: 0.000300\n",
      "2019-10-30 18:48:24,933 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:48:24,933 Saving new checkpoint.\n",
      "2019-10-30 18:48:25,076 Example #0\n",
      "2019-10-30 18:48:25,076 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:48:25,077 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:48:25,077 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:48:25,077 Example #1\n",
      "2019-10-30 18:48:25,077 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:48:25,077 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:48:25,077 \tHypothesis: Biblia haidhoofishi kuwa na sifa ya ustahili .\n",
      "2019-10-30 18:48:25,077 Example #2\n",
      "2019-10-30 18:48:25,077 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:48:25,077 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:48:25,077 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuchochea nini ?\n",
      "2019-10-30 18:48:25,077 Example #3\n",
      "2019-10-30 18:48:25,077 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:48:25,077 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:48:25,077 \tHypothesis: Miguu ya maua na mabusu .\n",
      "2019-10-30 18:48:25,077 Validation result at epoch  16, step   142000: bleu:  35.94, loss: 31173.1699, ppl:   3.1088, duration: 45.0023s\n",
      "2019-10-30 18:48:40,447 Epoch  16 Step:   142100 Batch Loss:     1.133529 Tokens per Sec:    17348, Lr: 0.000300\n",
      "2019-10-30 18:48:55,896 Epoch  16 Step:   142200 Batch Loss:     1.198405 Tokens per Sec:    16723, Lr: 0.000300\n",
      "2019-10-30 18:49:11,371 Epoch  16 Step:   142300 Batch Loss:     1.168423 Tokens per Sec:    16658, Lr: 0.000300\n",
      "2019-10-30 18:49:26,809 Epoch  16 Step:   142400 Batch Loss:     1.444300 Tokens per Sec:    16685, Lr: 0.000300\n",
      "2019-10-30 18:49:42,073 Epoch  16 Step:   142500 Batch Loss:     1.167660 Tokens per Sec:    16928, Lr: 0.000300\n",
      "2019-10-30 18:49:57,426 Epoch  16 Step:   142600 Batch Loss:     1.205439 Tokens per Sec:    16832, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 18:50:12,852 Epoch  16 Step:   142700 Batch Loss:     1.323791 Tokens per Sec:    16910, Lr: 0.000300\n",
      "2019-10-30 18:50:28,081 Epoch  16 Step:   142800 Batch Loss:     1.542299 Tokens per Sec:    16645, Lr: 0.000300\n",
      "2019-10-30 18:50:43,177 Epoch  16 Step:   142900 Batch Loss:     1.110668 Tokens per Sec:    16817, Lr: 0.000300\n",
      "2019-10-30 18:50:58,705 Epoch  16 Step:   143000 Batch Loss:     1.266969 Tokens per Sec:    16609, Lr: 0.000300\n",
      "2019-10-30 18:51:43,844 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:51:43,844 Saving new checkpoint.\n",
      "2019-10-30 18:51:43,988 Example #0\n",
      "2019-10-30 18:51:43,988 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:51:43,988 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:51:43,988 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:51:43,988 Example #1\n",
      "2019-10-30 18:51:43,988 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:51:43,988 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:51:43,988 \tHypothesis: Biblia haisemi kwamba watu wana sifa nzuri .\n",
      "2019-10-30 18:51:43,988 Example #2\n",
      "2019-10-30 18:51:43,989 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:51:43,989 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:51:43,989 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 18:51:43,989 Example #3\n",
      "2019-10-30 18:51:43,989 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:51:43,989 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:51:43,989 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 18:51:43,989 Validation result at epoch  16, step   143000: bleu:  36.41, loss: 31141.2910, ppl:   3.1052, duration: 45.2836s\n",
      "2019-10-30 18:51:59,442 Epoch  16 Step:   143100 Batch Loss:     1.170949 Tokens per Sec:    16837, Lr: 0.000300\n",
      "2019-10-30 18:52:14,645 Epoch  16 Step:   143200 Batch Loss:     1.323741 Tokens per Sec:    16948, Lr: 0.000300\n",
      "2019-10-30 18:52:29,943 Epoch  16 Step:   143300 Batch Loss:     1.221006 Tokens per Sec:    16383, Lr: 0.000300\n",
      "2019-10-30 18:52:45,137 Epoch  16 Step:   143400 Batch Loss:     1.318340 Tokens per Sec:    16750, Lr: 0.000300\n",
      "2019-10-30 18:53:00,437 Epoch  16 Step:   143500 Batch Loss:     1.281608 Tokens per Sec:    16736, Lr: 0.000300\n",
      "2019-10-30 18:53:15,827 Epoch  16 Step:   143600 Batch Loss:     1.298808 Tokens per Sec:    16782, Lr: 0.000300\n",
      "2019-10-30 18:53:31,093 Epoch  16 Step:   143700 Batch Loss:     1.153872 Tokens per Sec:    17076, Lr: 0.000300\n",
      "2019-10-30 18:53:46,117 Epoch  16 Step:   143800 Batch Loss:     1.246746 Tokens per Sec:    16966, Lr: 0.000300\n",
      "2019-10-30 18:54:01,592 Epoch  16 Step:   143900 Batch Loss:     1.077733 Tokens per Sec:    16866, Lr: 0.000300\n",
      "2019-10-30 18:54:16,980 Epoch  16 Step:   144000 Batch Loss:     1.152068 Tokens per Sec:    16560, Lr: 0.000300\n",
      "2019-10-30 18:55:01,532 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 18:55:01,532 Saving new checkpoint.\n",
      "2019-10-30 18:55:01,684 Example #0\n",
      "2019-10-30 18:55:01,684 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:55:01,684 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:55:01,684 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:55:01,685 Example #1\n",
      "2019-10-30 18:55:01,685 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:55:01,685 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:55:01,685 \tHypothesis: Biblia haidai kuwa na sifa ya ustahili .\n",
      "2019-10-30 18:55:01,685 Example #2\n",
      "2019-10-30 18:55:01,685 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:55:01,685 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:55:01,685 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea jinsi gani ?\n",
      "2019-10-30 18:55:01,685 Example #3\n",
      "2019-10-30 18:55:01,685 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:55:01,685 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:55:01,685 \tHypothesis: Mizigo na mabusu .\n",
      "2019-10-30 18:55:01,685 Validation result at epoch  16, step   144000: bleu:  35.68, loss: 31070.7070, ppl:   3.0972, duration: 44.7045s\n",
      "2019-10-30 18:55:16,933 Epoch  16 Step:   144100 Batch Loss:     1.440859 Tokens per Sec:    16552, Lr: 0.000300\n",
      "2019-10-30 18:55:32,200 Epoch  16 Step:   144200 Batch Loss:     1.277202 Tokens per Sec:    16599, Lr: 0.000300\n",
      "2019-10-30 18:55:47,268 Epoch  16 Step:   144300 Batch Loss:     1.389222 Tokens per Sec:    17050, Lr: 0.000300\n",
      "2019-10-30 18:56:02,554 Epoch  16 Step:   144400 Batch Loss:     1.496692 Tokens per Sec:    16863, Lr: 0.000300\n",
      "2019-10-30 18:56:17,864 Epoch  16 Step:   144500 Batch Loss:     1.222068 Tokens per Sec:    16827, Lr: 0.000300\n",
      "2019-10-30 18:56:33,105 Epoch  16 Step:   144600 Batch Loss:     1.272994 Tokens per Sec:    17059, Lr: 0.000300\n",
      "2019-10-30 18:56:48,222 Epoch  16 Step:   144700 Batch Loss:     1.217372 Tokens per Sec:    16693, Lr: 0.000300\n",
      "2019-10-30 18:57:03,388 Epoch  16 Step:   144800 Batch Loss:     1.234970 Tokens per Sec:    17076, Lr: 0.000300\n",
      "2019-10-30 18:57:18,627 Epoch  16 Step:   144900 Batch Loss:     1.203139 Tokens per Sec:    16529, Lr: 0.000300\n",
      "2019-10-30 18:57:33,870 Epoch  16 Step:   145000 Batch Loss:     1.311615 Tokens per Sec:    16620, Lr: 0.000300\n",
      "2019-10-30 18:58:18,791 Example #0\n",
      "2019-10-30 18:58:18,791 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 18:58:18,791 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:58:18,792 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 18:58:18,792 Example #1\n",
      "2019-10-30 18:58:18,792 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 18:58:18,792 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 18:58:18,792 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya upole .\n",
      "2019-10-30 18:58:18,792 Example #2\n",
      "2019-10-30 18:58:18,792 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 18:58:18,792 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 18:58:18,792 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 18:58:18,792 Example #3\n",
      "2019-10-30 18:58:18,792 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 18:58:18,792 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 18:58:18,792 \tHypothesis: Mabusu ya maua na busu .\n",
      "2019-10-30 18:58:18,792 Validation result at epoch  16, step   145000: bleu:  36.18, loss: 31191.1641, ppl:   3.1108, duration: 44.9221s\n",
      "2019-10-30 18:58:34,175 Epoch  16 Step:   145100 Batch Loss:     1.384310 Tokens per Sec:    16765, Lr: 0.000300\n",
      "2019-10-30 18:58:49,501 Epoch  16 Step:   145200 Batch Loss:     1.190557 Tokens per Sec:    17036, Lr: 0.000300\n",
      "2019-10-30 18:59:04,916 Epoch  16 Step:   145300 Batch Loss:     1.130760 Tokens per Sec:    16585, Lr: 0.000300\n",
      "2019-10-30 18:59:20,364 Epoch  16 Step:   145400 Batch Loss:     1.219051 Tokens per Sec:    16644, Lr: 0.000300\n",
      "2019-10-30 18:59:35,483 Epoch  16 Step:   145500 Batch Loss:     1.219537 Tokens per Sec:    16729, Lr: 0.000300\n",
      "2019-10-30 18:59:50,617 Epoch  16 Step:   145600 Batch Loss:     1.343397 Tokens per Sec:    17058, Lr: 0.000300\n",
      "2019-10-30 19:00:05,895 Epoch  16 Step:   145700 Batch Loss:     1.201508 Tokens per Sec:    16979, Lr: 0.000300\n",
      "2019-10-30 19:00:21,270 Epoch  16 Step:   145800 Batch Loss:     1.276195 Tokens per Sec:    16382, Lr: 0.000300\n",
      "2019-10-30 19:00:36,498 Epoch  16 Step:   145900 Batch Loss:     1.157724 Tokens per Sec:    16860, Lr: 0.000300\n",
      "2019-10-30 19:00:51,790 Epoch  16 Step:   146000 Batch Loss:     1.199011 Tokens per Sec:    17053, Lr: 0.000300\n",
      "2019-10-30 19:01:36,657 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:01:36,657 Saving new checkpoint.\n",
      "2019-10-30 19:01:36,800 Example #0\n",
      "2019-10-30 19:01:36,801 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:01:36,801 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:01:36,801 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:01:36,801 Example #1\n",
      "2019-10-30 19:01:36,801 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:01:36,801 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:01:36,801 \tHypothesis: Biblia haisemi kwamba mtu anapaswa kuwa na sifa ya uthabiti .\n",
      "2019-10-30 19:01:36,801 Example #2\n",
      "2019-10-30 19:01:36,801 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:01:36,801 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:01:36,801 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 19:01:36,801 Example #3\n",
      "2019-10-30 19:01:36,801 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:01:36,801 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:01:36,801 \tHypothesis: Watu huogopa na kubusu .\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 19:01:36,801 Validation result at epoch  16, step   146000: bleu:  36.33, loss: 31040.1992, ppl:   3.0938, duration: 45.0112s\n",
      "2019-10-30 19:01:51,956 Epoch  16 Step:   146100 Batch Loss:     1.209768 Tokens per Sec:    16950, Lr: 0.000300\n",
      "2019-10-30 19:02:07,395 Epoch  16 Step:   146200 Batch Loss:     1.229639 Tokens per Sec:    16693, Lr: 0.000300\n",
      "2019-10-30 19:02:22,564 Epoch  16 Step:   146300 Batch Loss:     1.338275 Tokens per Sec:    16770, Lr: 0.000300\n",
      "2019-10-30 19:02:37,925 Epoch  16 Step:   146400 Batch Loss:     1.238926 Tokens per Sec:    16720, Lr: 0.000300\n",
      "2019-10-30 19:02:53,519 Epoch  16 Step:   146500 Batch Loss:     1.310450 Tokens per Sec:    17051, Lr: 0.000300\n",
      "2019-10-30 19:03:08,943 Epoch  16 Step:   146600 Batch Loss:     1.239660 Tokens per Sec:    16891, Lr: 0.000300\n",
      "2019-10-30 19:03:24,182 Epoch  16 Step:   146700 Batch Loss:     1.263788 Tokens per Sec:    16757, Lr: 0.000300\n",
      "2019-10-30 19:03:39,328 Epoch  16 Step:   146800 Batch Loss:     1.275021 Tokens per Sec:    17076, Lr: 0.000300\n",
      "2019-10-30 19:03:54,485 Epoch  16 Step:   146900 Batch Loss:     1.185429 Tokens per Sec:    16801, Lr: 0.000300\n",
      "2019-10-30 19:04:10,069 Epoch  16 Step:   147000 Batch Loss:     1.312012 Tokens per Sec:    17201, Lr: 0.000300\n",
      "2019-10-30 19:04:54,726 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:04:54,726 Saving new checkpoint.\n",
      "2019-10-30 19:04:54,869 Example #0\n",
      "2019-10-30 19:04:54,869 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:04:54,869 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:04:54,869 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:04:54,869 Example #1\n",
      "2019-10-30 19:04:54,869 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:04:54,870 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:04:54,870 \tHypothesis: Biblia haisemi uthibitisho .\n",
      "2019-10-30 19:04:54,870 Example #2\n",
      "2019-10-30 19:04:54,870 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:04:54,870 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:04:54,870 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 19:04:54,870 Example #3\n",
      "2019-10-30 19:04:54,870 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:04:54,870 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:04:54,870 \tHypothesis: Mapigo ya makumbatio na mabusu .\n",
      "2019-10-30 19:04:54,870 Validation result at epoch  16, step   147000: bleu:  36.47, loss: 31003.8926, ppl:   3.0897, duration: 44.8006s\n",
      "2019-10-30 19:05:09,982 Epoch  16 Step:   147100 Batch Loss:     1.200923 Tokens per Sec:    17053, Lr: 0.000300\n",
      "2019-10-30 19:05:25,205 Epoch  16 Step:   147200 Batch Loss:     1.173207 Tokens per Sec:    16557, Lr: 0.000300\n",
      "2019-10-30 19:05:40,582 Epoch  16 Step:   147300 Batch Loss:     1.239446 Tokens per Sec:    16837, Lr: 0.000300\n",
      "2019-10-30 19:05:55,782 Epoch  16 Step:   147400 Batch Loss:     1.235522 Tokens per Sec:    16578, Lr: 0.000300\n",
      "2019-10-30 19:06:10,932 Epoch  16 Step:   147500 Batch Loss:     1.418857 Tokens per Sec:    16863, Lr: 0.000300\n",
      "2019-10-30 19:06:26,318 Epoch  16 Step:   147600 Batch Loss:     1.347340 Tokens per Sec:    16832, Lr: 0.000300\n",
      "2019-10-30 19:06:41,625 Epoch  16 Step:   147700 Batch Loss:     1.280019 Tokens per Sec:    16979, Lr: 0.000300\n",
      "2019-10-30 19:06:57,128 Epoch  16 Step:   147800 Batch Loss:     1.189786 Tokens per Sec:    16655, Lr: 0.000300\n",
      "2019-10-30 19:07:12,349 Epoch  16 Step:   147900 Batch Loss:     1.223044 Tokens per Sec:    16773, Lr: 0.000300\n",
      "2019-10-30 19:07:27,481 Epoch  16 Step:   148000 Batch Loss:     1.164613 Tokens per Sec:    17003, Lr: 0.000300\n",
      "2019-10-30 19:08:12,515 Example #0\n",
      "2019-10-30 19:08:12,515 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:08:12,515 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:08:12,515 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:08:12,515 Example #1\n",
      "2019-10-30 19:08:12,515 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:08:12,516 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:08:12,516 \tHypothesis: Biblia haisemi ukweli .\n",
      "2019-10-30 19:08:12,516 Example #2\n",
      "2019-10-30 19:08:12,516 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:08:12,516 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:08:12,516 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukuathiri nini ?\n",
      "2019-10-30 19:08:12,516 Example #3\n",
      "2019-10-30 19:08:12,516 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:08:12,516 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:08:12,516 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 19:08:12,516 Validation result at epoch  16, step   148000: bleu:  36.53, loss: 31210.7539, ppl:   3.1130, duration: 45.0345s\n",
      "2019-10-30 19:08:28,018 Epoch  16 Step:   148100 Batch Loss:     1.391120 Tokens per Sec:    17069, Lr: 0.000300\n",
      "2019-10-30 19:08:43,277 Epoch  16 Step:   148200 Batch Loss:     1.299118 Tokens per Sec:    17033, Lr: 0.000300\n",
      "2019-10-30 19:08:58,673 Epoch  16 Step:   148300 Batch Loss:     1.180246 Tokens per Sec:    16541, Lr: 0.000300\n",
      "2019-10-30 19:09:13,873 Epoch  16 Step:   148400 Batch Loss:     1.244104 Tokens per Sec:    16748, Lr: 0.000300\n",
      "2019-10-30 19:09:29,174 Epoch  16 Step:   148500 Batch Loss:     1.202351 Tokens per Sec:    16659, Lr: 0.000300\n",
      "2019-10-30 19:09:44,404 Epoch  16 Step:   148600 Batch Loss:     1.204386 Tokens per Sec:    17375, Lr: 0.000300\n",
      "2019-10-30 19:09:59,589 Epoch  16 Step:   148700 Batch Loss:     1.231230 Tokens per Sec:    16888, Lr: 0.000300\n",
      "2019-10-30 19:10:14,784 Epoch  16 Step:   148800 Batch Loss:     1.206091 Tokens per Sec:    17170, Lr: 0.000300\n",
      "2019-10-30 19:10:30,096 Epoch  16 Step:   148900 Batch Loss:     1.280333 Tokens per Sec:    17029, Lr: 0.000300\n",
      "2019-10-30 19:10:45,558 Epoch  16 Step:   149000 Batch Loss:     1.248644 Tokens per Sec:    17049, Lr: 0.000300\n",
      "2019-10-30 19:11:30,588 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:11:30,588 Saving new checkpoint.\n",
      "2019-10-30 19:11:30,729 Example #0\n",
      "2019-10-30 19:11:30,729 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:11:30,729 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:11:30,729 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:11:30,729 Example #1\n",
      "2019-10-30 19:11:30,729 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:11:30,729 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:11:30,730 \tHypothesis: Biblia haisemi kwamba mtu anapaswa kuwa na sifa nzuri .\n",
      "2019-10-30 19:11:30,730 Example #2\n",
      "2019-10-30 19:11:30,730 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:11:30,730 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:11:30,730 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yaweza kufanyaje ?\n",
      "2019-10-30 19:11:30,730 Example #3\n",
      "2019-10-30 19:11:30,730 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:11:30,730 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:11:30,730 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 19:11:30,730 Validation result at epoch  16, step   149000: bleu:  36.24, loss: 30965.7891, ppl:   3.0854, duration: 45.1717s\n",
      "2019-10-30 19:11:45,817 Epoch  16 Step:   149100 Batch Loss:     1.539169 Tokens per Sec:    16780, Lr: 0.000300\n",
      "2019-10-30 19:12:01,034 Epoch  16 Step:   149200 Batch Loss:     1.130629 Tokens per Sec:    16748, Lr: 0.000300\n",
      "2019-10-30 19:12:16,247 Epoch  16 Step:   149300 Batch Loss:     1.189481 Tokens per Sec:    17034, Lr: 0.000300\n",
      "2019-10-30 19:12:31,382 Epoch  16 Step:   149400 Batch Loss:     1.121992 Tokens per Sec:    16738, Lr: 0.000300\n",
      "2019-10-30 19:12:46,848 Epoch  16 Step:   149500 Batch Loss:     1.396655 Tokens per Sec:    16992, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 19:13:02,106 Epoch  16 Step:   149600 Batch Loss:     1.096492 Tokens per Sec:    16785, Lr: 0.000300\n",
      "2019-10-30 19:13:17,622 Epoch  16 Step:   149700 Batch Loss:     1.211549 Tokens per Sec:    16904, Lr: 0.000300\n",
      "2019-10-30 19:13:33,026 Epoch  16 Step:   149800 Batch Loss:     1.371854 Tokens per Sec:    16882, Lr: 0.000300\n",
      "2019-10-30 19:13:48,157 Epoch  16 Step:   149900 Batch Loss:     1.162665 Tokens per Sec:    16539, Lr: 0.000300\n",
      "2019-10-30 19:14:03,402 Epoch  16 Step:   150000 Batch Loss:     1.210113 Tokens per Sec:    16800, Lr: 0.000300\n",
      "2019-10-30 19:14:48,131 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:14:48,131 Saving new checkpoint.\n",
      "2019-10-30 19:14:48,278 Example #0\n",
      "2019-10-30 19:14:48,278 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:14:48,278 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:14:48,278 \tHypothesis: Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:14:48,278 Example #1\n",
      "2019-10-30 19:14:48,278 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:14:48,278 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:14:48,278 \tHypothesis: Biblia haisemi uthibitisho .\n",
      "2019-10-30 19:14:48,279 Example #2\n",
      "2019-10-30 19:14:48,279 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:14:48,279 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:14:48,279 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 19:14:48,279 Example #3\n",
      "2019-10-30 19:14:48,279 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:14:48,279 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:14:48,279 \tHypothesis: Vikumbusho na mabusu .\n",
      "2019-10-30 19:14:48,279 Validation result at epoch  16, step   150000: bleu:  36.33, loss: 30939.5449, ppl:   3.0825, duration: 44.8763s\n",
      "2019-10-30 19:15:03,633 Epoch  16 Step:   150100 Batch Loss:     1.106232 Tokens per Sec:    16908, Lr: 0.000300\n",
      "2019-10-30 19:15:10,868 Epoch  16: total training loss 11969.36\n",
      "2019-10-30 19:15:10,869 EPOCH 17\n",
      "2019-10-30 19:15:20,037 Epoch  17 Step:   150200 Batch Loss:     1.314996 Tokens per Sec:    14980, Lr: 0.000300\n",
      "2019-10-30 19:15:35,369 Epoch  17 Step:   150300 Batch Loss:     1.108461 Tokens per Sec:    16855, Lr: 0.000300\n",
      "2019-10-30 19:15:50,755 Epoch  17 Step:   150400 Batch Loss:     1.375903 Tokens per Sec:    16798, Lr: 0.000300\n",
      "2019-10-30 19:16:05,919 Epoch  17 Step:   150500 Batch Loss:     1.223656 Tokens per Sec:    16637, Lr: 0.000300\n",
      "2019-10-30 19:16:21,037 Epoch  17 Step:   150600 Batch Loss:     1.364205 Tokens per Sec:    17247, Lr: 0.000300\n",
      "2019-10-30 19:16:36,516 Epoch  17 Step:   150700 Batch Loss:     1.311609 Tokens per Sec:    16821, Lr: 0.000300\n",
      "2019-10-30 19:16:51,480 Epoch  17 Step:   150800 Batch Loss:     1.340581 Tokens per Sec:    16630, Lr: 0.000300\n",
      "2019-10-30 19:17:06,943 Epoch  17 Step:   150900 Batch Loss:     1.247688 Tokens per Sec:    16784, Lr: 0.000300\n",
      "2019-10-30 19:17:22,153 Epoch  17 Step:   151000 Batch Loss:     1.276533 Tokens per Sec:    16986, Lr: 0.000300\n",
      "2019-10-30 19:18:07,040 Example #0\n",
      "2019-10-30 19:18:07,040 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:18:07,040 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:18:07,040 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:18:07,040 Example #1\n",
      "2019-10-30 19:18:07,040 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:18:07,040 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:18:07,040 \tHypothesis: Biblia haishutumu sifa .\n",
      "2019-10-30 19:18:07,040 Example #2\n",
      "2019-10-30 19:18:07,040 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:18:07,040 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:18:07,041 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 19:18:07,041 Example #3\n",
      "2019-10-30 19:18:07,041 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:18:07,041 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:18:07,041 \tHypothesis: Vitu vya kumbatio na busu .\n",
      "2019-10-30 19:18:07,041 Validation result at epoch  17, step   151000: bleu:  36.40, loss: 30992.3613, ppl:   3.0884, duration: 44.8872s\n",
      "2019-10-30 19:18:22,149 Epoch  17 Step:   151100 Batch Loss:     1.155724 Tokens per Sec:    16777, Lr: 0.000300\n",
      "2019-10-30 19:18:37,514 Epoch  17 Step:   151200 Batch Loss:     1.098455 Tokens per Sec:    17066, Lr: 0.000300\n",
      "2019-10-30 19:18:52,576 Epoch  17 Step:   151300 Batch Loss:     1.307987 Tokens per Sec:    16899, Lr: 0.000300\n",
      "2019-10-30 19:19:07,826 Epoch  17 Step:   151400 Batch Loss:     1.404006 Tokens per Sec:    16861, Lr: 0.000300\n",
      "2019-10-30 19:19:23,228 Epoch  17 Step:   151500 Batch Loss:     1.126505 Tokens per Sec:    16737, Lr: 0.000300\n",
      "2019-10-30 19:19:38,545 Epoch  17 Step:   151600 Batch Loss:     1.315738 Tokens per Sec:    16922, Lr: 0.000300\n",
      "2019-10-30 19:19:53,886 Epoch  17 Step:   151700 Batch Loss:     1.261084 Tokens per Sec:    16888, Lr: 0.000300\n",
      "2019-10-30 19:20:09,073 Epoch  17 Step:   151800 Batch Loss:     1.312335 Tokens per Sec:    16958, Lr: 0.000300\n",
      "2019-10-30 19:20:24,289 Epoch  17 Step:   151900 Batch Loss:     1.198663 Tokens per Sec:    16643, Lr: 0.000300\n",
      "2019-10-30 19:20:39,404 Epoch  17 Step:   152000 Batch Loss:     1.237068 Tokens per Sec:    16664, Lr: 0.000300\n",
      "2019-10-30 19:21:23,937 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:21:23,938 Saving new checkpoint.\n",
      "2019-10-30 19:21:24,087 Example #0\n",
      "2019-10-30 19:21:24,087 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:21:24,087 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:21:24,087 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:21:24,087 Example #1\n",
      "2019-10-30 19:21:24,087 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:21:24,087 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:21:24,087 \tHypothesis: Biblia haishutumu sifa ya ustahili .\n",
      "2019-10-30 19:21:24,087 Example #2\n",
      "2019-10-30 19:21:24,087 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:21:24,087 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:21:24,087 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 19:21:24,087 Example #3\n",
      "2019-10-30 19:21:24,088 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:21:24,088 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:21:24,088 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 19:21:24,088 Validation result at epoch  17, step   152000: bleu:  36.49, loss: 30908.8379, ppl:   3.0790, duration: 44.6838s\n",
      "2019-10-30 19:21:39,072 Epoch  17 Step:   152100 Batch Loss:     1.138366 Tokens per Sec:    17099, Lr: 0.000300\n",
      "2019-10-30 19:21:54,317 Epoch  17 Step:   152200 Batch Loss:     1.372975 Tokens per Sec:    16929, Lr: 0.000300\n",
      "2019-10-30 19:22:09,611 Epoch  17 Step:   152300 Batch Loss:     1.315568 Tokens per Sec:    16747, Lr: 0.000300\n",
      "2019-10-30 19:22:24,782 Epoch  17 Step:   152400 Batch Loss:     1.221465 Tokens per Sec:    17105, Lr: 0.000300\n",
      "2019-10-30 19:22:39,869 Epoch  17 Step:   152500 Batch Loss:     1.103561 Tokens per Sec:    16889, Lr: 0.000300\n",
      "2019-10-30 19:22:55,078 Epoch  17 Step:   152600 Batch Loss:     1.345225 Tokens per Sec:    17030, Lr: 0.000300\n",
      "2019-10-30 19:23:10,342 Epoch  17 Step:   152700 Batch Loss:     1.251232 Tokens per Sec:    17086, Lr: 0.000300\n",
      "2019-10-30 19:23:25,613 Epoch  17 Step:   152800 Batch Loss:     1.436362 Tokens per Sec:    17046, Lr: 0.000300\n",
      "2019-10-30 19:23:40,793 Epoch  17 Step:   152900 Batch Loss:     1.062885 Tokens per Sec:    16873, Lr: 0.000300\n",
      "2019-10-30 19:23:55,839 Epoch  17 Step:   153000 Batch Loss:     1.339080 Tokens per Sec:    16555, Lr: 0.000300\n",
      "2019-10-30 19:24:40,531 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:24:40,532 Saving new checkpoint.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 19:24:40,681 Example #0\n",
      "2019-10-30 19:24:40,681 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:24:40,681 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:24:40,681 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:24:40,681 Example #1\n",
      "2019-10-30 19:24:40,682 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:24:40,682 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:24:40,682 \tHypothesis: Biblia haisemi kwamba mtu anapaswa kukubali sifa .\n",
      "2019-10-30 19:24:40,682 Example #2\n",
      "2019-10-30 19:24:40,682 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:24:40,682 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:24:40,682 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 19:24:40,682 Example #3\n",
      "2019-10-30 19:24:40,682 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:24:40,682 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:24:40,682 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 19:24:40,682 Validation result at epoch  17, step   153000: bleu:  36.16, loss: 30863.5801, ppl:   3.0740, duration: 44.8431s\n",
      "2019-10-30 19:24:55,863 Epoch  17 Step:   153100 Batch Loss:     1.042285 Tokens per Sec:    16825, Lr: 0.000300\n",
      "2019-10-30 19:25:11,043 Epoch  17 Step:   153200 Batch Loss:     1.201543 Tokens per Sec:    17112, Lr: 0.000300\n",
      "2019-10-30 19:25:26,310 Epoch  17 Step:   153300 Batch Loss:     1.321681 Tokens per Sec:    17104, Lr: 0.000300\n",
      "2019-10-30 19:25:41,583 Epoch  17 Step:   153400 Batch Loss:     1.180876 Tokens per Sec:    16864, Lr: 0.000300\n",
      "2019-10-30 19:25:56,452 Epoch  17 Step:   153500 Batch Loss:     1.081798 Tokens per Sec:    16944, Lr: 0.000300\n",
      "2019-10-30 19:26:11,585 Epoch  17 Step:   153600 Batch Loss:     1.297169 Tokens per Sec:    16428, Lr: 0.000300\n",
      "2019-10-30 19:26:26,666 Epoch  17 Step:   153700 Batch Loss:     1.369807 Tokens per Sec:    17062, Lr: 0.000300\n",
      "2019-10-30 19:26:41,911 Epoch  17 Step:   153800 Batch Loss:     1.150483 Tokens per Sec:    16790, Lr: 0.000300\n",
      "2019-10-30 19:26:57,106 Epoch  17 Step:   153900 Batch Loss:     1.198850 Tokens per Sec:    16861, Lr: 0.000300\n",
      "2019-10-30 19:27:12,481 Epoch  17 Step:   154000 Batch Loss:     1.318948 Tokens per Sec:    16805, Lr: 0.000300\n",
      "2019-10-30 19:27:57,313 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:27:57,313 Saving new checkpoint.\n",
      "2019-10-30 19:27:57,462 Example #0\n",
      "2019-10-30 19:27:57,462 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:27:57,462 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:27:57,462 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:27:57,462 Example #1\n",
      "2019-10-30 19:27:57,462 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:27:57,462 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:27:57,462 \tHypothesis: Biblia haisemi kwamba mtu anapaswa kukubali sifa .\n",
      "2019-10-30 19:27:57,462 Example #2\n",
      "2019-10-30 19:27:57,462 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:27:57,462 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:27:57,463 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-30 19:27:57,463 Example #3\n",
      "2019-10-30 19:27:57,463 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:27:57,463 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:27:57,463 \tHypothesis: Mapigo ya makumbatio na busu .\n",
      "2019-10-30 19:27:57,463 Validation result at epoch  17, step   154000: bleu:  36.57, loss: 30814.6680, ppl:   3.0685, duration: 44.9809s\n",
      "2019-10-30 19:28:12,845 Epoch  17 Step:   154100 Batch Loss:     1.237370 Tokens per Sec:    17205, Lr: 0.000300\n",
      "2019-10-30 19:28:27,848 Epoch  17 Step:   154200 Batch Loss:     1.274300 Tokens per Sec:    16880, Lr: 0.000300\n",
      "2019-10-30 19:28:43,109 Epoch  17 Step:   154300 Batch Loss:     1.268117 Tokens per Sec:    16751, Lr: 0.000300\n",
      "2019-10-30 19:28:58,361 Epoch  17 Step:   154400 Batch Loss:     1.489659 Tokens per Sec:    17001, Lr: 0.000300\n",
      "2019-10-30 19:29:13,594 Epoch  17 Step:   154500 Batch Loss:     1.121245 Tokens per Sec:    17154, Lr: 0.000300\n",
      "2019-10-30 19:29:28,864 Epoch  17 Step:   154600 Batch Loss:     1.340321 Tokens per Sec:    16353, Lr: 0.000300\n",
      "2019-10-30 19:29:44,293 Epoch  17 Step:   154700 Batch Loss:     1.166156 Tokens per Sec:    17060, Lr: 0.000300\n",
      "2019-10-30 19:29:59,593 Epoch  17 Step:   154800 Batch Loss:     1.265141 Tokens per Sec:    17064, Lr: 0.000300\n",
      "2019-10-30 19:30:15,010 Epoch  17 Step:   154900 Batch Loss:     1.262155 Tokens per Sec:    16943, Lr: 0.000300\n",
      "2019-10-30 19:30:30,091 Epoch  17 Step:   155000 Batch Loss:     1.156398 Tokens per Sec:    16661, Lr: 0.000300\n",
      "2019-10-30 19:31:14,636 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:31:14,636 Saving new checkpoint.\n",
      "2019-10-30 19:31:14,781 Example #0\n",
      "2019-10-30 19:31:14,782 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:31:14,782 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:31:14,782 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:31:14,782 Example #1\n",
      "2019-10-30 19:31:14,782 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:31:14,782 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:31:14,782 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kustahili .\n",
      "2019-10-30 19:31:14,782 Example #2\n",
      "2019-10-30 19:31:14,782 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:31:14,782 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:31:14,782 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 19:31:14,782 Example #3\n",
      "2019-10-30 19:31:14,782 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:31:14,782 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:31:14,782 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 19:31:14,783 Validation result at epoch  17, step   155000: bleu:  36.30, loss: 30763.0000, ppl:   3.0627, duration: 44.6906s\n",
      "2019-10-30 19:31:30,152 Epoch  17 Step:   155100 Batch Loss:     1.286391 Tokens per Sec:    17019, Lr: 0.000300\n",
      "2019-10-30 19:31:45,392 Epoch  17 Step:   155200 Batch Loss:     1.292687 Tokens per Sec:    16801, Lr: 0.000300\n",
      "2019-10-30 19:32:00,716 Epoch  17 Step:   155300 Batch Loss:     1.319944 Tokens per Sec:    16521, Lr: 0.000300\n",
      "2019-10-30 19:32:15,842 Epoch  17 Step:   155400 Batch Loss:     1.360505 Tokens per Sec:    16916, Lr: 0.000300\n",
      "2019-10-30 19:32:31,273 Epoch  17 Step:   155500 Batch Loss:     1.340303 Tokens per Sec:    16808, Lr: 0.000300\n",
      "2019-10-30 19:32:46,630 Epoch  17 Step:   155600 Batch Loss:     1.260600 Tokens per Sec:    16860, Lr: 0.000300\n",
      "2019-10-30 19:33:01,641 Epoch  17 Step:   155700 Batch Loss:     1.261232 Tokens per Sec:    16889, Lr: 0.000300\n",
      "2019-10-30 19:33:16,936 Epoch  17 Step:   155800 Batch Loss:     1.305854 Tokens per Sec:    17018, Lr: 0.000300\n",
      "2019-10-30 19:33:32,150 Epoch  17 Step:   155900 Batch Loss:     1.325399 Tokens per Sec:    16185, Lr: 0.000300\n",
      "2019-10-30 19:33:47,438 Epoch  17 Step:   156000 Batch Loss:     1.360082 Tokens per Sec:    16881, Lr: 0.000300\n",
      "2019-10-30 19:34:31,859 Example #0\n",
      "2019-10-30 19:34:31,859 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:34:31,859 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:34:31,859 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:34:31,859 Example #1\n",
      "2019-10-30 19:34:31,860 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:34:31,860 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:34:31,860 \tHypothesis: Biblia haiendelei kuwa na sifa ya kustahili .\n",
      "2019-10-30 19:34:31,860 Example #2\n",
      "2019-10-30 19:34:31,860 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:34:31,860 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:34:31,860 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 19:34:31,860 Example #3\n",
      "2019-10-30 19:34:31,860 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:34:31,860 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:34:31,860 \tHypothesis: Miti ya makumbatio na busu .\n",
      "2019-10-30 19:34:31,861 Validation result at epoch  17, step   156000: bleu:  36.13, loss: 30876.6035, ppl:   3.0754, duration: 44.4220s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 19:34:47,293 Epoch  17 Step:   156100 Batch Loss:     1.245958 Tokens per Sec:    16939, Lr: 0.000300\n",
      "2019-10-30 19:35:02,430 Epoch  17 Step:   156200 Batch Loss:     1.564103 Tokens per Sec:    16957, Lr: 0.000300\n",
      "2019-10-30 19:35:17,547 Epoch  17 Step:   156300 Batch Loss:     1.352366 Tokens per Sec:    17202, Lr: 0.000300\n",
      "2019-10-30 19:35:32,654 Epoch  17 Step:   156400 Batch Loss:     1.225413 Tokens per Sec:    16989, Lr: 0.000300\n",
      "2019-10-30 19:35:47,746 Epoch  17 Step:   156500 Batch Loss:     1.358704 Tokens per Sec:    16844, Lr: 0.000300\n",
      "2019-10-30 19:36:02,946 Epoch  17 Step:   156600 Batch Loss:     1.316393 Tokens per Sec:    16960, Lr: 0.000300\n",
      "2019-10-30 19:36:18,318 Epoch  17 Step:   156700 Batch Loss:     1.242395 Tokens per Sec:    16674, Lr: 0.000300\n",
      "2019-10-30 19:36:33,499 Epoch  17 Step:   156800 Batch Loss:     1.342937 Tokens per Sec:    16993, Lr: 0.000300\n",
      "2019-10-30 19:36:48,523 Epoch  17 Step:   156900 Batch Loss:     1.184473 Tokens per Sec:    16873, Lr: 0.000300\n",
      "2019-10-30 19:37:03,718 Epoch  17 Step:   157000 Batch Loss:     1.147076 Tokens per Sec:    16766, Lr: 0.000300\n",
      "2019-10-30 19:37:48,548 Example #0\n",
      "2019-10-30 19:37:48,548 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:37:48,549 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:37:48,549 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:37:48,549 Example #1\n",
      "2019-10-30 19:37:48,549 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:37:48,549 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:37:48,549 \tHypothesis: Biblia haishughulishi .\n",
      "2019-10-30 19:37:48,549 Example #2\n",
      "2019-10-30 19:37:48,549 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:37:48,549 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:37:48,549 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukukufanyaje ?\n",
      "2019-10-30 19:37:48,549 Example #3\n",
      "2019-10-30 19:37:48,549 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:37:48,549 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:37:48,549 \tHypothesis: Miti ya makumbatio na busu .\n",
      "2019-10-30 19:37:48,549 Validation result at epoch  17, step   157000: bleu:  36.15, loss: 30818.5645, ppl:   3.0689, duration: 44.8307s\n",
      "2019-10-30 19:38:03,794 Epoch  17 Step:   157100 Batch Loss:     1.272468 Tokens per Sec:    16941, Lr: 0.000300\n",
      "2019-10-30 19:38:18,952 Epoch  17 Step:   157200 Batch Loss:     1.239985 Tokens per Sec:    16812, Lr: 0.000300\n",
      "2019-10-30 19:38:34,246 Epoch  17 Step:   157300 Batch Loss:     1.218699 Tokens per Sec:    17143, Lr: 0.000300\n",
      "2019-10-30 19:38:49,500 Epoch  17 Step:   157400 Batch Loss:     1.191211 Tokens per Sec:    16758, Lr: 0.000300\n",
      "2019-10-30 19:39:04,604 Epoch  17 Step:   157500 Batch Loss:     1.367463 Tokens per Sec:    16931, Lr: 0.000300\n",
      "2019-10-30 19:39:20,006 Epoch  17 Step:   157600 Batch Loss:     1.211358 Tokens per Sec:    17108, Lr: 0.000300\n",
      "2019-10-30 19:39:35,566 Epoch  17 Step:   157700 Batch Loss:     1.408555 Tokens per Sec:    16534, Lr: 0.000300\n",
      "2019-10-30 19:39:50,856 Epoch  17 Step:   157800 Batch Loss:     1.480244 Tokens per Sec:    16740, Lr: 0.000300\n",
      "2019-10-30 19:40:06,021 Epoch  17 Step:   157900 Batch Loss:     1.339068 Tokens per Sec:    16849, Lr: 0.000300\n",
      "2019-10-30 19:40:20,847 Epoch  17 Step:   158000 Batch Loss:     1.170450 Tokens per Sec:    17011, Lr: 0.000300\n",
      "2019-10-30 19:41:05,282 Example #0\n",
      "2019-10-30 19:41:05,282 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:41:05,282 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:41:05,283 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:41:05,283 Example #1\n",
      "2019-10-30 19:41:05,283 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:41:05,283 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:41:05,283 \tHypothesis: Biblia haiendi ustahili .\n",
      "2019-10-30 19:41:05,283 Example #2\n",
      "2019-10-30 19:41:05,283 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:41:05,283 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:41:05,283 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 19:41:05,283 Example #3\n",
      "2019-10-30 19:41:05,283 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:41:05,283 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:41:05,283 \tHypothesis: Mapigo ya makumbatio na busu .\n",
      "2019-10-30 19:41:05,283 Validation result at epoch  17, step   158000: bleu:  36.56, loss: 30851.7129, ppl:   3.0726, duration: 44.4359s\n",
      "2019-10-30 19:41:20,498 Epoch  17 Step:   158100 Batch Loss:     1.236609 Tokens per Sec:    17251, Lr: 0.000300\n",
      "2019-10-30 19:41:35,481 Epoch  17 Step:   158200 Batch Loss:     1.208307 Tokens per Sec:    16920, Lr: 0.000300\n",
      "2019-10-30 19:41:50,729 Epoch  17 Step:   158300 Batch Loss:     1.585167 Tokens per Sec:    16475, Lr: 0.000300\n",
      "2019-10-30 19:42:05,877 Epoch  17 Step:   158400 Batch Loss:     1.289040 Tokens per Sec:    16780, Lr: 0.000300\n",
      "2019-10-30 19:42:21,122 Epoch  17 Step:   158500 Batch Loss:     1.282845 Tokens per Sec:    16793, Lr: 0.000300\n",
      "2019-10-30 19:42:36,714 Epoch  17 Step:   158600 Batch Loss:     1.253459 Tokens per Sec:    17095, Lr: 0.000300\n",
      "2019-10-30 19:42:52,053 Epoch  17 Step:   158700 Batch Loss:     1.193402 Tokens per Sec:    16975, Lr: 0.000300\n",
      "2019-10-30 19:43:07,203 Epoch  17 Step:   158800 Batch Loss:     1.316344 Tokens per Sec:    17063, Lr: 0.000300\n",
      "2019-10-30 19:43:22,272 Epoch  17 Step:   158900 Batch Loss:     1.343973 Tokens per Sec:    16502, Lr: 0.000300\n",
      "2019-10-30 19:43:37,595 Epoch  17 Step:   159000 Batch Loss:     1.411287 Tokens per Sec:    17098, Lr: 0.000300\n",
      "2019-10-30 19:44:22,451 Example #0\n",
      "2019-10-30 19:44:22,451 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:44:22,451 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:44:22,451 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:44:22,451 Example #1\n",
      "2019-10-30 19:44:22,451 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:44:22,451 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:44:22,451 \tHypothesis: Biblia haisemi kwamba mtu ana sifa .\n",
      "2019-10-30 19:44:22,451 Example #2\n",
      "2019-10-30 19:44:22,451 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:44:22,451 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:44:22,451 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 19:44:22,451 Example #3\n",
      "2019-10-30 19:44:22,452 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:44:22,452 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:44:22,452 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 19:44:22,452 Validation result at epoch  17, step   159000: bleu:  36.52, loss: 30773.6172, ppl:   3.0639, duration: 44.8567s\n",
      "2019-10-30 19:44:37,754 Epoch  17 Step:   159100 Batch Loss:     1.165453 Tokens per Sec:    16606, Lr: 0.000300\n",
      "2019-10-30 19:44:53,003 Epoch  17 Step:   159200 Batch Loss:     1.177133 Tokens per Sec:    17041, Lr: 0.000300\n",
      "2019-10-30 19:45:08,437 Epoch  17 Step:   159300 Batch Loss:     1.126228 Tokens per Sec:    16621, Lr: 0.000300\n",
      "2019-10-30 19:45:23,646 Epoch  17 Step:   159400 Batch Loss:     1.274497 Tokens per Sec:    16753, Lr: 0.000300\n",
      "2019-10-30 19:45:38,844 Epoch  17 Step:   159500 Batch Loss:     1.304387 Tokens per Sec:    17115, Lr: 0.000300\n",
      "2019-10-30 19:45:43,640 Epoch  17: total training loss 11883.43\n",
      "2019-10-30 19:45:43,640 EPOCH 18\n",
      "2019-10-30 19:45:55,233 Epoch  18 Step:   159600 Batch Loss:     1.319071 Tokens per Sec:    14986, Lr: 0.000300\n",
      "2019-10-30 19:46:10,348 Epoch  18 Step:   159700 Batch Loss:     1.275680 Tokens per Sec:    17214, Lr: 0.000300\n",
      "2019-10-30 19:46:25,616 Epoch  18 Step:   159800 Batch Loss:     1.329814 Tokens per Sec:    16710, Lr: 0.000300\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 19:46:40,728 Epoch  18 Step:   159900 Batch Loss:     1.254178 Tokens per Sec:    16704, Lr: 0.000300\n",
      "2019-10-30 19:46:55,926 Epoch  18 Step:   160000 Batch Loss:     1.173581 Tokens per Sec:    16715, Lr: 0.000300\n",
      "2019-10-30 19:47:40,563 Example #0\n",
      "2019-10-30 19:47:40,564 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:47:40,564 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:47:40,564 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:47:40,564 Example #1\n",
      "2019-10-30 19:47:40,564 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:47:40,564 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:47:40,564 \tHypothesis: Biblia haisemi kwamba mtu ana sifa .\n",
      "2019-10-30 19:47:40,564 Example #2\n",
      "2019-10-30 19:47:40,564 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:47:40,564 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:47:40,564 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 19:47:40,564 Example #3\n",
      "2019-10-30 19:47:40,565 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:47:40,565 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:47:40,565 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 19:47:40,565 Validation result at epoch  18, step   160000: bleu:  36.37, loss: 30843.4414, ppl:   3.0717, duration: 44.6385s\n",
      "2019-10-30 19:47:55,856 Epoch  18 Step:   160100 Batch Loss:     1.294109 Tokens per Sec:    16907, Lr: 0.000300\n",
      "2019-10-30 19:48:11,096 Epoch  18 Step:   160200 Batch Loss:     1.366931 Tokens per Sec:    17106, Lr: 0.000300\n",
      "2019-10-30 19:48:26,129 Epoch  18 Step:   160300 Batch Loss:     1.174002 Tokens per Sec:    17239, Lr: 0.000300\n",
      "2019-10-30 19:48:41,208 Epoch  18 Step:   160400 Batch Loss:     1.202843 Tokens per Sec:    16801, Lr: 0.000300\n",
      "2019-10-30 19:48:56,218 Epoch  18 Step:   160500 Batch Loss:     1.247769 Tokens per Sec:    16917, Lr: 0.000300\n",
      "2019-10-30 19:49:11,332 Epoch  18 Step:   160600 Batch Loss:     1.273895 Tokens per Sec:    16903, Lr: 0.000300\n",
      "2019-10-30 19:49:26,409 Epoch  18 Step:   160700 Batch Loss:     1.268689 Tokens per Sec:    16634, Lr: 0.000300\n",
      "2019-10-30 19:49:41,663 Epoch  18 Step:   160800 Batch Loss:     1.343122 Tokens per Sec:    17220, Lr: 0.000300\n",
      "2019-10-30 19:49:57,008 Epoch  18 Step:   160900 Batch Loss:     1.115717 Tokens per Sec:    16844, Lr: 0.000300\n",
      "2019-10-30 19:50:12,382 Epoch  18 Step:   161000 Batch Loss:     1.266323 Tokens per Sec:    16821, Lr: 0.000300\n",
      "2019-10-30 19:50:57,056 Example #0\n",
      "2019-10-30 19:50:57,056 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:50:57,056 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:50:57,056 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:50:57,056 Example #1\n",
      "2019-10-30 19:50:57,056 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:50:57,057 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:50:57,057 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kustahili .\n",
      "2019-10-30 19:50:57,057 Example #2\n",
      "2019-10-30 19:50:57,057 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:50:57,057 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:50:57,057 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 19:50:57,057 Example #3\n",
      "2019-10-30 19:50:57,057 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:50:57,057 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:50:57,057 \tHypothesis: Mikono ya makumbatio na mabusu .\n",
      "2019-10-30 19:50:57,057 Validation result at epoch  18, step   161000: bleu:  36.46, loss: 30862.9570, ppl:   3.0739, duration: 44.6742s\n",
      "2019-10-30 19:51:12,279 Epoch  18 Step:   161100 Batch Loss:     1.261525 Tokens per Sec:    16958, Lr: 0.000210\n",
      "2019-10-30 19:51:27,520 Epoch  18 Step:   161200 Batch Loss:     1.126362 Tokens per Sec:    16939, Lr: 0.000210\n",
      "2019-10-30 19:51:42,586 Epoch  18 Step:   161300 Batch Loss:     1.258019 Tokens per Sec:    16866, Lr: 0.000210\n",
      "2019-10-30 19:51:57,973 Epoch  18 Step:   161400 Batch Loss:     1.060682 Tokens per Sec:    16473, Lr: 0.000210\n",
      "2019-10-30 19:52:13,164 Epoch  18 Step:   161500 Batch Loss:     1.293781 Tokens per Sec:    16907, Lr: 0.000210\n",
      "2019-10-30 19:52:28,555 Epoch  18 Step:   161600 Batch Loss:     1.247512 Tokens per Sec:    17195, Lr: 0.000210\n",
      "2019-10-30 19:52:43,729 Epoch  18 Step:   161700 Batch Loss:     1.259162 Tokens per Sec:    17026, Lr: 0.000210\n",
      "2019-10-30 19:52:58,974 Epoch  18 Step:   161800 Batch Loss:     1.056582 Tokens per Sec:    17214, Lr: 0.000210\n",
      "2019-10-30 19:53:14,095 Epoch  18 Step:   161900 Batch Loss:     1.275141 Tokens per Sec:    16540, Lr: 0.000210\n",
      "2019-10-30 19:53:29,425 Epoch  18 Step:   162000 Batch Loss:     1.256301 Tokens per Sec:    16929, Lr: 0.000210\n",
      "2019-10-30 19:54:14,074 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:54:14,074 Saving new checkpoint.\n",
      "2019-10-30 19:54:14,221 Example #0\n",
      "2019-10-30 19:54:14,221 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:54:14,221 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:54:14,221 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:54:14,221 Example #1\n",
      "2019-10-30 19:54:14,221 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:54:14,221 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:54:14,221 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kudai ukweli .\n",
      "2019-10-30 19:54:14,221 Example #2\n",
      "2019-10-30 19:54:14,221 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:54:14,221 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:54:14,221 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 19:54:14,221 Example #3\n",
      "2019-10-30 19:54:14,222 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:54:14,222 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:54:14,222 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 19:54:14,222 Validation result at epoch  18, step   162000: bleu:  36.44, loss: 30537.5742, ppl:   3.0377, duration: 44.7965s\n",
      "2019-10-30 19:54:29,652 Epoch  18 Step:   162100 Batch Loss:     1.261258 Tokens per Sec:    16921, Lr: 0.000210\n",
      "2019-10-30 19:54:44,726 Epoch  18 Step:   162200 Batch Loss:     1.105515 Tokens per Sec:    17056, Lr: 0.000210\n",
      "2019-10-30 19:54:59,994 Epoch  18 Step:   162300 Batch Loss:     1.077771 Tokens per Sec:    17309, Lr: 0.000210\n",
      "2019-10-30 19:55:15,291 Epoch  18 Step:   162400 Batch Loss:     1.186782 Tokens per Sec:    17116, Lr: 0.000210\n",
      "2019-10-30 19:55:30,384 Epoch  18 Step:   162500 Batch Loss:     1.210697 Tokens per Sec:    16949, Lr: 0.000210\n",
      "2019-10-30 19:55:45,511 Epoch  18 Step:   162600 Batch Loss:     1.218688 Tokens per Sec:    16934, Lr: 0.000210\n",
      "2019-10-30 19:56:00,657 Epoch  18 Step:   162700 Batch Loss:     1.065111 Tokens per Sec:    17191, Lr: 0.000210\n",
      "2019-10-30 19:56:15,878 Epoch  18 Step:   162800 Batch Loss:     1.294979 Tokens per Sec:    16949, Lr: 0.000210\n",
      "2019-10-30 19:56:31,013 Epoch  18 Step:   162900 Batch Loss:     1.315203 Tokens per Sec:    16984, Lr: 0.000210\n",
      "2019-10-30 19:56:46,246 Epoch  18 Step:   163000 Batch Loss:     1.205967 Tokens per Sec:    16476, Lr: 0.000210\n",
      "2019-10-30 19:57:30,666 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 19:57:30,667 Saving new checkpoint.\n",
      "2019-10-30 19:57:30,817 Example #0\n",
      "2019-10-30 19:57:30,818 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 19:57:30,818 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 19:57:30,818 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 19:57:30,818 Example #1\n",
      "2019-10-30 19:57:30,818 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 19:57:30,818 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 19:57:30,818 \tHypothesis: Biblia haidumii sifa ya ukarimu .\n",
      "2019-10-30 19:57:30,818 Example #2\n",
      "2019-10-30 19:57:30,818 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 19:57:30,818 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 19:57:30,818 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 19:57:30,818 Example #3\n",
      "2019-10-30 19:57:30,818 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 19:57:30,818 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 19:57:30,819 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 19:57:30,819 Validation result at epoch  18, step   163000: bleu:  36.45, loss: 30440.1348, ppl:   3.0270, duration: 44.5715s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 19:57:45,947 Epoch  18 Step:   163100 Batch Loss:     1.137918 Tokens per Sec:    16883, Lr: 0.000210\n",
      "2019-10-30 19:58:01,104 Epoch  18 Step:   163200 Batch Loss:     1.615095 Tokens per Sec:    17471, Lr: 0.000210\n",
      "2019-10-30 19:58:16,457 Epoch  18 Step:   163300 Batch Loss:     1.164998 Tokens per Sec:    17183, Lr: 0.000210\n",
      "2019-10-30 19:58:31,790 Epoch  18 Step:   163400 Batch Loss:     1.265183 Tokens per Sec:    16938, Lr: 0.000210\n",
      "2019-10-30 19:58:47,080 Epoch  18 Step:   163500 Batch Loss:     1.184676 Tokens per Sec:    16987, Lr: 0.000210\n",
      "2019-10-30 19:59:02,214 Epoch  18 Step:   163600 Batch Loss:     1.341150 Tokens per Sec:    17138, Lr: 0.000210\n",
      "2019-10-30 19:59:17,533 Epoch  18 Step:   163700 Batch Loss:     1.167970 Tokens per Sec:    16883, Lr: 0.000210\n",
      "2019-10-30 19:59:32,576 Epoch  18 Step:   163800 Batch Loss:     1.144973 Tokens per Sec:    16809, Lr: 0.000210\n",
      "2019-10-30 19:59:47,662 Epoch  18 Step:   163900 Batch Loss:     1.248380 Tokens per Sec:    16784, Lr: 0.000210\n",
      "2019-10-30 20:00:02,601 Epoch  18 Step:   164000 Batch Loss:     1.229164 Tokens per Sec:    16856, Lr: 0.000210\n",
      "2019-10-30 20:00:47,127 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:00:47,128 Saving new checkpoint.\n",
      "2019-10-30 20:00:47,270 Example #0\n",
      "2019-10-30 20:00:47,270 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:00:47,270 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:00:47,270 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:00:47,270 Example #1\n",
      "2019-10-30 20:00:47,270 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:00:47,270 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:00:47,270 \tHypothesis: Biblia haidumii kuwa na sifa ya kukubali .\n",
      "2019-10-30 20:00:47,270 Example #2\n",
      "2019-10-30 20:00:47,271 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:00:47,271 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:00:47,271 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:00:47,271 Example #3\n",
      "2019-10-30 20:00:47,271 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:00:47,271 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:00:47,271 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 20:00:47,271 Validation result at epoch  18, step   164000: bleu:  36.35, loss: 30418.2461, ppl:   3.0245, duration: 44.6700s\n",
      "2019-10-30 20:01:02,671 Epoch  18 Step:   164100 Batch Loss:     1.323315 Tokens per Sec:    17027, Lr: 0.000210\n",
      "2019-10-30 20:01:17,823 Epoch  18 Step:   164200 Batch Loss:     1.174056 Tokens per Sec:    17045, Lr: 0.000210\n",
      "2019-10-30 20:01:32,940 Epoch  18 Step:   164300 Batch Loss:     1.059824 Tokens per Sec:    17040, Lr: 0.000210\n",
      "2019-10-30 20:01:48,343 Epoch  18 Step:   164400 Batch Loss:     1.533371 Tokens per Sec:    16904, Lr: 0.000210\n",
      "2019-10-30 20:02:03,518 Epoch  18 Step:   164500 Batch Loss:     1.179893 Tokens per Sec:    16927, Lr: 0.000210\n",
      "2019-10-30 20:02:18,721 Epoch  18 Step:   164600 Batch Loss:     1.036696 Tokens per Sec:    17048, Lr: 0.000210\n",
      "2019-10-30 20:02:34,305 Epoch  18 Step:   164700 Batch Loss:     1.249154 Tokens per Sec:    16801, Lr: 0.000210\n",
      "2019-10-30 20:02:49,524 Epoch  18 Step:   164800 Batch Loss:     1.210328 Tokens per Sec:    16515, Lr: 0.000210\n",
      "2019-10-30 20:03:04,938 Epoch  18 Step:   164900 Batch Loss:     1.195243 Tokens per Sec:    16528, Lr: 0.000210\n",
      "2019-10-30 20:03:20,134 Epoch  18 Step:   165000 Batch Loss:     1.250687 Tokens per Sec:    17093, Lr: 0.000210\n",
      "2019-10-30 20:04:04,976 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:04:04,977 Saving new checkpoint.\n",
      "2019-10-30 20:04:05,124 Example #0\n",
      "2019-10-30 20:04:05,124 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:04:05,124 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:04:05,124 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:04:05,124 Example #1\n",
      "2019-10-30 20:04:05,125 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:04:05,125 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:04:05,125 \tHypothesis: Biblia haisemi kwamba mtu ana sifa nzuri .\n",
      "2019-10-30 20:04:05,125 Example #2\n",
      "2019-10-30 20:04:05,125 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:04:05,125 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:04:05,125 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:04:05,125 Example #3\n",
      "2019-10-30 20:04:05,125 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:04:05,125 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:04:05,125 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 20:04:05,125 Validation result at epoch  18, step   165000: bleu:  36.58, loss: 30333.6855, ppl:   3.0153, duration: 44.9909s\n",
      "2019-10-30 20:04:20,521 Epoch  18 Step:   165100 Batch Loss:     1.493602 Tokens per Sec:    16532, Lr: 0.000210\n",
      "2019-10-30 20:04:35,567 Epoch  18 Step:   165200 Batch Loss:     1.190054 Tokens per Sec:    17023, Lr: 0.000210\n",
      "2019-10-30 20:04:50,689 Epoch  18 Step:   165300 Batch Loss:     1.306011 Tokens per Sec:    16760, Lr: 0.000210\n",
      "2019-10-30 20:05:05,893 Epoch  18 Step:   165400 Batch Loss:     1.279737 Tokens per Sec:    16881, Lr: 0.000210\n",
      "2019-10-30 20:05:21,018 Epoch  18 Step:   165500 Batch Loss:     1.372440 Tokens per Sec:    17218, Lr: 0.000210\n",
      "2019-10-30 20:05:36,247 Epoch  18 Step:   165600 Batch Loss:     1.222129 Tokens per Sec:    17062, Lr: 0.000210\n",
      "2019-10-30 20:05:51,634 Epoch  18 Step:   165700 Batch Loss:     1.081993 Tokens per Sec:    16559, Lr: 0.000210\n",
      "2019-10-30 20:06:06,788 Epoch  18 Step:   165800 Batch Loss:     1.282286 Tokens per Sec:    17227, Lr: 0.000210\n",
      "2019-10-30 20:06:21,886 Epoch  18 Step:   165900 Batch Loss:     1.176910 Tokens per Sec:    16840, Lr: 0.000210\n",
      "2019-10-30 20:06:37,218 Epoch  18 Step:   166000 Batch Loss:     1.146613 Tokens per Sec:    16762, Lr: 0.000210\n",
      "2019-10-30 20:07:21,623 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:07:21,624 Saving new checkpoint.\n",
      "2019-10-30 20:07:21,773 Example #0\n",
      "2019-10-30 20:07:21,773 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:07:21,773 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:07:21,773 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:07:21,773 Example #1\n",
      "2019-10-30 20:07:21,774 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:07:21,774 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:07:21,774 \tHypothesis: Biblia haisemi kwamba mtu anapaswa kuwa na sifa nzuri .\n",
      "2019-10-30 20:07:21,774 Example #2\n",
      "2019-10-30 20:07:21,774 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:07:21,774 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:07:21,774 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 20:07:21,774 Example #3\n",
      "2019-10-30 20:07:21,774 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:07:21,774 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:07:21,774 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 20:07:21,774 Validation result at epoch  18, step   166000: bleu:  36.58, loss: 30200.5664, ppl:   3.0007, duration: 44.5555s\n",
      "2019-10-30 20:07:36,864 Epoch  18 Step:   166100 Batch Loss:     1.159460 Tokens per Sec:    16994, Lr: 0.000210\n",
      "2019-10-30 20:07:52,048 Epoch  18 Step:   166200 Batch Loss:     1.124269 Tokens per Sec:    16654, Lr: 0.000210\n",
      "2019-10-30 20:08:07,567 Epoch  18 Step:   166300 Batch Loss:     1.168533 Tokens per Sec:    16613, Lr: 0.000210\n",
      "2019-10-30 20:08:23,050 Epoch  18 Step:   166400 Batch Loss:     1.508017 Tokens per Sec:    16850, Lr: 0.000210\n",
      "2019-10-30 20:08:38,159 Epoch  18 Step:   166500 Batch Loss:     1.098642 Tokens per Sec:    16765, Lr: 0.000210\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 20:08:53,375 Epoch  18 Step:   166600 Batch Loss:     1.169299 Tokens per Sec:    17076, Lr: 0.000210\n",
      "2019-10-30 20:09:08,513 Epoch  18 Step:   166700 Batch Loss:     1.275020 Tokens per Sec:    16889, Lr: 0.000210\n",
      "2019-10-30 20:09:23,657 Epoch  18 Step:   166800 Batch Loss:     1.129969 Tokens per Sec:    17168, Lr: 0.000210\n",
      "2019-10-30 20:09:38,758 Epoch  18 Step:   166900 Batch Loss:     1.248040 Tokens per Sec:    17209, Lr: 0.000210\n",
      "2019-10-30 20:09:54,088 Epoch  18 Step:   167000 Batch Loss:     1.058634 Tokens per Sec:    16953, Lr: 0.000210\n",
      "2019-10-30 20:10:38,420 Example #0\n",
      "2019-10-30 20:10:38,421 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:10:38,421 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:10:38,421 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:10:38,421 Example #1\n",
      "2019-10-30 20:10:38,421 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:10:38,421 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:10:38,421 \tHypothesis: Biblia haidumii kuwa na sifa ya ustahili .\n",
      "2019-10-30 20:10:38,421 Example #2\n",
      "2019-10-30 20:10:38,421 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:10:38,421 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:10:38,421 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 20:10:38,421 Example #3\n",
      "2019-10-30 20:10:38,422 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:10:38,422 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:10:38,422 \tHypothesis: Vyakula na busu .\n",
      "2019-10-30 20:10:38,422 Validation result at epoch  18, step   167000: bleu:  36.60, loss: 30290.3398, ppl:   3.0105, duration: 44.3328s\n",
      "2019-10-30 20:10:53,549 Epoch  18 Step:   167100 Batch Loss:     1.344986 Tokens per Sec:    17231, Lr: 0.000210\n",
      "2019-10-30 20:11:08,681 Epoch  18 Step:   167200 Batch Loss:     1.211796 Tokens per Sec:    16620, Lr: 0.000210\n",
      "2019-10-30 20:11:23,744 Epoch  18 Step:   167300 Batch Loss:     1.383071 Tokens per Sec:    17060, Lr: 0.000210\n",
      "2019-10-30 20:11:38,996 Epoch  18 Step:   167400 Batch Loss:     1.145980 Tokens per Sec:    16925, Lr: 0.000210\n",
      "2019-10-30 20:11:54,146 Epoch  18 Step:   167500 Batch Loss:     1.228930 Tokens per Sec:    16824, Lr: 0.000210\n",
      "2019-10-30 20:12:09,512 Epoch  18 Step:   167600 Batch Loss:     1.311080 Tokens per Sec:    16574, Lr: 0.000210\n",
      "2019-10-30 20:12:24,861 Epoch  18 Step:   167700 Batch Loss:     1.132678 Tokens per Sec:    16796, Lr: 0.000210\n",
      "2019-10-30 20:12:40,063 Epoch  18 Step:   167800 Batch Loss:     1.568928 Tokens per Sec:    16825, Lr: 0.000210\n",
      "2019-10-30 20:12:54,878 Epoch  18 Step:   167900 Batch Loss:     1.420977 Tokens per Sec:    16938, Lr: 0.000210\n",
      "2019-10-30 20:13:09,862 Epoch  18 Step:   168000 Batch Loss:     1.295819 Tokens per Sec:    16829, Lr: 0.000210\n",
      "2019-10-30 20:13:54,362 Example #0\n",
      "2019-10-30 20:13:54,362 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:13:54,362 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:13:54,362 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:13:54,362 Example #1\n",
      "2019-10-30 20:13:54,362 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:13:54,362 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:13:54,363 \tHypothesis: Biblia haidumu sifa .\n",
      "2019-10-30 20:13:54,363 Example #2\n",
      "2019-10-30 20:13:54,363 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:13:54,363 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:13:54,363 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 20:13:54,363 Example #3\n",
      "2019-10-30 20:13:54,363 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:13:54,363 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:13:54,363 \tHypothesis: Maua na busu .\n",
      "2019-10-30 20:13:54,363 Validation result at epoch  18, step   168000: bleu:  36.65, loss: 30225.7031, ppl:   3.0034, duration: 44.5007s\n",
      "2019-10-30 20:14:09,474 Epoch  18 Step:   168100 Batch Loss:     1.281242 Tokens per Sec:    16999, Lr: 0.000210\n",
      "2019-10-30 20:14:24,691 Epoch  18 Step:   168200 Batch Loss:     1.295966 Tokens per Sec:    17033, Lr: 0.000210\n",
      "2019-10-30 20:14:39,891 Epoch  18 Step:   168300 Batch Loss:     1.176017 Tokens per Sec:    17228, Lr: 0.000210\n",
      "2019-10-30 20:14:54,903 Epoch  18 Step:   168400 Batch Loss:     1.223194 Tokens per Sec:    16704, Lr: 0.000210\n",
      "2019-10-30 20:15:09,871 Epoch  18 Step:   168500 Batch Loss:     1.239384 Tokens per Sec:    17002, Lr: 0.000210\n",
      "2019-10-30 20:15:25,031 Epoch  18 Step:   168600 Batch Loss:     1.672403 Tokens per Sec:    16499, Lr: 0.000210\n",
      "2019-10-30 20:15:40,351 Epoch  18 Step:   168700 Batch Loss:     1.164077 Tokens per Sec:    16499, Lr: 0.000210\n",
      "2019-10-30 20:15:55,517 Epoch  18 Step:   168800 Batch Loss:     1.162398 Tokens per Sec:    16839, Lr: 0.000210\n",
      "2019-10-30 20:16:10,671 Epoch  18 Step:   168900 Batch Loss:     1.363044 Tokens per Sec:    17196, Lr: 0.000210\n",
      "2019-10-30 20:16:12,210 Epoch  18: total training loss 11654.86\n",
      "2019-10-30 20:16:12,211 EPOCH 19\n",
      "2019-10-30 20:16:26,800 Epoch  19 Step:   169000 Batch Loss:     1.218347 Tokens per Sec:    15577, Lr: 0.000210\n",
      "2019-10-30 20:17:11,312 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:17:11,312 Saving new checkpoint.\n",
      "2019-10-30 20:17:11,460 Example #0\n",
      "2019-10-30 20:17:11,460 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:17:11,460 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:17:11,460 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:17:11,460 Example #1\n",
      "2019-10-30 20:17:11,461 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:17:11,461 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:17:11,461 \tHypothesis: Biblia haidumii sifa .\n",
      "2019-10-30 20:17:11,461 Example #2\n",
      "2019-10-30 20:17:11,461 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:17:11,461 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:17:11,461 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:17:11,461 Example #3\n",
      "2019-10-30 20:17:11,461 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:17:11,461 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:17:11,461 \tHypothesis: Miguu ya makumbi na mabusu .\n",
      "2019-10-30 20:17:11,461 Validation result at epoch  19, step   169000: bleu:  36.61, loss: 30177.4199, ppl:   2.9982, duration: 44.6609s\n",
      "2019-10-30 20:17:26,761 Epoch  19 Step:   169100 Batch Loss:     1.164982 Tokens per Sec:    16717, Lr: 0.000210\n",
      "2019-10-30 20:17:41,884 Epoch  19 Step:   169200 Batch Loss:     1.204698 Tokens per Sec:    16915, Lr: 0.000210\n",
      "2019-10-30 20:17:56,839 Epoch  19 Step:   169300 Batch Loss:     1.372404 Tokens per Sec:    17249, Lr: 0.000210\n",
      "2019-10-30 20:18:12,093 Epoch  19 Step:   169400 Batch Loss:     1.359704 Tokens per Sec:    16879, Lr: 0.000210\n",
      "2019-10-30 20:18:27,303 Epoch  19 Step:   169500 Batch Loss:     1.142899 Tokens per Sec:    17174, Lr: 0.000210\n",
      "2019-10-30 20:18:42,352 Epoch  19 Step:   169600 Batch Loss:     1.327133 Tokens per Sec:    16952, Lr: 0.000210\n",
      "2019-10-30 20:18:57,279 Epoch  19 Step:   169700 Batch Loss:     1.225493 Tokens per Sec:    17003, Lr: 0.000210\n",
      "2019-10-30 20:19:12,469 Epoch  19 Step:   169800 Batch Loss:     1.282857 Tokens per Sec:    17328, Lr: 0.000210\n",
      "2019-10-30 20:19:27,624 Epoch  19 Step:   169900 Batch Loss:     1.366768 Tokens per Sec:    16947, Lr: 0.000210\n",
      "2019-10-30 20:19:42,491 Epoch  19 Step:   170000 Batch Loss:     1.206115 Tokens per Sec:    17075, Lr: 0.000210\n",
      "2019-10-30 20:20:27,085 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:20:27,085 Saving new checkpoint.\n",
      "2019-10-30 20:20:27,238 Example #0\n",
      "2019-10-30 20:20:27,239 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:20:27,239 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:20:27,239 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:20:27,239 Example #1\n",
      "2019-10-30 20:20:27,239 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:20:27,239 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:20:27,239 \tHypothesis: Biblia haidumii sifa .\n",
      "2019-10-30 20:20:27,239 Example #2\n",
      "2019-10-30 20:20:27,239 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:20:27,239 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:20:27,239 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:20:27,239 Example #3\n",
      "2019-10-30 20:20:27,239 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:20:27,239 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:20:27,240 \tHypothesis: Mizigo na busu .\n",
      "2019-10-30 20:20:27,240 Validation result at epoch  19, step   170000: bleu:  36.49, loss: 30123.8125, ppl:   2.9923, duration: 44.7483s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 20:20:42,308 Epoch  19 Step:   170100 Batch Loss:     1.243415 Tokens per Sec:    16721, Lr: 0.000210\n",
      "2019-10-30 20:20:57,460 Epoch  19 Step:   170200 Batch Loss:     1.244234 Tokens per Sec:    16914, Lr: 0.000210\n",
      "2019-10-30 20:21:12,756 Epoch  19 Step:   170300 Batch Loss:     1.258534 Tokens per Sec:    16901, Lr: 0.000210\n",
      "2019-10-30 20:21:28,027 Epoch  19 Step:   170400 Batch Loss:     1.078662 Tokens per Sec:    16821, Lr: 0.000210\n",
      "2019-10-30 20:21:43,225 Epoch  19 Step:   170500 Batch Loss:     1.039290 Tokens per Sec:    17016, Lr: 0.000210\n",
      "2019-10-30 20:21:58,204 Epoch  19 Step:   170600 Batch Loss:     1.257514 Tokens per Sec:    17048, Lr: 0.000210\n",
      "2019-10-30 20:22:13,745 Epoch  19 Step:   170700 Batch Loss:     1.169393 Tokens per Sec:    16987, Lr: 0.000210\n",
      "2019-10-30 20:22:29,260 Epoch  19 Step:   170800 Batch Loss:     1.280177 Tokens per Sec:    16780, Lr: 0.000210\n",
      "2019-10-30 20:22:44,633 Epoch  19 Step:   170900 Batch Loss:     1.259434 Tokens per Sec:    16839, Lr: 0.000210\n",
      "2019-10-30 20:22:59,903 Epoch  19 Step:   171000 Batch Loss:     1.229532 Tokens per Sec:    16924, Lr: 0.000210\n",
      "2019-10-30 20:23:44,288 Example #0\n",
      "2019-10-30 20:23:44,289 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:23:44,289 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:23:44,289 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:23:44,289 Example #1\n",
      "2019-10-30 20:23:44,289 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:23:44,289 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:23:44,289 \tHypothesis: Biblia haidumii sifa ya uthabiti .\n",
      "2019-10-30 20:23:44,289 Example #2\n",
      "2019-10-30 20:23:44,289 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:23:44,289 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:23:44,289 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:23:44,289 Example #3\n",
      "2019-10-30 20:23:44,289 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:23:44,289 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:23:44,290 \tHypothesis: Vitu vya kukumbatia na busu .\n",
      "2019-10-30 20:23:44,290 Validation result at epoch  19, step   171000: bleu:  36.57, loss: 30197.5918, ppl:   3.0004, duration: 44.3858s\n",
      "2019-10-30 20:23:59,666 Epoch  19 Step:   171100 Batch Loss:     1.236889 Tokens per Sec:    16961, Lr: 0.000210\n",
      "2019-10-30 20:24:14,833 Epoch  19 Step:   171200 Batch Loss:     1.263221 Tokens per Sec:    17067, Lr: 0.000210\n",
      "2019-10-30 20:24:30,030 Epoch  19 Step:   171300 Batch Loss:     1.233663 Tokens per Sec:    17021, Lr: 0.000210\n",
      "2019-10-30 20:24:45,086 Epoch  19 Step:   171400 Batch Loss:     0.869421 Tokens per Sec:    17073, Lr: 0.000210\n",
      "2019-10-30 20:25:00,091 Epoch  19 Step:   171500 Batch Loss:     0.921747 Tokens per Sec:    16788, Lr: 0.000210\n",
      "2019-10-30 20:25:15,304 Epoch  19 Step:   171600 Batch Loss:     1.158996 Tokens per Sec:    17076, Lr: 0.000210\n",
      "2019-10-30 20:25:30,340 Epoch  19 Step:   171700 Batch Loss:     1.059600 Tokens per Sec:    17003, Lr: 0.000210\n",
      "2019-10-30 20:25:45,548 Epoch  19 Step:   171800 Batch Loss:     1.283834 Tokens per Sec:    16928, Lr: 0.000210\n",
      "2019-10-30 20:26:00,577 Epoch  19 Step:   171900 Batch Loss:     1.069551 Tokens per Sec:    17251, Lr: 0.000210\n",
      "2019-10-30 20:26:15,731 Epoch  19 Step:   172000 Batch Loss:     1.353219 Tokens per Sec:    17034, Lr: 0.000210\n",
      "2019-10-30 20:27:00,201 Example #0\n",
      "2019-10-30 20:27:00,201 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:27:00,201 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:27:00,201 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:27:00,201 Example #1\n",
      "2019-10-30 20:27:00,201 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:27:00,201 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:27:00,201 \tHypothesis: Biblia haidai sifa .\n",
      "2019-10-30 20:27:00,201 Example #2\n",
      "2019-10-30 20:27:00,202 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:27:00,202 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:27:00,202 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:27:00,202 Example #3\n",
      "2019-10-30 20:27:00,202 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:27:00,202 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:27:00,202 \tHypothesis: Maua na busu .\n",
      "2019-10-30 20:27:00,202 Validation result at epoch  19, step   172000: bleu:  36.95, loss: 30229.2930, ppl:   3.0038, duration: 44.4704s\n",
      "2019-10-30 20:27:15,551 Epoch  19 Step:   172100 Batch Loss:     1.269140 Tokens per Sec:    16818, Lr: 0.000210\n",
      "2019-10-30 20:27:30,516 Epoch  19 Step:   172200 Batch Loss:     1.076837 Tokens per Sec:    17025, Lr: 0.000210\n",
      "2019-10-30 20:27:45,615 Epoch  19 Step:   172300 Batch Loss:     1.124279 Tokens per Sec:    17071, Lr: 0.000210\n",
      "2019-10-30 20:28:00,749 Epoch  19 Step:   172400 Batch Loss:     1.207156 Tokens per Sec:    17294, Lr: 0.000210\n",
      "2019-10-30 20:28:15,858 Epoch  19 Step:   172500 Batch Loss:     1.289950 Tokens per Sec:    17112, Lr: 0.000210\n",
      "2019-10-30 20:28:31,163 Epoch  19 Step:   172600 Batch Loss:     1.344469 Tokens per Sec:    17009, Lr: 0.000210\n",
      "2019-10-30 20:28:46,340 Epoch  19 Step:   172700 Batch Loss:     1.395403 Tokens per Sec:    16879, Lr: 0.000210\n",
      "2019-10-30 20:29:01,438 Epoch  19 Step:   172800 Batch Loss:     1.168459 Tokens per Sec:    16986, Lr: 0.000210\n",
      "2019-10-30 20:29:16,454 Epoch  19 Step:   172900 Batch Loss:     1.149828 Tokens per Sec:    17122, Lr: 0.000210\n",
      "2019-10-30 20:29:31,473 Epoch  19 Step:   173000 Batch Loss:     1.167848 Tokens per Sec:    17187, Lr: 0.000210\n",
      "2019-10-30 20:30:15,656 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:30:15,656 Saving new checkpoint.\n",
      "2019-10-30 20:30:15,798 Example #0\n",
      "2019-10-30 20:30:15,798 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:30:15,798 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:30:15,798 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:30:15,798 Example #1\n",
      "2019-10-30 20:30:15,799 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:30:15,799 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:30:15,799 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya ustahili .\n",
      "2019-10-30 20:30:15,799 Example #2\n",
      "2019-10-30 20:30:15,799 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:30:15,799 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:30:15,799 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:30:15,799 Example #3\n",
      "2019-10-30 20:30:15,799 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:30:15,799 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:30:15,799 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 20:30:15,799 Validation result at epoch  19, step   173000: bleu:  36.64, loss: 30027.2773, ppl:   2.9818, duration: 44.3256s\n",
      "2019-10-30 20:30:30,768 Epoch  19 Step:   173100 Batch Loss:     1.103242 Tokens per Sec:    17063, Lr: 0.000210\n",
      "2019-10-30 20:30:45,796 Epoch  19 Step:   173200 Batch Loss:     1.344423 Tokens per Sec:    16753, Lr: 0.000210\n",
      "2019-10-30 20:31:00,922 Epoch  19 Step:   173300 Batch Loss:     1.405547 Tokens per Sec:    16651, Lr: 0.000210\n",
      "2019-10-30 20:31:16,365 Epoch  19 Step:   173400 Batch Loss:     1.301607 Tokens per Sec:    16921, Lr: 0.000210\n",
      "2019-10-30 20:31:31,173 Epoch  19 Step:   173500 Batch Loss:     1.323263 Tokens per Sec:    16759, Lr: 0.000210\n",
      "2019-10-30 20:31:46,368 Epoch  19 Step:   173600 Batch Loss:     1.198779 Tokens per Sec:    17330, Lr: 0.000210\n",
      "2019-10-30 20:32:01,564 Epoch  19 Step:   173700 Batch Loss:     1.262989 Tokens per Sec:    17479, Lr: 0.000210\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 20:32:16,816 Epoch  19 Step:   173800 Batch Loss:     1.251721 Tokens per Sec:    16864, Lr: 0.000210\n",
      "2019-10-30 20:32:32,046 Epoch  19 Step:   173900 Batch Loss:     1.258760 Tokens per Sec:    17100, Lr: 0.000210\n",
      "2019-10-30 20:32:47,398 Epoch  19 Step:   174000 Batch Loss:     1.327185 Tokens per Sec:    16712, Lr: 0.000210\n",
      "2019-10-30 20:33:31,924 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:33:31,924 Saving new checkpoint.\n",
      "2019-10-30 20:33:32,074 Example #0\n",
      "2019-10-30 20:33:32,075 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:33:32,075 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:33:32,075 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:33:32,075 Example #1\n",
      "2019-10-30 20:33:32,075 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:33:32,075 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:33:32,075 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya ustahili .\n",
      "2019-10-30 20:33:32,075 Example #2\n",
      "2019-10-30 20:33:32,075 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:33:32,075 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:33:32,075 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 20:33:32,075 Example #3\n",
      "2019-10-30 20:33:32,075 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:33:32,075 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:33:32,076 \tHypothesis: Viganja vya makumbatio na busu .\n",
      "2019-10-30 20:33:32,076 Validation result at epoch  19, step   174000: bleu:  37.01, loss: 30011.8418, ppl:   2.9802, duration: 44.6767s\n",
      "2019-10-30 20:33:47,432 Epoch  19 Step:   174100 Batch Loss:     1.261401 Tokens per Sec:    16627, Lr: 0.000210\n",
      "2019-10-30 20:34:02,348 Epoch  19 Step:   174200 Batch Loss:     1.282682 Tokens per Sec:    17087, Lr: 0.000210\n",
      "2019-10-30 20:34:17,387 Epoch  19 Step:   174300 Batch Loss:     1.374914 Tokens per Sec:    16847, Lr: 0.000210\n",
      "2019-10-30 20:34:32,530 Epoch  19 Step:   174400 Batch Loss:     1.162338 Tokens per Sec:    16978, Lr: 0.000210\n",
      "2019-10-30 20:34:47,733 Epoch  19 Step:   174500 Batch Loss:     1.235536 Tokens per Sec:    16897, Lr: 0.000210\n",
      "2019-10-30 20:35:03,046 Epoch  19 Step:   174600 Batch Loss:     1.186296 Tokens per Sec:    16935, Lr: 0.000210\n",
      "2019-10-30 20:35:18,143 Epoch  19 Step:   174700 Batch Loss:     1.065387 Tokens per Sec:    17039, Lr: 0.000210\n",
      "2019-10-30 20:35:33,291 Epoch  19 Step:   174800 Batch Loss:     1.143026 Tokens per Sec:    17114, Lr: 0.000210\n",
      "2019-10-30 20:35:48,523 Epoch  19 Step:   174900 Batch Loss:     1.174230 Tokens per Sec:    16982, Lr: 0.000210\n",
      "2019-10-30 20:36:03,514 Epoch  19 Step:   175000 Batch Loss:     1.283409 Tokens per Sec:    16853, Lr: 0.000210\n",
      "2019-10-30 20:36:47,912 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:36:47,912 Saving new checkpoint.\n",
      "2019-10-30 20:36:48,060 Example #0\n",
      "2019-10-30 20:36:48,061 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:36:48,061 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:36:48,061 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:36:48,061 Example #1\n",
      "2019-10-30 20:36:48,061 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:36:48,061 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:36:48,061 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali .\n",
      "2019-10-30 20:36:48,061 Example #2\n",
      "2019-10-30 20:36:48,061 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:36:48,061 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:36:48,061 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukukuaje ?\n",
      "2019-10-30 20:36:48,061 Example #3\n",
      "2019-10-30 20:36:48,061 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:36:48,061 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:36:48,062 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 20:36:48,062 Validation result at epoch  19, step   175000: bleu:  37.09, loss: 29989.2598, ppl:   2.9777, duration: 44.5469s\n",
      "2019-10-30 20:37:03,149 Epoch  19 Step:   175100 Batch Loss:     1.232152 Tokens per Sec:    16715, Lr: 0.000210\n",
      "2019-10-30 20:37:18,283 Epoch  19 Step:   175200 Batch Loss:     1.176266 Tokens per Sec:    17103, Lr: 0.000210\n",
      "2019-10-30 20:37:33,423 Epoch  19 Step:   175300 Batch Loss:     1.169291 Tokens per Sec:    17041, Lr: 0.000210\n",
      "2019-10-30 20:37:48,358 Epoch  19 Step:   175400 Batch Loss:     1.379339 Tokens per Sec:    17216, Lr: 0.000210\n",
      "2019-10-30 20:38:03,345 Epoch  19 Step:   175500 Batch Loss:     1.117328 Tokens per Sec:    17253, Lr: 0.000210\n",
      "2019-10-30 20:38:18,327 Epoch  19 Step:   175600 Batch Loss:     1.132014 Tokens per Sec:    17050, Lr: 0.000210\n",
      "2019-10-30 20:38:33,260 Epoch  19 Step:   175700 Batch Loss:     1.196284 Tokens per Sec:    17086, Lr: 0.000210\n",
      "2019-10-30 20:38:48,401 Epoch  19 Step:   175800 Batch Loss:     1.165941 Tokens per Sec:    17133, Lr: 0.000210\n",
      "2019-10-30 20:39:03,457 Epoch  19 Step:   175900 Batch Loss:     1.148011 Tokens per Sec:    16916, Lr: 0.000210\n",
      "2019-10-30 20:39:18,844 Epoch  19 Step:   176000 Batch Loss:     1.188490 Tokens per Sec:    17083, Lr: 0.000210\n",
      "2019-10-30 20:40:03,155 Example #0\n",
      "2019-10-30 20:40:03,156 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:40:03,156 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:40:03,156 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:40:03,156 Example #1\n",
      "2019-10-30 20:40:03,156 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:40:03,156 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:40:03,156 \tHypothesis: Biblia haidumii sifa ya kukubali sababu .\n",
      "2019-10-30 20:40:03,156 Example #2\n",
      "2019-10-30 20:40:03,156 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:40:03,156 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:40:03,156 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 20:40:03,156 Example #3\n",
      "2019-10-30 20:40:03,156 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:40:03,156 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:40:03,156 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 20:40:03,156 Validation result at epoch  19, step   176000: bleu:  36.82, loss: 29995.8574, ppl:   2.9784, duration: 44.3117s\n",
      "2019-10-30 20:40:18,056 Epoch  19 Step:   176100 Batch Loss:     1.152991 Tokens per Sec:    17070, Lr: 0.000210\n",
      "2019-10-30 20:40:32,953 Epoch  19 Step:   176200 Batch Loss:     1.128376 Tokens per Sec:    16980, Lr: 0.000210\n",
      "2019-10-30 20:40:48,265 Epoch  19 Step:   176300 Batch Loss:     1.305430 Tokens per Sec:    16681, Lr: 0.000210\n",
      "2019-10-30 20:41:03,388 Epoch  19 Step:   176400 Batch Loss:     1.228206 Tokens per Sec:    16874, Lr: 0.000210\n",
      "2019-10-30 20:41:18,447 Epoch  19 Step:   176500 Batch Loss:     1.055689 Tokens per Sec:    17054, Lr: 0.000210\n",
      "2019-10-30 20:41:33,661 Epoch  19 Step:   176600 Batch Loss:     1.093556 Tokens per Sec:    16930, Lr: 0.000210\n",
      "2019-10-30 20:41:48,899 Epoch  19 Step:   176700 Batch Loss:     1.086127 Tokens per Sec:    16502, Lr: 0.000210\n",
      "2019-10-30 20:42:03,974 Epoch  19 Step:   176800 Batch Loss:     1.156211 Tokens per Sec:    17060, Lr: 0.000210\n",
      "2019-10-30 20:42:19,013 Epoch  19 Step:   176900 Batch Loss:     1.311759 Tokens per Sec:    17068, Lr: 0.000210\n",
      "2019-10-30 20:42:34,198 Epoch  19 Step:   177000 Batch Loss:     1.164605 Tokens per Sec:    16826, Lr: 0.000210\n",
      "2019-10-30 20:43:18,456 Example #0\n",
      "2019-10-30 20:43:18,456 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:43:18,456 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:43:18,456 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:43:18,456 Example #1\n",
      "2019-10-30 20:43:18,456 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:43:18,456 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:43:18,456 \tHypothesis: Biblia haidumu sifa .\n",
      "2019-10-30 20:43:18,456 Example #2\n",
      "2019-10-30 20:43:18,456 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:43:18,456 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:43:18,456 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 20:43:18,456 Example #3\n",
      "2019-10-30 20:43:18,457 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:43:18,457 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:43:18,457 \tHypothesis: Mabusu ya makumbatio na mabusu .\n",
      "2019-10-30 20:43:18,457 Validation result at epoch  19, step   177000: bleu:  36.77, loss: 30005.1699, ppl:   2.9794, duration: 44.2584s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 20:43:33,422 Epoch  19 Step:   177100 Batch Loss:     1.150686 Tokens per Sec:    17095, Lr: 0.000210\n",
      "2019-10-30 20:43:48,615 Epoch  19 Step:   177200 Batch Loss:     1.091440 Tokens per Sec:    16725, Lr: 0.000210\n",
      "2019-10-30 20:44:03,747 Epoch  19 Step:   177300 Batch Loss:     1.082447 Tokens per Sec:    17219, Lr: 0.000210\n",
      "2019-10-30 20:44:19,086 Epoch  19 Step:   177400 Batch Loss:     1.294839 Tokens per Sec:    17012, Lr: 0.000210\n",
      "2019-10-30 20:44:34,201 Epoch  19 Step:   177500 Batch Loss:     1.415480 Tokens per Sec:    17007, Lr: 0.000210\n",
      "2019-10-30 20:44:49,027 Epoch  19 Step:   177600 Batch Loss:     1.183569 Tokens per Sec:    17002, Lr: 0.000210\n",
      "2019-10-30 20:45:04,103 Epoch  19 Step:   177700 Batch Loss:     1.277583 Tokens per Sec:    17464, Lr: 0.000210\n",
      "2019-10-30 20:45:19,565 Epoch  19 Step:   177800 Batch Loss:     1.403165 Tokens per Sec:    17038, Lr: 0.000210\n",
      "2019-10-30 20:45:34,823 Epoch  19 Step:   177900 Batch Loss:     1.152424 Tokens per Sec:    16798, Lr: 0.000210\n",
      "2019-10-30 20:45:50,158 Epoch  19 Step:   178000 Batch Loss:     1.205569 Tokens per Sec:    17169, Lr: 0.000210\n",
      "2019-10-30 20:46:34,614 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:46:34,615 Saving new checkpoint.\n",
      "2019-10-30 20:46:34,762 Example #0\n",
      "2019-10-30 20:46:34,762 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:46:34,762 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:46:34,762 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:46:34,762 Example #1\n",
      "2019-10-30 20:46:34,762 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:46:34,762 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:46:34,762 \tHypothesis: Biblia haidumu sifa ya kukubali sababu .\n",
      "2019-10-30 20:46:34,763 Example #2\n",
      "2019-10-30 20:46:34,763 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:46:34,763 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:46:34,763 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokea ?\n",
      "2019-10-30 20:46:34,763 Example #3\n",
      "2019-10-30 20:46:34,763 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:46:34,763 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:46:34,763 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 20:46:34,763 Validation result at epoch  19, step   178000: bleu:  37.06, loss: 29901.9160, ppl:   2.9683, duration: 44.6040s\n",
      "2019-10-30 20:46:49,533 Epoch  19 Step:   178100 Batch Loss:     1.274754 Tokens per Sec:    16874, Lr: 0.000210\n",
      "2019-10-30 20:47:04,513 Epoch  19 Step:   178200 Batch Loss:     1.349603 Tokens per Sec:    17478, Lr: 0.000210\n",
      "2019-10-30 20:47:17,420 Epoch  19: total training loss 11505.08\n",
      "2019-10-30 20:47:17,420 EPOCH 20\n",
      "2019-10-30 20:47:20,846 Epoch  20 Step:   178300 Batch Loss:     1.273184 Tokens per Sec:    11503, Lr: 0.000210\n",
      "2019-10-30 20:47:35,791 Epoch  20 Step:   178400 Batch Loss:     1.133331 Tokens per Sec:    17199, Lr: 0.000210\n",
      "2019-10-30 20:47:50,959 Epoch  20 Step:   178500 Batch Loss:     1.271588 Tokens per Sec:    17216, Lr: 0.000210\n",
      "2019-10-30 20:48:05,830 Epoch  20 Step:   178600 Batch Loss:     1.184117 Tokens per Sec:    16677, Lr: 0.000210\n",
      "2019-10-30 20:48:21,043 Epoch  20 Step:   178700 Batch Loss:     1.296374 Tokens per Sec:    16791, Lr: 0.000210\n",
      "2019-10-30 20:48:36,251 Epoch  20 Step:   178800 Batch Loss:     1.432207 Tokens per Sec:    17291, Lr: 0.000210\n",
      "2019-10-30 20:48:51,295 Epoch  20 Step:   178900 Batch Loss:     1.203746 Tokens per Sec:    16917, Lr: 0.000210\n",
      "2019-10-30 20:49:06,388 Epoch  20 Step:   179000 Batch Loss:     1.148069 Tokens per Sec:    16861, Lr: 0.000210\n",
      "2019-10-30 20:49:50,970 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:49:50,970 Saving new checkpoint.\n",
      "2019-10-30 20:49:51,166 Example #0\n",
      "2019-10-30 20:49:51,167 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:49:51,167 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:49:51,167 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:49:51,167 Example #1\n",
      "2019-10-30 20:49:51,167 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:49:51,167 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:49:51,167 \tHypothesis: Biblia haidhihirishi sifa ya ustahili .\n",
      "2019-10-30 20:49:51,167 Example #2\n",
      "2019-10-30 20:49:51,167 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:49:51,167 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:49:51,167 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:49:51,167 Example #3\n",
      "2019-10-30 20:49:51,167 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:49:51,167 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:49:51,167 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 20:49:51,168 Validation result at epoch  20, step   179000: bleu:  37.05, loss: 29897.1621, ppl:   2.9677, duration: 44.7788s\n",
      "2019-10-30 20:50:06,393 Epoch  20 Step:   179100 Batch Loss:     1.205973 Tokens per Sec:    16945, Lr: 0.000210\n",
      "2019-10-30 20:50:21,619 Epoch  20 Step:   179200 Batch Loss:     1.236022 Tokens per Sec:    17170, Lr: 0.000210\n",
      "2019-10-30 20:50:36,604 Epoch  20 Step:   179300 Batch Loss:     1.306327 Tokens per Sec:    16924, Lr: 0.000210\n",
      "2019-10-30 20:50:51,565 Epoch  20 Step:   179400 Batch Loss:     1.533250 Tokens per Sec:    16961, Lr: 0.000210\n",
      "2019-10-30 20:51:06,917 Epoch  20 Step:   179500 Batch Loss:     1.315156 Tokens per Sec:    16899, Lr: 0.000210\n",
      "2019-10-30 20:51:21,856 Epoch  20 Step:   179600 Batch Loss:     1.157646 Tokens per Sec:    16872, Lr: 0.000210\n",
      "2019-10-30 20:51:36,815 Epoch  20 Step:   179700 Batch Loss:     1.270754 Tokens per Sec:    17147, Lr: 0.000210\n",
      "2019-10-30 20:51:51,931 Epoch  20 Step:   179800 Batch Loss:     1.337108 Tokens per Sec:    16871, Lr: 0.000210\n",
      "2019-10-30 20:52:07,267 Epoch  20 Step:   179900 Batch Loss:     1.238299 Tokens per Sec:    16979, Lr: 0.000210\n",
      "2019-10-30 20:52:22,460 Epoch  20 Step:   180000 Batch Loss:     1.220954 Tokens per Sec:    17004, Lr: 0.000210\n",
      "2019-10-30 20:53:06,466 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 20:53:06,466 Saving new checkpoint.\n",
      "2019-10-30 20:53:06,608 Example #0\n",
      "2019-10-30 20:53:06,608 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:53:06,608 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:53:06,608 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:53:06,609 Example #1\n",
      "2019-10-30 20:53:06,609 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:53:06,609 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:53:06,609 \tHypothesis: Biblia haidumii kuwa na sifa ya kudai ukweli .\n",
      "2019-10-30 20:53:06,609 Example #2\n",
      "2019-10-30 20:53:06,609 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:53:06,609 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:53:06,609 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 20:53:06,609 Example #3\n",
      "2019-10-30 20:53:06,609 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:53:06,609 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:53:06,609 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 20:53:06,609 Validation result at epoch  20, step   180000: bleu:  37.04, loss: 29835.0176, ppl:   2.9610, duration: 44.1488s\n",
      "2019-10-30 20:53:21,541 Epoch  20 Step:   180100 Batch Loss:     1.201617 Tokens per Sec:    17278, Lr: 0.000210\n",
      "2019-10-30 20:53:36,529 Epoch  20 Step:   180200 Batch Loss:     1.388799 Tokens per Sec:    17073, Lr: 0.000210\n",
      "2019-10-30 20:53:51,459 Epoch  20 Step:   180300 Batch Loss:     1.238800 Tokens per Sec:    16869, Lr: 0.000210\n",
      "2019-10-30 20:54:06,562 Epoch  20 Step:   180400 Batch Loss:     1.046736 Tokens per Sec:    17061, Lr: 0.000210\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 20:54:21,447 Epoch  20 Step:   180500 Batch Loss:     1.256428 Tokens per Sec:    17019, Lr: 0.000210\n",
      "2019-10-30 20:54:36,583 Epoch  20 Step:   180600 Batch Loss:     1.237368 Tokens per Sec:    17056, Lr: 0.000210\n",
      "2019-10-30 20:54:51,652 Epoch  20 Step:   180700 Batch Loss:     1.190692 Tokens per Sec:    17036, Lr: 0.000210\n",
      "2019-10-30 20:55:06,838 Epoch  20 Step:   180800 Batch Loss:     1.146542 Tokens per Sec:    17136, Lr: 0.000210\n",
      "2019-10-30 20:55:21,864 Epoch  20 Step:   180900 Batch Loss:     1.136540 Tokens per Sec:    17254, Lr: 0.000210\n",
      "2019-10-30 20:55:36,832 Epoch  20 Step:   181000 Batch Loss:     0.948396 Tokens per Sec:    16937, Lr: 0.000210\n",
      "2019-10-30 20:56:21,336 Example #0\n",
      "2019-10-30 20:56:21,337 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:56:21,337 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:56:21,337 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 20:56:21,337 Example #1\n",
      "2019-10-30 20:56:21,337 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:56:21,337 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:56:21,337 \tHypothesis: Biblia haidumii kuwa na sifa .\n",
      "2019-10-30 20:56:21,337 Example #2\n",
      "2019-10-30 20:56:21,337 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:56:21,337 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:56:21,337 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 20:56:21,338 Example #3\n",
      "2019-10-30 20:56:21,338 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:56:21,338 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:56:21,338 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-30 20:56:21,338 Validation result at epoch  20, step   181000: bleu:  36.89, loss: 29920.9512, ppl:   2.9703, duration: 44.5053s\n",
      "2019-10-30 20:56:36,509 Epoch  20 Step:   181100 Batch Loss:     0.947537 Tokens per Sec:    17157, Lr: 0.000210\n",
      "2019-10-30 20:56:51,317 Epoch  20 Step:   181200 Batch Loss:     1.052046 Tokens per Sec:    17319, Lr: 0.000210\n",
      "2019-10-30 20:57:06,396 Epoch  20 Step:   181300 Batch Loss:     1.342552 Tokens per Sec:    16863, Lr: 0.000210\n",
      "2019-10-30 20:57:21,481 Epoch  20 Step:   181400 Batch Loss:     1.333331 Tokens per Sec:    17323, Lr: 0.000210\n",
      "2019-10-30 20:57:36,652 Epoch  20 Step:   181500 Batch Loss:     1.239136 Tokens per Sec:    17167, Lr: 0.000210\n",
      "2019-10-30 20:57:51,878 Epoch  20 Step:   181600 Batch Loss:     1.161860 Tokens per Sec:    16812, Lr: 0.000210\n",
      "2019-10-30 20:58:07,020 Epoch  20 Step:   181700 Batch Loss:     1.406612 Tokens per Sec:    17192, Lr: 0.000210\n",
      "2019-10-30 20:58:22,207 Epoch  20 Step:   181800 Batch Loss:     1.106829 Tokens per Sec:    17038, Lr: 0.000210\n",
      "2019-10-30 20:58:37,329 Epoch  20 Step:   181900 Batch Loss:     1.226819 Tokens per Sec:    16980, Lr: 0.000210\n",
      "2019-10-30 20:58:52,623 Epoch  20 Step:   182000 Batch Loss:     1.272130 Tokens per Sec:    16908, Lr: 0.000210\n",
      "2019-10-30 20:59:36,754 Example #0\n",
      "2019-10-30 20:59:36,754 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 20:59:36,755 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:59:36,755 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 20:59:36,755 Example #1\n",
      "2019-10-30 20:59:36,755 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 20:59:36,755 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 20:59:36,755 \tHypothesis: Biblia haidumii sifa ya kudai ukweli .\n",
      "2019-10-30 20:59:36,755 Example #2\n",
      "2019-10-30 20:59:36,755 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 20:59:36,755 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 20:59:36,755 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 20:59:36,755 Example #3\n",
      "2019-10-30 20:59:36,755 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 20:59:36,755 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 20:59:36,755 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 20:59:36,755 Validation result at epoch  20, step   182000: bleu:  36.79, loss: 29908.9746, ppl:   2.9690, duration: 44.1315s\n",
      "2019-10-30 20:59:51,656 Epoch  20 Step:   182100 Batch Loss:     1.180255 Tokens per Sec:    17014, Lr: 0.000210\n",
      "2019-10-30 21:00:06,957 Epoch  20 Step:   182200 Batch Loss:     1.088711 Tokens per Sec:    17098, Lr: 0.000210\n",
      "2019-10-30 21:00:22,233 Epoch  20 Step:   182300 Batch Loss:     1.275636 Tokens per Sec:    16989, Lr: 0.000210\n",
      "2019-10-30 21:00:37,494 Epoch  20 Step:   182400 Batch Loss:     1.225051 Tokens per Sec:    17118, Lr: 0.000210\n",
      "2019-10-30 21:00:52,630 Epoch  20 Step:   182500 Batch Loss:     1.197972 Tokens per Sec:    16873, Lr: 0.000210\n",
      "2019-10-30 21:01:08,087 Epoch  20 Step:   182600 Batch Loss:     1.195938 Tokens per Sec:    17000, Lr: 0.000210\n",
      "2019-10-30 21:01:23,324 Epoch  20 Step:   182700 Batch Loss:     1.178848 Tokens per Sec:    16768, Lr: 0.000210\n",
      "2019-10-30 21:01:38,476 Epoch  20 Step:   182800 Batch Loss:     1.210547 Tokens per Sec:    16928, Lr: 0.000210\n",
      "2019-10-30 21:01:53,574 Epoch  20 Step:   182900 Batch Loss:     1.296697 Tokens per Sec:    17176, Lr: 0.000210\n",
      "2019-10-30 21:02:08,667 Epoch  20 Step:   183000 Batch Loss:     1.246986 Tokens per Sec:    16900, Lr: 0.000210\n",
      "2019-10-30 21:02:53,136 Example #0\n",
      "2019-10-30 21:02:53,136 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:02:53,136 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:02:53,137 \tHypothesis: Alipata mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:02:53,137 Example #1\n",
      "2019-10-30 21:02:53,137 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:02:53,137 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:02:53,137 \tHypothesis: Biblia haisemi kwamba mtu ana sifa nzuri .\n",
      "2019-10-30 21:02:53,137 Example #2\n",
      "2019-10-30 21:02:53,137 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:02:53,137 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:02:53,137 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 21:02:53,137 Example #3\n",
      "2019-10-30 21:02:53,137 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:02:53,137 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:02:53,137 \tHypothesis: Mabusu ya makumbatio na mabusu .\n",
      "2019-10-30 21:02:53,137 Validation result at epoch  20, step   183000: bleu:  36.94, loss: 29935.2676, ppl:   2.9719, duration: 44.4697s\n",
      "2019-10-30 21:03:08,123 Epoch  20 Step:   183100 Batch Loss:     1.098717 Tokens per Sec:    16774, Lr: 0.000210\n",
      "2019-10-30 21:03:23,038 Epoch  20 Step:   183200 Batch Loss:     1.161568 Tokens per Sec:    17058, Lr: 0.000210\n",
      "2019-10-30 21:03:38,182 Epoch  20 Step:   183300 Batch Loss:     1.319703 Tokens per Sec:    17264, Lr: 0.000210\n",
      "2019-10-30 21:03:53,056 Epoch  20 Step:   183400 Batch Loss:     1.231599 Tokens per Sec:    17241, Lr: 0.000210\n",
      "2019-10-30 21:04:08,071 Epoch  20 Step:   183500 Batch Loss:     1.152490 Tokens per Sec:    17050, Lr: 0.000210\n",
      "2019-10-30 21:04:22,835 Epoch  20 Step:   183600 Batch Loss:     1.132916 Tokens per Sec:    16604, Lr: 0.000210\n",
      "2019-10-30 21:04:38,012 Epoch  20 Step:   183700 Batch Loss:     1.149288 Tokens per Sec:    17569, Lr: 0.000210\n",
      "2019-10-30 21:04:53,070 Epoch  20 Step:   183800 Batch Loss:     1.273781 Tokens per Sec:    16735, Lr: 0.000210\n",
      "2019-10-30 21:05:08,133 Epoch  20 Step:   183900 Batch Loss:     1.197352 Tokens per Sec:    17171, Lr: 0.000210\n",
      "2019-10-30 21:05:23,221 Epoch  20 Step:   184000 Batch Loss:     1.172288 Tokens per Sec:    17147, Lr: 0.000210\n",
      "2019-10-30 21:06:08,133 Example #0\n",
      "2019-10-30 21:06:08,134 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:06:08,134 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:06:08,134 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:06:08,134 Example #1\n",
      "2019-10-30 21:06:08,134 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:06:08,134 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:06:08,134 \tHypothesis: Biblia haisemi kwamba kuna sifa .\n",
      "2019-10-30 21:06:08,134 Example #2\n",
      "2019-10-30 21:06:08,134 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:06:08,134 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:06:08,134 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 21:06:08,135 Example #3\n",
      "2019-10-30 21:06:08,135 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:06:08,135 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:06:08,135 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 21:06:08,135 Validation result at epoch  20, step   184000: bleu:  36.62, loss: 29881.8145, ppl:   2.9661, duration: 44.9130s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 21:06:23,457 Epoch  20 Step:   184100 Batch Loss:     1.161898 Tokens per Sec:    17014, Lr: 0.000210\n",
      "2019-10-30 21:06:38,247 Epoch  20 Step:   184200 Batch Loss:     1.208127 Tokens per Sec:    17164, Lr: 0.000210\n",
      "2019-10-30 21:06:53,146 Epoch  20 Step:   184300 Batch Loss:     1.054315 Tokens per Sec:    17451, Lr: 0.000210\n",
      "2019-10-30 21:07:08,222 Epoch  20 Step:   184400 Batch Loss:     1.199059 Tokens per Sec:    17287, Lr: 0.000210\n",
      "2019-10-30 21:07:22,880 Epoch  20 Step:   184500 Batch Loss:     1.157886 Tokens per Sec:    16868, Lr: 0.000210\n",
      "2019-10-30 21:07:37,769 Epoch  20 Step:   184600 Batch Loss:     1.001206 Tokens per Sec:    17222, Lr: 0.000210\n",
      "2019-10-30 21:07:53,025 Epoch  20 Step:   184700 Batch Loss:     1.232881 Tokens per Sec:    16997, Lr: 0.000210\n",
      "2019-10-30 21:08:08,081 Epoch  20 Step:   184800 Batch Loss:     1.371944 Tokens per Sec:    16727, Lr: 0.000210\n",
      "2019-10-30 21:08:22,953 Epoch  20 Step:   184900 Batch Loss:     1.313066 Tokens per Sec:    17056, Lr: 0.000210\n",
      "2019-10-30 21:08:38,094 Epoch  20 Step:   185000 Batch Loss:     1.328404 Tokens per Sec:    17078, Lr: 0.000210\n",
      "2019-10-30 21:09:22,091 Example #0\n",
      "2019-10-30 21:09:22,091 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:09:22,091 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:09:22,091 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:09:22,091 Example #1\n",
      "2019-10-30 21:09:22,091 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:09:22,091 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:09:22,091 \tHypothesis: Biblia haidumii sifa .\n",
      "2019-10-30 21:09:22,091 Example #2\n",
      "2019-10-30 21:09:22,091 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:09:22,092 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:09:22,092 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 21:09:22,092 Example #3\n",
      "2019-10-30 21:09:22,092 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:09:22,092 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:09:22,092 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 21:09:22,092 Validation result at epoch  20, step   185000: bleu:  36.88, loss: 29937.5762, ppl:   2.9721, duration: 43.9978s\n",
      "2019-10-30 21:09:37,234 Epoch  20 Step:   185100 Batch Loss:     1.288484 Tokens per Sec:    17226, Lr: 0.000210\n",
      "2019-10-30 21:09:52,174 Epoch  20 Step:   185200 Batch Loss:     1.082991 Tokens per Sec:    17335, Lr: 0.000210\n",
      "2019-10-30 21:10:07,315 Epoch  20 Step:   185300 Batch Loss:     1.059006 Tokens per Sec:    17014, Lr: 0.000210\n",
      "2019-10-30 21:10:22,703 Epoch  20 Step:   185400 Batch Loss:     1.065461 Tokens per Sec:    17150, Lr: 0.000210\n",
      "2019-10-30 21:10:37,676 Epoch  20 Step:   185500 Batch Loss:     1.135629 Tokens per Sec:    16993, Lr: 0.000210\n",
      "2019-10-30 21:10:52,694 Epoch  20 Step:   185600 Batch Loss:     1.149547 Tokens per Sec:    16962, Lr: 0.000210\n",
      "2019-10-30 21:11:07,626 Epoch  20 Step:   185700 Batch Loss:     1.226756 Tokens per Sec:    16836, Lr: 0.000210\n",
      "2019-10-30 21:11:22,912 Epoch  20 Step:   185800 Batch Loss:     1.327106 Tokens per Sec:    17123, Lr: 0.000210\n",
      "2019-10-30 21:11:38,234 Epoch  20 Step:   185900 Batch Loss:     1.134332 Tokens per Sec:    16892, Lr: 0.000210\n",
      "2019-10-30 21:11:53,569 Epoch  20 Step:   186000 Batch Loss:     1.207627 Tokens per Sec:    16953, Lr: 0.000210\n",
      "2019-10-30 21:12:37,710 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:12:37,710 Saving new checkpoint.\n",
      "2019-10-30 21:12:37,855 Example #0\n",
      "2019-10-30 21:12:37,856 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:12:37,856 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:12:37,856 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:12:37,856 Example #1\n",
      "2019-10-30 21:12:37,856 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:12:37,856 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:12:37,856 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sifa .\n",
      "2019-10-30 21:12:37,856 Example #2\n",
      "2019-10-30 21:12:37,856 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:12:37,856 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:12:37,856 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukufanyaje ?\n",
      "2019-10-30 21:12:37,856 Example #3\n",
      "2019-10-30 21:12:37,856 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:12:37,857 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:12:37,857 \tHypothesis: Mikono ya makumbatio na mabusu .\n",
      "2019-10-30 21:12:37,857 Validation result at epoch  20, step   186000: bleu:  36.53, loss: 29682.0371, ppl:   2.9446, duration: 44.2875s\n",
      "2019-10-30 21:12:52,680 Epoch  20 Step:   186100 Batch Loss:     1.112084 Tokens per Sec:    16898, Lr: 0.000210\n",
      "2019-10-30 21:13:07,718 Epoch  20 Step:   186200 Batch Loss:     1.369121 Tokens per Sec:    17021, Lr: 0.000210\n",
      "2019-10-30 21:13:22,702 Epoch  20 Step:   186300 Batch Loss:     1.191655 Tokens per Sec:    16727, Lr: 0.000210\n",
      "2019-10-30 21:13:37,662 Epoch  20 Step:   186400 Batch Loss:     1.409016 Tokens per Sec:    16944, Lr: 0.000210\n",
      "2019-10-30 21:13:52,789 Epoch  20 Step:   186500 Batch Loss:     1.281305 Tokens per Sec:    16994, Lr: 0.000210\n",
      "2019-10-30 21:14:07,831 Epoch  20 Step:   186600 Batch Loss:     1.185324 Tokens per Sec:    17251, Lr: 0.000210\n",
      "2019-10-30 21:14:22,794 Epoch  20 Step:   186700 Batch Loss:     1.224825 Tokens per Sec:    17134, Lr: 0.000210\n",
      "2019-10-30 21:14:37,711 Epoch  20 Step:   186800 Batch Loss:     1.359918 Tokens per Sec:    17134, Lr: 0.000210\n",
      "2019-10-30 21:14:52,670 Epoch  20 Step:   186900 Batch Loss:     1.157308 Tokens per Sec:    16957, Lr: 0.000210\n",
      "2019-10-30 21:15:07,669 Epoch  20 Step:   187000 Batch Loss:     1.194524 Tokens per Sec:    17012, Lr: 0.000210\n",
      "2019-10-30 21:15:51,765 Example #0\n",
      "2019-10-30 21:15:51,766 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:15:51,766 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:15:51,766 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:15:51,766 Example #1\n",
      "2019-10-30 21:15:51,766 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:15:51,766 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:15:51,766 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 21:15:51,766 Example #2\n",
      "2019-10-30 21:15:51,766 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:15:51,766 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:15:51,766 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 21:15:51,766 Example #3\n",
      "2019-10-30 21:15:51,766 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:15:51,766 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:15:51,766 \tHypothesis: Vitu vya kukumbatia na kubusu .\n",
      "2019-10-30 21:15:51,767 Validation result at epoch  20, step   187000: bleu:  36.93, loss: 29816.9453, ppl:   2.9591, duration: 44.0973s\n",
      "2019-10-30 21:16:06,849 Epoch  20 Step:   187100 Batch Loss:     1.149229 Tokens per Sec:    17064, Lr: 0.000210\n",
      "2019-10-30 21:16:21,938 Epoch  20 Step:   187200 Batch Loss:     1.305015 Tokens per Sec:    17271, Lr: 0.000210\n",
      "2019-10-30 21:16:36,692 Epoch  20 Step:   187300 Batch Loss:     1.249844 Tokens per Sec:    17009, Lr: 0.000210\n",
      "2019-10-30 21:16:51,702 Epoch  20 Step:   187400 Batch Loss:     1.308746 Tokens per Sec:    16925, Lr: 0.000210\n",
      "2019-10-30 21:17:06,623 Epoch  20 Step:   187500 Batch Loss:     1.019689 Tokens per Sec:    17041, Lr: 0.000210\n",
      "2019-10-30 21:17:21,700 Epoch  20 Step:   187600 Batch Loss:     1.327942 Tokens per Sec:    17159, Lr: 0.000210\n",
      "2019-10-30 21:17:33,062 Epoch  20: total training loss 11450.30\n",
      "2019-10-30 21:17:33,062 EPOCH 21\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 21:17:37,753 Epoch  21 Step:   187700 Batch Loss:     1.214533 Tokens per Sec:    12520, Lr: 0.000210\n",
      "2019-10-30 21:17:52,755 Epoch  21 Step:   187800 Batch Loss:     1.261676 Tokens per Sec:    17215, Lr: 0.000210\n",
      "2019-10-30 21:18:07,740 Epoch  21 Step:   187900 Batch Loss:     1.179022 Tokens per Sec:    17035, Lr: 0.000210\n",
      "2019-10-30 21:18:22,759 Epoch  21 Step:   188000 Batch Loss:     1.143797 Tokens per Sec:    17228, Lr: 0.000210\n",
      "2019-10-30 21:19:07,861 Example #0\n",
      "2019-10-30 21:19:07,862 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:19:07,862 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:19:07,862 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:19:07,862 Example #1\n",
      "2019-10-30 21:19:07,862 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:19:07,862 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:19:07,862 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sifa .\n",
      "2019-10-30 21:19:07,862 Example #2\n",
      "2019-10-30 21:19:07,862 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:19:07,863 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:19:07,863 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 21:19:07,863 Example #3\n",
      "2019-10-30 21:19:07,863 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:19:07,863 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:19:07,863 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 21:19:07,863 Validation result at epoch  21, step   188000: bleu:  37.33, loss: 29805.3516, ppl:   2.9578, duration: 45.1034s\n",
      "2019-10-30 21:19:23,402 Epoch  21 Step:   188100 Batch Loss:     1.203063 Tokens per Sec:    16744, Lr: 0.000210\n",
      "2019-10-30 21:19:38,601 Epoch  21 Step:   188200 Batch Loss:     1.235713 Tokens per Sec:    17101, Lr: 0.000210\n",
      "2019-10-30 21:19:53,848 Epoch  21 Step:   188300 Batch Loss:     1.245322 Tokens per Sec:    17181, Lr: 0.000210\n",
      "2019-10-30 21:20:08,757 Epoch  21 Step:   188400 Batch Loss:     1.058830 Tokens per Sec:    17410, Lr: 0.000210\n",
      "2019-10-30 21:20:23,605 Epoch  21 Step:   188500 Batch Loss:     1.353571 Tokens per Sec:    17315, Lr: 0.000210\n",
      "2019-10-30 21:20:38,496 Epoch  21 Step:   188600 Batch Loss:     1.333243 Tokens per Sec:    16912, Lr: 0.000210\n",
      "2019-10-30 21:20:53,471 Epoch  21 Step:   188700 Batch Loss:     1.169215 Tokens per Sec:    17411, Lr: 0.000210\n",
      "2019-10-30 21:21:08,432 Epoch  21 Step:   188800 Batch Loss:     1.298739 Tokens per Sec:    16932, Lr: 0.000210\n",
      "2019-10-30 21:21:23,489 Epoch  21 Step:   188900 Batch Loss:     1.207062 Tokens per Sec:    17450, Lr: 0.000210\n",
      "2019-10-30 21:21:38,596 Epoch  21 Step:   189000 Batch Loss:     1.098269 Tokens per Sec:    16928, Lr: 0.000210\n",
      "2019-10-30 21:22:22,789 Example #0\n",
      "2019-10-30 21:22:22,789 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:22:22,790 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:22:22,790 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:22:22,790 Example #1\n",
      "2019-10-30 21:22:22,790 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:22:22,790 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:22:22,790 \tHypothesis: Biblia haidumu sifa .\n",
      "2019-10-30 21:22:22,790 Example #2\n",
      "2019-10-30 21:22:22,790 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:22:22,790 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:22:22,790 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 21:22:22,790 Example #3\n",
      "2019-10-30 21:22:22,790 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:22:22,790 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:22:22,790 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 21:22:22,790 Validation result at epoch  21, step   189000: bleu:  36.83, loss: 29883.9805, ppl:   2.9663, duration: 44.1938s\n",
      "2019-10-30 21:22:37,915 Epoch  21 Step:   189100 Batch Loss:     1.034272 Tokens per Sec:    17081, Lr: 0.000210\n",
      "2019-10-30 21:22:52,735 Epoch  21 Step:   189200 Batch Loss:     1.156201 Tokens per Sec:    16780, Lr: 0.000210\n",
      "2019-10-30 21:23:07,957 Epoch  21 Step:   189300 Batch Loss:     1.237862 Tokens per Sec:    17006, Lr: 0.000210\n",
      "2019-10-30 21:23:23,242 Epoch  21 Step:   189400 Batch Loss:     1.134435 Tokens per Sec:    17002, Lr: 0.000210\n",
      "2019-10-30 21:23:38,315 Epoch  21 Step:   189500 Batch Loss:     1.360839 Tokens per Sec:    16853, Lr: 0.000210\n",
      "2019-10-30 21:23:53,265 Epoch  21 Step:   189600 Batch Loss:     1.153959 Tokens per Sec:    16989, Lr: 0.000210\n",
      "2019-10-30 21:24:08,457 Epoch  21 Step:   189700 Batch Loss:     1.114173 Tokens per Sec:    16856, Lr: 0.000210\n",
      "2019-10-30 21:24:23,484 Epoch  21 Step:   189800 Batch Loss:     1.108255 Tokens per Sec:    17143, Lr: 0.000210\n",
      "2019-10-30 21:24:38,538 Epoch  21 Step:   189900 Batch Loss:     1.144553 Tokens per Sec:    17021, Lr: 0.000210\n",
      "2019-10-30 21:24:53,621 Epoch  21 Step:   190000 Batch Loss:     1.185366 Tokens per Sec:    17084, Lr: 0.000210\n",
      "2019-10-30 21:25:37,767 Example #0\n",
      "2019-10-30 21:25:37,767 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:25:37,767 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:25:37,767 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:25:37,767 Example #1\n",
      "2019-10-30 21:25:37,767 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:25:37,767 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:25:37,767 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sifa .\n",
      "2019-10-30 21:25:37,767 Example #2\n",
      "2019-10-30 21:25:37,767 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:25:37,768 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:25:37,768 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 21:25:37,768 Example #3\n",
      "2019-10-30 21:25:37,768 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:25:37,768 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:25:37,768 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 21:25:37,768 Validation result at epoch  21, step   190000: bleu:  37.01, loss: 29882.8008, ppl:   2.9662, duration: 44.1462s\n",
      "2019-10-30 21:25:52,894 Epoch  21 Step:   190100 Batch Loss:     1.409625 Tokens per Sec:    17115, Lr: 0.000210\n",
      "2019-10-30 21:26:07,857 Epoch  21 Step:   190200 Batch Loss:     1.220080 Tokens per Sec:    17164, Lr: 0.000210\n",
      "2019-10-30 21:26:22,873 Epoch  21 Step:   190300 Batch Loss:     1.133982 Tokens per Sec:    17188, Lr: 0.000210\n",
      "2019-10-30 21:26:37,835 Epoch  21 Step:   190400 Batch Loss:     1.339440 Tokens per Sec:    17115, Lr: 0.000210\n",
      "2019-10-30 21:26:52,851 Epoch  21 Step:   190500 Batch Loss:     1.268748 Tokens per Sec:    17113, Lr: 0.000210\n",
      "2019-10-30 21:27:08,236 Epoch  21 Step:   190600 Batch Loss:     1.257584 Tokens per Sec:    17127, Lr: 0.000210\n",
      "2019-10-30 21:27:23,280 Epoch  21 Step:   190700 Batch Loss:     1.099686 Tokens per Sec:    16708, Lr: 0.000210\n",
      "2019-10-30 21:27:38,389 Epoch  21 Step:   190800 Batch Loss:     1.095316 Tokens per Sec:    17029, Lr: 0.000210\n",
      "2019-10-30 21:27:53,520 Epoch  21 Step:   190900 Batch Loss:     1.294973 Tokens per Sec:    17128, Lr: 0.000210\n",
      "2019-10-30 21:28:08,320 Epoch  21 Step:   191000 Batch Loss:     1.228262 Tokens per Sec:    16711, Lr: 0.000210\n",
      "2019-10-30 21:28:52,532 Example #0\n",
      "2019-10-30 21:28:52,532 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:28:52,532 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:28:52,532 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:28:52,532 Example #1\n",
      "2019-10-30 21:28:52,532 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:28:52,532 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:28:52,532 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya ustahili .\n",
      "2019-10-30 21:28:52,532 Example #2\n",
      "2019-10-30 21:28:52,532 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:28:52,532 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:28:52,532 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 21:28:52,533 Example #3\n",
      "2019-10-30 21:28:52,533 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:28:52,533 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:28:52,533 \tHypothesis: Makumbatio na mabusu .\n",
      "2019-10-30 21:28:52,533 Validation result at epoch  21, step   191000: bleu:  36.50, loss: 29804.8496, ppl:   2.9578, duration: 44.2125s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 21:29:07,820 Epoch  21 Step:   191100 Batch Loss:     1.226411 Tokens per Sec:    17383, Lr: 0.000210\n",
      "2019-10-30 21:29:22,909 Epoch  21 Step:   191200 Batch Loss:     1.208407 Tokens per Sec:    17302, Lr: 0.000210\n",
      "2019-10-30 21:29:38,260 Epoch  21 Step:   191300 Batch Loss:     1.081885 Tokens per Sec:    17231, Lr: 0.000210\n",
      "2019-10-30 21:29:53,361 Epoch  21 Step:   191400 Batch Loss:     1.297270 Tokens per Sec:    16848, Lr: 0.000210\n",
      "2019-10-30 21:30:08,369 Epoch  21 Step:   191500 Batch Loss:     1.125385 Tokens per Sec:    16930, Lr: 0.000210\n",
      "2019-10-30 21:30:23,684 Epoch  21 Step:   191600 Batch Loss:     1.226914 Tokens per Sec:    17078, Lr: 0.000210\n",
      "2019-10-30 21:30:38,663 Epoch  21 Step:   191700 Batch Loss:     1.224357 Tokens per Sec:    16906, Lr: 0.000210\n",
      "2019-10-30 21:30:53,655 Epoch  21 Step:   191800 Batch Loss:     1.240927 Tokens per Sec:    17012, Lr: 0.000210\n",
      "2019-10-30 21:31:08,845 Epoch  21 Step:   191900 Batch Loss:     1.220801 Tokens per Sec:    16649, Lr: 0.000210\n",
      "2019-10-30 21:31:23,957 Epoch  21 Step:   192000 Batch Loss:     1.102200 Tokens per Sec:    16657, Lr: 0.000210\n",
      "2019-10-30 21:32:08,132 Example #0\n",
      "2019-10-30 21:32:08,132 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:32:08,132 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:32:08,132 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:32:08,132 Example #1\n",
      "2019-10-30 21:32:08,132 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:32:08,132 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:32:08,132 \tHypothesis: Biblia haidhuru sifa ya uthabiti .\n",
      "2019-10-30 21:32:08,132 Example #2\n",
      "2019-10-30 21:32:08,132 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:32:08,132 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:32:08,132 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 21:32:08,133 Example #3\n",
      "2019-10-30 21:32:08,133 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:32:08,133 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:32:08,133 \tHypothesis: Vipande vya makumbatio na mabusu .\n",
      "2019-10-30 21:32:08,133 Validation result at epoch  21, step   192000: bleu:  36.91, loss: 29788.2207, ppl:   2.9560, duration: 44.1750s\n",
      "2019-10-30 21:32:23,398 Epoch  21 Step:   192100 Batch Loss:     1.362336 Tokens per Sec:    17333, Lr: 0.000147\n",
      "2019-10-30 21:32:38,549 Epoch  21 Step:   192200 Batch Loss:     1.015269 Tokens per Sec:    16994, Lr: 0.000147\n",
      "2019-10-30 21:32:53,718 Epoch  21 Step:   192300 Batch Loss:     1.201159 Tokens per Sec:    17050, Lr: 0.000147\n",
      "2019-10-30 21:33:08,836 Epoch  21 Step:   192400 Batch Loss:     1.382093 Tokens per Sec:    16823, Lr: 0.000147\n",
      "2019-10-30 21:33:23,884 Epoch  21 Step:   192500 Batch Loss:     1.200028 Tokens per Sec:    16948, Lr: 0.000147\n",
      "2019-10-30 21:33:38,725 Epoch  21 Step:   192600 Batch Loss:     1.208245 Tokens per Sec:    16782, Lr: 0.000147\n",
      "2019-10-30 21:33:53,759 Epoch  21 Step:   192700 Batch Loss:     1.136645 Tokens per Sec:    16966, Lr: 0.000147\n",
      "2019-10-30 21:34:08,846 Epoch  21 Step:   192800 Batch Loss:     1.269618 Tokens per Sec:    16913, Lr: 0.000147\n",
      "2019-10-30 21:34:24,086 Epoch  21 Step:   192900 Batch Loss:     1.224761 Tokens per Sec:    17140, Lr: 0.000147\n",
      "2019-10-30 21:34:39,089 Epoch  21 Step:   193000 Batch Loss:     1.094950 Tokens per Sec:    16971, Lr: 0.000147\n",
      "2019-10-30 21:35:23,344 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:35:23,344 Saving new checkpoint.\n",
      "2019-10-30 21:35:23,488 Example #0\n",
      "2019-10-30 21:35:23,488 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:35:23,488 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:35:23,488 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:35:23,489 Example #1\n",
      "2019-10-30 21:35:23,489 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:35:23,489 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:35:23,489 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 21:35:23,489 Example #2\n",
      "2019-10-30 21:35:23,489 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:35:23,489 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:35:23,489 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 21:35:23,489 Example #3\n",
      "2019-10-30 21:35:23,489 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:35:23,489 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:35:23,489 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 21:35:23,489 Validation result at epoch  21, step   193000: bleu:  37.22, loss: 29647.8320, ppl:   2.9409, duration: 44.4001s\n",
      "2019-10-30 21:35:38,626 Epoch  21 Step:   193100 Batch Loss:     1.286645 Tokens per Sec:    16934, Lr: 0.000147\n",
      "2019-10-30 21:35:53,425 Epoch  21 Step:   193200 Batch Loss:     1.259809 Tokens per Sec:    17226, Lr: 0.000147\n",
      "2019-10-30 21:36:08,450 Epoch  21 Step:   193300 Batch Loss:     1.171221 Tokens per Sec:    17071, Lr: 0.000147\n",
      "2019-10-30 21:36:23,502 Epoch  21 Step:   193400 Batch Loss:     1.267434 Tokens per Sec:    16932, Lr: 0.000147\n",
      "2019-10-30 21:36:38,387 Epoch  21 Step:   193500 Batch Loss:     1.014548 Tokens per Sec:    17038, Lr: 0.000147\n",
      "2019-10-30 21:36:53,293 Epoch  21 Step:   193600 Batch Loss:     1.180915 Tokens per Sec:    17399, Lr: 0.000147\n",
      "2019-10-30 21:37:08,450 Epoch  21 Step:   193700 Batch Loss:     1.260294 Tokens per Sec:    17116, Lr: 0.000147\n",
      "2019-10-30 21:37:23,539 Epoch  21 Step:   193800 Batch Loss:     1.113027 Tokens per Sec:    17063, Lr: 0.000147\n",
      "2019-10-30 21:37:38,662 Epoch  21 Step:   193900 Batch Loss:     1.214873 Tokens per Sec:    17317, Lr: 0.000147\n",
      "2019-10-30 21:37:53,784 Epoch  21 Step:   194000 Batch Loss:     1.147084 Tokens per Sec:    17323, Lr: 0.000147\n",
      "2019-10-30 21:38:37,787 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:38:37,787 Saving new checkpoint.\n",
      "2019-10-30 21:38:37,927 Example #0\n",
      "2019-10-30 21:38:37,928 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:38:37,928 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:38:37,928 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:38:37,928 Example #1\n",
      "2019-10-30 21:38:37,928 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:38:37,928 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:38:37,928 \tHypothesis: Biblia haidhoofishi sifa ya ustahili .\n",
      "2019-10-30 21:38:37,928 Example #2\n",
      "2019-10-30 21:38:37,928 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:38:37,928 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:38:37,928 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 21:38:37,928 Example #3\n",
      "2019-10-30 21:38:37,928 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:38:37,928 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:38:37,929 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 21:38:37,929 Validation result at epoch  21, step   194000: bleu:  36.91, loss: 29573.9414, ppl:   2.9330, duration: 44.1442s\n",
      "2019-10-30 21:38:52,788 Epoch  21 Step:   194100 Batch Loss:     1.299108 Tokens per Sec:    17193, Lr: 0.000147\n",
      "2019-10-30 21:39:07,889 Epoch  21 Step:   194200 Batch Loss:     1.277178 Tokens per Sec:    17221, Lr: 0.000147\n",
      "2019-10-30 21:39:23,077 Epoch  21 Step:   194300 Batch Loss:     1.138549 Tokens per Sec:    16929, Lr: 0.000147\n",
      "2019-10-30 21:39:38,044 Epoch  21 Step:   194400 Batch Loss:     0.975900 Tokens per Sec:    17340, Lr: 0.000147\n",
      "2019-10-30 21:39:53,148 Epoch  21 Step:   194500 Batch Loss:     1.193050 Tokens per Sec:    17275, Lr: 0.000147\n",
      "2019-10-30 21:40:08,252 Epoch  21 Step:   194600 Batch Loss:     1.103965 Tokens per Sec:    16899, Lr: 0.000147\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 21:40:23,283 Epoch  21 Step:   194700 Batch Loss:     1.245748 Tokens per Sec:    17059, Lr: 0.000147\n",
      "2019-10-30 21:40:38,389 Epoch  21 Step:   194800 Batch Loss:     1.257745 Tokens per Sec:    16789, Lr: 0.000147\n",
      "2019-10-30 21:40:53,359 Epoch  21 Step:   194900 Batch Loss:     1.163816 Tokens per Sec:    16987, Lr: 0.000147\n",
      "2019-10-30 21:41:08,556 Epoch  21 Step:   195000 Batch Loss:     1.125486 Tokens per Sec:    17143, Lr: 0.000147\n",
      "2019-10-30 21:41:52,948 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:41:52,948 Saving new checkpoint.\n",
      "2019-10-30 21:41:53,095 Example #0\n",
      "2019-10-30 21:41:53,095 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:41:53,095 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:41:53,095 \tHypothesis: Alipata mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:41:53,095 Example #1\n",
      "2019-10-30 21:41:53,095 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:41:53,095 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:41:53,095 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 21:41:53,095 Example #2\n",
      "2019-10-30 21:41:53,095 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:41:53,095 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:41:53,095 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 21:41:53,095 Example #3\n",
      "2019-10-30 21:41:53,096 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:41:53,096 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:41:53,096 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 21:41:53,096 Validation result at epoch  21, step   195000: bleu:  37.15, loss: 29530.6953, ppl:   2.9284, duration: 44.5396s\n",
      "2019-10-30 21:42:08,079 Epoch  21 Step:   195100 Batch Loss:     1.238387 Tokens per Sec:    16928, Lr: 0.000147\n",
      "2019-10-30 21:42:23,021 Epoch  21 Step:   195200 Batch Loss:     1.062662 Tokens per Sec:    17074, Lr: 0.000147\n",
      "2019-10-30 21:42:37,927 Epoch  21 Step:   195300 Batch Loss:     1.261589 Tokens per Sec:    16815, Lr: 0.000147\n",
      "2019-10-30 21:42:53,170 Epoch  21 Step:   195400 Batch Loss:     1.377000 Tokens per Sec:    17207, Lr: 0.000147\n",
      "2019-10-30 21:43:08,023 Epoch  21 Step:   195500 Batch Loss:     1.130000 Tokens per Sec:    17204, Lr: 0.000147\n",
      "2019-10-30 21:43:22,995 Epoch  21 Step:   195600 Batch Loss:     1.193030 Tokens per Sec:    17278, Lr: 0.000147\n",
      "2019-10-30 21:43:38,072 Epoch  21 Step:   195700 Batch Loss:     1.297513 Tokens per Sec:    17194, Lr: 0.000147\n",
      "2019-10-30 21:43:53,287 Epoch  21 Step:   195800 Batch Loss:     1.165596 Tokens per Sec:    16969, Lr: 0.000147\n",
      "2019-10-30 21:44:08,504 Epoch  21 Step:   195900 Batch Loss:     1.328814 Tokens per Sec:    17115, Lr: 0.000147\n",
      "2019-10-30 21:44:23,362 Epoch  21 Step:   196000 Batch Loss:     1.083058 Tokens per Sec:    16888, Lr: 0.000147\n",
      "2019-10-30 21:45:07,365 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:45:07,365 Saving new checkpoint.\n",
      "2019-10-30 21:45:07,507 Example #0\n",
      "2019-10-30 21:45:07,507 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:45:07,507 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:45:07,507 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:45:07,507 Example #1\n",
      "2019-10-30 21:45:07,507 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:45:07,507 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:45:07,507 \tHypothesis: Biblia haidumii sifa ya uthabiti .\n",
      "2019-10-30 21:45:07,507 Example #2\n",
      "2019-10-30 21:45:07,507 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:45:07,508 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:45:07,508 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-30 21:45:07,508 Example #3\n",
      "2019-10-30 21:45:07,508 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:45:07,508 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:45:07,508 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 21:45:07,508 Validation result at epoch  21, step   196000: bleu:  37.14, loss: 29436.1797, ppl:   2.9184, duration: 44.1455s\n",
      "2019-10-30 21:45:22,659 Epoch  21 Step:   196100 Batch Loss:     1.165199 Tokens per Sec:    17182, Lr: 0.000147\n",
      "2019-10-30 21:45:37,669 Epoch  21 Step:   196200 Batch Loss:     1.192823 Tokens per Sec:    17386, Lr: 0.000147\n",
      "2019-10-30 21:45:52,518 Epoch  21 Step:   196300 Batch Loss:     1.189078 Tokens per Sec:    17236, Lr: 0.000147\n",
      "2019-10-30 21:46:07,590 Epoch  21 Step:   196400 Batch Loss:     1.330821 Tokens per Sec:    16866, Lr: 0.000147\n",
      "2019-10-30 21:46:22,823 Epoch  21 Step:   196500 Batch Loss:     1.365327 Tokens per Sec:    16813, Lr: 0.000147\n",
      "2019-10-30 21:46:37,950 Epoch  21 Step:   196600 Batch Loss:     1.314700 Tokens per Sec:    17276, Lr: 0.000147\n",
      "2019-10-30 21:46:53,124 Epoch  21 Step:   196700 Batch Loss:     1.201135 Tokens per Sec:    16882, Lr: 0.000147\n",
      "2019-10-30 21:47:08,115 Epoch  21 Step:   196800 Batch Loss:     1.027463 Tokens per Sec:    17127, Lr: 0.000147\n",
      "2019-10-30 21:47:23,288 Epoch  21 Step:   196900 Batch Loss:     1.151063 Tokens per Sec:    16975, Lr: 0.000147\n",
      "2019-10-30 21:47:38,344 Epoch  21 Step:   197000 Batch Loss:     1.286903 Tokens per Sec:    16845, Lr: 0.000147\n",
      "2019-10-30 21:48:22,721 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:48:22,721 Saving new checkpoint.\n",
      "2019-10-30 21:48:22,867 Example #0\n",
      "2019-10-30 21:48:22,867 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:48:22,867 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:48:22,867 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:48:22,867 Example #1\n",
      "2019-10-30 21:48:22,868 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:48:22,868 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:48:22,868 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 21:48:22,868 Example #2\n",
      "2019-10-30 21:48:22,868 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:48:22,868 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:48:22,868 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 21:48:22,868 Example #3\n",
      "2019-10-30 21:48:22,868 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:48:22,868 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:48:22,868 \tHypothesis: Nyasi za makumbatio na mabusu .\n",
      "2019-10-30 21:48:22,868 Validation result at epoch  21, step   197000: bleu:  37.10, loss: 29434.2383, ppl:   2.9182, duration: 44.5239s\n",
      "2019-10-30 21:48:31,669 Epoch  21: total training loss 11337.92\n",
      "2019-10-30 21:48:31,670 EPOCH 22\n",
      "2019-10-30 21:48:39,254 Epoch  22 Step:   197100 Batch Loss:     1.233984 Tokens per Sec:    14229, Lr: 0.000147\n",
      "2019-10-30 21:48:54,457 Epoch  22 Step:   197200 Batch Loss:     1.362437 Tokens per Sec:    17251, Lr: 0.000147\n",
      "2019-10-30 21:49:09,316 Epoch  22 Step:   197300 Batch Loss:     1.217106 Tokens per Sec:    17388, Lr: 0.000147\n",
      "2019-10-30 21:49:24,356 Epoch  22 Step:   197400 Batch Loss:     1.106646 Tokens per Sec:    17154, Lr: 0.000147\n",
      "2019-10-30 21:49:39,303 Epoch  22 Step:   197500 Batch Loss:     1.108408 Tokens per Sec:    17094, Lr: 0.000147\n",
      "2019-10-30 21:49:54,098 Epoch  22 Step:   197600 Batch Loss:     1.149620 Tokens per Sec:    16838, Lr: 0.000147\n",
      "2019-10-30 21:50:09,109 Epoch  22 Step:   197700 Batch Loss:     1.254587 Tokens per Sec:    17259, Lr: 0.000147\n",
      "2019-10-30 21:50:24,047 Epoch  22 Step:   197800 Batch Loss:     1.350021 Tokens per Sec:    17327, Lr: 0.000147\n",
      "2019-10-30 21:50:39,030 Epoch  22 Step:   197900 Batch Loss:     1.097796 Tokens per Sec:    16619, Lr: 0.000147\n",
      "2019-10-30 21:50:54,208 Epoch  22 Step:   198000 Batch Loss:     0.979952 Tokens per Sec:    17064, Lr: 0.000147\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 21:51:38,623 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 21:51:38,623 Saving new checkpoint.\n",
      "2019-10-30 21:51:38,770 Example #0\n",
      "2019-10-30 21:51:38,770 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:51:38,770 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:51:38,770 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:51:38,770 Example #1\n",
      "2019-10-30 21:51:38,770 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:51:38,770 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:51:38,770 \tHypothesis: Biblia haidumii sifa ya uthabiti .\n",
      "2019-10-30 21:51:38,770 Example #2\n",
      "2019-10-30 21:51:38,771 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:51:38,771 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:51:38,771 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 21:51:38,771 Example #3\n",
      "2019-10-30 21:51:38,771 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:51:38,771 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:51:38,771 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 21:51:38,771 Validation result at epoch  22, step   198000: bleu:  37.16, loss: 29360.5898, ppl:   2.9104, duration: 44.5618s\n",
      "2019-10-30 21:51:53,941 Epoch  22 Step:   198100 Batch Loss:     1.379707 Tokens per Sec:    16887, Lr: 0.000147\n",
      "2019-10-30 21:52:08,846 Epoch  22 Step:   198200 Batch Loss:     1.119455 Tokens per Sec:    16978, Lr: 0.000147\n",
      "2019-10-30 21:52:23,762 Epoch  22 Step:   198300 Batch Loss:     1.164280 Tokens per Sec:    17173, Lr: 0.000147\n",
      "2019-10-30 21:52:38,963 Epoch  22 Step:   198400 Batch Loss:     1.156776 Tokens per Sec:    17317, Lr: 0.000147\n",
      "2019-10-30 21:52:53,897 Epoch  22 Step:   198500 Batch Loss:     1.101401 Tokens per Sec:    17047, Lr: 0.000147\n",
      "2019-10-30 21:53:08,745 Epoch  22 Step:   198600 Batch Loss:     1.039910 Tokens per Sec:    17014, Lr: 0.000147\n",
      "2019-10-30 21:53:23,852 Epoch  22 Step:   198700 Batch Loss:     1.144072 Tokens per Sec:    16763, Lr: 0.000147\n",
      "2019-10-30 21:53:39,002 Epoch  22 Step:   198800 Batch Loss:     1.173471 Tokens per Sec:    17078, Lr: 0.000147\n",
      "2019-10-30 21:53:53,992 Epoch  22 Step:   198900 Batch Loss:     1.102703 Tokens per Sec:    17174, Lr: 0.000147\n",
      "2019-10-30 21:54:09,125 Epoch  22 Step:   199000 Batch Loss:     1.224718 Tokens per Sec:    17118, Lr: 0.000147\n",
      "2019-10-30 21:54:53,358 Example #0\n",
      "2019-10-30 21:54:53,358 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:54:53,358 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:54:53,358 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:54:53,358 Example #1\n",
      "2019-10-30 21:54:53,358 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:54:53,358 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:54:53,359 \tHypothesis: Biblia haidumii sifa ya uchangamfu .\n",
      "2019-10-30 21:54:53,359 Example #2\n",
      "2019-10-30 21:54:53,359 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:54:53,359 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:54:53,359 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuchochea nini ?\n",
      "2019-10-30 21:54:53,359 Example #3\n",
      "2019-10-30 21:54:53,359 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:54:53,359 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:54:53,359 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 21:54:53,359 Validation result at epoch  22, step   199000: bleu:  37.13, loss: 29496.4570, ppl:   2.9248, duration: 44.2341s\n",
      "2019-10-30 21:55:08,333 Epoch  22 Step:   199100 Batch Loss:     1.310269 Tokens per Sec:    17305, Lr: 0.000147\n",
      "2019-10-30 21:55:23,224 Epoch  22 Step:   199200 Batch Loss:     1.272616 Tokens per Sec:    17020, Lr: 0.000147\n",
      "2019-10-30 21:55:38,186 Epoch  22 Step:   199300 Batch Loss:     1.238474 Tokens per Sec:    17202, Lr: 0.000147\n",
      "2019-10-30 21:55:53,159 Epoch  22 Step:   199400 Batch Loss:     1.088622 Tokens per Sec:    17317, Lr: 0.000147\n",
      "2019-10-30 21:56:08,005 Epoch  22 Step:   199500 Batch Loss:     1.238190 Tokens per Sec:    17318, Lr: 0.000147\n",
      "2019-10-30 21:56:23,147 Epoch  22 Step:   199600 Batch Loss:     1.269024 Tokens per Sec:    17014, Lr: 0.000147\n",
      "2019-10-30 21:56:38,093 Epoch  22 Step:   199700 Batch Loss:     1.266702 Tokens per Sec:    17251, Lr: 0.000147\n",
      "2019-10-30 21:56:52,936 Epoch  22 Step:   199800 Batch Loss:     1.246232 Tokens per Sec:    17049, Lr: 0.000147\n",
      "2019-10-30 21:57:08,028 Epoch  22 Step:   199900 Batch Loss:     1.072017 Tokens per Sec:    17085, Lr: 0.000147\n",
      "2019-10-30 21:57:23,042 Epoch  22 Step:   200000 Batch Loss:     1.300480 Tokens per Sec:    17118, Lr: 0.000147\n",
      "2019-10-30 21:58:07,036 Example #0\n",
      "2019-10-30 21:58:07,036 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 21:58:07,036 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 21:58:07,036 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 21:58:07,036 Example #1\n",
      "2019-10-30 21:58:07,036 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 21:58:07,036 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 21:58:07,036 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sifa .\n",
      "2019-10-30 21:58:07,036 Example #2\n",
      "2019-10-30 21:58:07,036 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 21:58:07,036 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 21:58:07,036 \tHypothesis: Ni baadhi ya mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuchochea nini ?\n",
      "2019-10-30 21:58:07,037 Example #3\n",
      "2019-10-30 21:58:07,037 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 21:58:07,037 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 21:58:07,037 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 21:58:07,037 Validation result at epoch  22, step   200000: bleu:  37.21, loss: 29402.9766, ppl:   2.9149, duration: 43.9947s\n",
      "2019-10-30 21:58:22,188 Epoch  22 Step:   200100 Batch Loss:     1.081029 Tokens per Sec:    16926, Lr: 0.000147\n",
      "2019-10-30 21:58:37,042 Epoch  22 Step:   200200 Batch Loss:     1.296541 Tokens per Sec:    16956, Lr: 0.000147\n",
      "2019-10-30 21:58:51,850 Epoch  22 Step:   200300 Batch Loss:     1.265443 Tokens per Sec:    17338, Lr: 0.000147\n",
      "2019-10-30 21:59:07,072 Epoch  22 Step:   200400 Batch Loss:     1.249202 Tokens per Sec:    16727, Lr: 0.000147\n",
      "2019-10-30 21:59:21,966 Epoch  22 Step:   200500 Batch Loss:     1.074937 Tokens per Sec:    16644, Lr: 0.000147\n",
      "2019-10-30 21:59:36,773 Epoch  22 Step:   200600 Batch Loss:     1.234357 Tokens per Sec:    16972, Lr: 0.000147\n",
      "2019-10-30 21:59:51,862 Epoch  22 Step:   200700 Batch Loss:     1.161408 Tokens per Sec:    16909, Lr: 0.000147\n",
      "2019-10-30 22:00:06,759 Epoch  22 Step:   200800 Batch Loss:     1.135239 Tokens per Sec:    17075, Lr: 0.000147\n",
      "2019-10-30 22:00:21,822 Epoch  22 Step:   200900 Batch Loss:     1.271557 Tokens per Sec:    17351, Lr: 0.000147\n",
      "2019-10-30 22:00:36,986 Epoch  22 Step:   201000 Batch Loss:     1.194657 Tokens per Sec:    17028, Lr: 0.000147\n",
      "2019-10-30 22:01:21,224 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:01:21,224 Saving new checkpoint.\n",
      "2019-10-30 22:01:21,368 Example #0\n",
      "2019-10-30 22:01:21,369 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:01:21,369 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:01:21,369 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:01:21,369 Example #1\n",
      "2019-10-30 22:01:21,369 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:01:21,369 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:01:21,369 \tHypothesis: Biblia haidumii sifa .\n",
      "2019-10-30 22:01:21,369 Example #2\n",
      "2019-10-30 22:01:21,369 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:01:21,369 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:01:21,369 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 22:01:21,369 Example #3\n",
      "2019-10-30 22:01:21,369 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:01:21,369 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:01:21,369 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:01:21,369 Validation result at epoch  22, step   201000: bleu:  37.39, loss: 29291.9746, ppl:   2.9031, duration: 44.3829s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 22:01:36,440 Epoch  22 Step:   201100 Batch Loss:     1.437305 Tokens per Sec:    17158, Lr: 0.000147\n",
      "2019-10-30 22:01:51,472 Epoch  22 Step:   201200 Batch Loss:     1.261781 Tokens per Sec:    17118, Lr: 0.000147\n",
      "2019-10-30 22:02:06,612 Epoch  22 Step:   201300 Batch Loss:     1.156787 Tokens per Sec:    17091, Lr: 0.000147\n",
      "2019-10-30 22:02:21,823 Epoch  22 Step:   201400 Batch Loss:     1.084113 Tokens per Sec:    17218, Lr: 0.000147\n",
      "2019-10-30 22:02:36,758 Epoch  22 Step:   201500 Batch Loss:     1.368121 Tokens per Sec:    16659, Lr: 0.000147\n",
      "2019-10-30 22:02:51,685 Epoch  22 Step:   201600 Batch Loss:     1.266443 Tokens per Sec:    17182, Lr: 0.000147\n",
      "2019-10-30 22:03:06,609 Epoch  22 Step:   201700 Batch Loss:     1.123488 Tokens per Sec:    17015, Lr: 0.000147\n",
      "2019-10-30 22:03:21,818 Epoch  22 Step:   201800 Batch Loss:     1.173587 Tokens per Sec:    17157, Lr: 0.000147\n",
      "2019-10-30 22:03:36,840 Epoch  22 Step:   201900 Batch Loss:     1.019738 Tokens per Sec:    16892, Lr: 0.000147\n",
      "2019-10-30 22:03:52,153 Epoch  22 Step:   202000 Batch Loss:     1.186344 Tokens per Sec:    16546, Lr: 0.000147\n",
      "2019-10-30 22:04:36,136 Example #0\n",
      "2019-10-30 22:04:36,136 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:04:36,136 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:04:36,136 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:04:36,136 Example #1\n",
      "2019-10-30 22:04:36,136 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:04:36,136 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:04:36,136 \tHypothesis: Biblia haidumii sifa .\n",
      "2019-10-30 22:04:36,136 Example #2\n",
      "2019-10-30 22:04:36,136 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:04:36,136 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:04:36,137 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 22:04:36,137 Example #3\n",
      "2019-10-30 22:04:36,137 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:04:36,137 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:04:36,137 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-30 22:04:36,137 Validation result at epoch  22, step   202000: bleu:  37.14, loss: 29369.0117, ppl:   2.9113, duration: 43.9828s\n",
      "2019-10-30 22:04:51,268 Epoch  22 Step:   202100 Batch Loss:     1.236611 Tokens per Sec:    17213, Lr: 0.000147\n",
      "2019-10-30 22:05:06,116 Epoch  22 Step:   202200 Batch Loss:     1.120352 Tokens per Sec:    16950, Lr: 0.000147\n",
      "2019-10-30 22:05:21,239 Epoch  22 Step:   202300 Batch Loss:     1.179760 Tokens per Sec:    17071, Lr: 0.000147\n",
      "2019-10-30 22:05:36,348 Epoch  22 Step:   202400 Batch Loss:     1.122172 Tokens per Sec:    17209, Lr: 0.000147\n",
      "2019-10-30 22:05:51,210 Epoch  22 Step:   202500 Batch Loss:     1.161563 Tokens per Sec:    16976, Lr: 0.000147\n",
      "2019-10-30 22:06:06,023 Epoch  22 Step:   202600 Batch Loss:     1.238361 Tokens per Sec:    17066, Lr: 0.000147\n",
      "2019-10-30 22:06:21,015 Epoch  22 Step:   202700 Batch Loss:     1.363392 Tokens per Sec:    17158, Lr: 0.000147\n",
      "2019-10-30 22:06:35,891 Epoch  22 Step:   202800 Batch Loss:     1.289940 Tokens per Sec:    17112, Lr: 0.000147\n",
      "2019-10-30 22:06:50,779 Epoch  22 Step:   202900 Batch Loss:     1.103521 Tokens per Sec:    17574, Lr: 0.000147\n",
      "2019-10-30 22:07:05,775 Epoch  22 Step:   203000 Batch Loss:     1.205803 Tokens per Sec:    17201, Lr: 0.000147\n",
      "2019-10-30 22:07:49,749 Example #0\n",
      "2019-10-30 22:07:49,749 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:07:49,749 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:07:49,749 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:07:49,749 Example #1\n",
      "2019-10-30 22:07:49,749 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:07:49,749 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:07:49,749 \tHypothesis: Biblia haidumii sifa ya uchangamfu .\n",
      "2019-10-30 22:07:49,749 Example #2\n",
      "2019-10-30 22:07:49,750 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:07:49,750 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:07:49,750 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 22:07:49,750 Example #3\n",
      "2019-10-30 22:07:49,750 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:07:49,750 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:07:49,750 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:07:49,750 Validation result at epoch  22, step   203000: bleu:  37.46, loss: 29327.3945, ppl:   2.9069, duration: 43.9748s\n",
      "2019-10-30 22:08:04,695 Epoch  22 Step:   203100 Batch Loss:     1.243009 Tokens per Sec:    17163, Lr: 0.000147\n",
      "2019-10-30 22:08:19,608 Epoch  22 Step:   203200 Batch Loss:     1.150165 Tokens per Sec:    17088, Lr: 0.000147\n",
      "2019-10-30 22:08:34,911 Epoch  22 Step:   203300 Batch Loss:     1.207773 Tokens per Sec:    17383, Lr: 0.000147\n",
      "2019-10-30 22:08:49,914 Epoch  22 Step:   203400 Batch Loss:     1.103108 Tokens per Sec:    17160, Lr: 0.000147\n",
      "2019-10-30 22:09:05,102 Epoch  22 Step:   203500 Batch Loss:     1.122439 Tokens per Sec:    17558, Lr: 0.000147\n",
      "2019-10-30 22:09:20,007 Epoch  22 Step:   203600 Batch Loss:     1.114909 Tokens per Sec:    17115, Lr: 0.000147\n",
      "2019-10-30 22:09:34,952 Epoch  22 Step:   203700 Batch Loss:     1.100193 Tokens per Sec:    17145, Lr: 0.000147\n",
      "2019-10-30 22:09:50,259 Epoch  22 Step:   203800 Batch Loss:     1.170925 Tokens per Sec:    17143, Lr: 0.000147\n",
      "2019-10-30 22:10:05,337 Epoch  22 Step:   203900 Batch Loss:     1.135243 Tokens per Sec:    17126, Lr: 0.000147\n",
      "2019-10-30 22:10:20,304 Epoch  22 Step:   204000 Batch Loss:     1.218400 Tokens per Sec:    17470, Lr: 0.000147\n",
      "2019-10-30 22:11:04,683 Example #0\n",
      "2019-10-30 22:11:04,683 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:11:04,683 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:11:04,683 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:11:04,683 Example #1\n",
      "2019-10-30 22:11:04,683 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:11:04,683 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:11:04,683 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:11:04,683 Example #2\n",
      "2019-10-30 22:11:04,684 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:11:04,684 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:11:04,684 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 22:11:04,684 Example #3\n",
      "2019-10-30 22:11:04,684 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:11:04,684 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:11:04,684 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:11:04,684 Validation result at epoch  22, step   204000: bleu:  37.50, loss: 29321.8242, ppl:   2.9063, duration: 44.3797s\n",
      "2019-10-30 22:11:19,885 Epoch  22 Step:   204100 Batch Loss:     1.206318 Tokens per Sec:    16459, Lr: 0.000147\n",
      "2019-10-30 22:11:34,875 Epoch  22 Step:   204200 Batch Loss:     1.195137 Tokens per Sec:    16886, Lr: 0.000147\n",
      "2019-10-30 22:11:49,852 Epoch  22 Step:   204300 Batch Loss:     1.229366 Tokens per Sec:    17214, Lr: 0.000147\n",
      "2019-10-30 22:12:04,915 Epoch  22 Step:   204400 Batch Loss:     1.170631 Tokens per Sec:    17306, Lr: 0.000147\n",
      "2019-10-30 22:12:19,863 Epoch  22 Step:   204500 Batch Loss:     1.113690 Tokens per Sec:    17373, Lr: 0.000147\n",
      "2019-10-30 22:12:34,778 Epoch  22 Step:   204600 Batch Loss:     1.261625 Tokens per Sec:    16984, Lr: 0.000147\n",
      "2019-10-30 22:12:49,806 Epoch  22 Step:   204700 Batch Loss:     1.098156 Tokens per Sec:    17067, Lr: 0.000147\n",
      "2019-10-30 22:13:04,850 Epoch  22 Step:   204800 Batch Loss:     1.253085 Tokens per Sec:    17372, Lr: 0.000147\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 22:13:19,892 Epoch  22 Step:   204900 Batch Loss:     1.229861 Tokens per Sec:    16884, Lr: 0.000147\n",
      "2019-10-30 22:13:34,790 Epoch  22 Step:   205000 Batch Loss:     1.147763 Tokens per Sec:    17114, Lr: 0.000147\n",
      "2019-10-30 22:14:19,067 Example #0\n",
      "2019-10-30 22:14:19,067 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:14:19,067 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:14:19,067 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:14:19,067 Example #1\n",
      "2019-10-30 22:14:19,067 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:14:19,067 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:14:19,067 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sifa .\n",
      "2019-10-30 22:14:19,067 Example #2\n",
      "2019-10-30 22:14:19,068 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:14:19,068 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:14:19,068 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 22:14:19,068 Example #3\n",
      "2019-10-30 22:14:19,068 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:14:19,068 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:14:19,068 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 22:14:19,068 Validation result at epoch  22, step   205000: bleu:  37.45, loss: 29341.5254, ppl:   2.9083, duration: 44.2770s\n",
      "2019-10-30 22:14:34,139 Epoch  22 Step:   205100 Batch Loss:     1.200515 Tokens per Sec:    16975, Lr: 0.000147\n",
      "2019-10-30 22:14:48,987 Epoch  22 Step:   205200 Batch Loss:     1.209322 Tokens per Sec:    17291, Lr: 0.000147\n",
      "2019-10-30 22:15:03,722 Epoch  22 Step:   205300 Batch Loss:     1.144541 Tokens per Sec:    17148, Lr: 0.000147\n",
      "2019-10-30 22:15:18,655 Epoch  22 Step:   205400 Batch Loss:     1.034600 Tokens per Sec:    17000, Lr: 0.000147\n",
      "2019-10-30 22:15:33,864 Epoch  22 Step:   205500 Batch Loss:     1.259002 Tokens per Sec:    16593, Lr: 0.000147\n",
      "2019-10-30 22:15:49,065 Epoch  22 Step:   205600 Batch Loss:     1.351808 Tokens per Sec:    17462, Lr: 0.000147\n",
      "2019-10-30 22:16:03,959 Epoch  22 Step:   205700 Batch Loss:     1.051645 Tokens per Sec:    17244, Lr: 0.000147\n",
      "2019-10-30 22:16:18,996 Epoch  22 Step:   205800 Batch Loss:     1.061875 Tokens per Sec:    17180, Lr: 0.000147\n",
      "2019-10-30 22:16:34,018 Epoch  22 Step:   205900 Batch Loss:     1.230103 Tokens per Sec:    17469, Lr: 0.000147\n",
      "2019-10-30 22:16:49,269 Epoch  22 Step:   206000 Batch Loss:     1.216141 Tokens per Sec:    17095, Lr: 0.000147\n",
      "2019-10-30 22:17:33,356 Example #0\n",
      "2019-10-30 22:17:33,356 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:17:33,356 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:17:33,356 \tHypothesis: Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:17:33,356 Example #1\n",
      "2019-10-30 22:17:33,356 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:17:33,356 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:17:33,356 \tHypothesis: Biblia haidumii sifa .\n",
      "2019-10-30 22:17:33,356 Example #2\n",
      "2019-10-30 22:17:33,356 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:17:33,356 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:17:33,356 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:17:33,356 Example #3\n",
      "2019-10-30 22:17:33,357 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:17:33,357 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:17:33,357 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-30 22:17:33,357 Validation result at epoch  22, step   206000: bleu:  37.43, loss: 29334.0410, ppl:   2.9076, duration: 44.0874s\n",
      "2019-10-30 22:17:48,509 Epoch  22 Step:   206100 Batch Loss:     1.319171 Tokens per Sec:    16944, Lr: 0.000147\n",
      "2019-10-30 22:18:03,799 Epoch  22 Step:   206200 Batch Loss:     0.961281 Tokens per Sec:    17244, Lr: 0.000147\n",
      "2019-10-30 22:18:18,897 Epoch  22 Step:   206300 Batch Loss:     1.268240 Tokens per Sec:    17171, Lr: 0.000147\n",
      "2019-10-30 22:18:33,933 Epoch  22 Step:   206400 Batch Loss:     1.350381 Tokens per Sec:    16988, Lr: 0.000147\n",
      "2019-10-30 22:18:40,900 Epoch  22: total training loss 11201.93\n",
      "2019-10-30 22:18:40,900 EPOCH 23\n",
      "2019-10-30 22:18:51,322 Epoch  23 Step:   206500 Batch Loss:     1.146800 Tokens per Sec:    13544, Lr: 0.000147\n",
      "2019-10-30 22:19:07,213 Epoch  23 Step:   206600 Batch Loss:     1.341570 Tokens per Sec:    16292, Lr: 0.000147\n",
      "2019-10-30 22:19:22,604 Epoch  23 Step:   206700 Batch Loss:     1.157835 Tokens per Sec:    16819, Lr: 0.000147\n",
      "2019-10-30 22:19:37,788 Epoch  23 Step:   206800 Batch Loss:     1.078444 Tokens per Sec:    16975, Lr: 0.000147\n",
      "2019-10-30 22:19:52,795 Epoch  23 Step:   206900 Batch Loss:     1.093424 Tokens per Sec:    17143, Lr: 0.000147\n",
      "2019-10-30 22:20:07,716 Epoch  23 Step:   207000 Batch Loss:     1.246622 Tokens per Sec:    17097, Lr: 0.000147\n",
      "2019-10-30 22:20:51,739 Example #0\n",
      "2019-10-30 22:20:51,739 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:20:51,739 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:20:51,739 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:20:51,740 Example #1\n",
      "2019-10-30 22:20:51,740 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:20:51,740 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:20:51,740 \tHypothesis: Biblia haidumii sifa ya uchangamfu .\n",
      "2019-10-30 22:20:51,740 Example #2\n",
      "2019-10-30 22:20:51,740 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:20:51,740 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:20:51,740 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kuwa na matokeo gani ?\n",
      "2019-10-30 22:20:51,740 Example #3\n",
      "2019-10-30 22:20:51,740 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:20:51,740 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:20:51,740 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:20:51,740 Validation result at epoch  23, step   207000: bleu:  37.26, loss: 29393.3164, ppl:   2.9138, duration: 44.0234s\n",
      "2019-10-30 22:21:06,879 Epoch  23 Step:   207100 Batch Loss:     1.132242 Tokens per Sec:    17213, Lr: 0.000103\n",
      "2019-10-30 22:21:22,042 Epoch  23 Step:   207200 Batch Loss:     0.961234 Tokens per Sec:    17005, Lr: 0.000103\n",
      "2019-10-30 22:21:36,946 Epoch  23 Step:   207300 Batch Loss:     1.118590 Tokens per Sec:    16993, Lr: 0.000103\n",
      "2019-10-30 22:21:51,927 Epoch  23 Step:   207400 Batch Loss:     1.263821 Tokens per Sec:    17129, Lr: 0.000103\n",
      "2019-10-30 22:22:06,947 Epoch  23 Step:   207500 Batch Loss:     1.132622 Tokens per Sec:    17263, Lr: 0.000103\n",
      "2019-10-30 22:22:21,851 Epoch  23 Step:   207600 Batch Loss:     1.178226 Tokens per Sec:    17373, Lr: 0.000103\n",
      "2019-10-30 22:22:36,826 Epoch  23 Step:   207700 Batch Loss:     1.200523 Tokens per Sec:    17266, Lr: 0.000103\n",
      "2019-10-30 22:22:51,806 Epoch  23 Step:   207800 Batch Loss:     1.214514 Tokens per Sec:    17202, Lr: 0.000103\n",
      "2019-10-30 22:23:06,732 Epoch  23 Step:   207900 Batch Loss:     1.086230 Tokens per Sec:    17158, Lr: 0.000103\n",
      "2019-10-30 22:23:21,778 Epoch  23 Step:   208000 Batch Loss:     1.342783 Tokens per Sec:    17501, Lr: 0.000103\n",
      "2019-10-30 22:24:05,656 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:24:05,656 Saving new checkpoint.\n",
      "2019-10-30 22:24:05,840 Example #0\n",
      "2019-10-30 22:24:05,841 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:24:05,841 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:24:05,841 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:24:05,841 Example #1\n",
      "2019-10-30 22:24:05,841 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:24:05,841 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:24:05,841 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:24:05,841 Example #2\n",
      "2019-10-30 22:24:05,841 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:24:05,841 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:24:05,841 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:24:05,841 Example #3\n",
      "2019-10-30 22:24:05,841 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:24:05,841 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:24:05,842 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 22:24:05,842 Validation result at epoch  23, step   208000: bleu:  37.26, loss: 29155.8574, ppl:   2.8888, duration: 44.0627s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 22:24:20,646 Epoch  23 Step:   208100 Batch Loss:     1.195918 Tokens per Sec:    17065, Lr: 0.000103\n",
      "2019-10-30 22:24:35,411 Epoch  23 Step:   208200 Batch Loss:     1.245192 Tokens per Sec:    17071, Lr: 0.000103\n",
      "2019-10-30 22:24:50,150 Epoch  23 Step:   208300 Batch Loss:     1.020122 Tokens per Sec:    17334, Lr: 0.000103\n",
      "2019-10-30 22:25:05,110 Epoch  23 Step:   208400 Batch Loss:     1.136102 Tokens per Sec:    17499, Lr: 0.000103\n",
      "2019-10-30 22:25:20,201 Epoch  23 Step:   208500 Batch Loss:     1.044354 Tokens per Sec:    17151, Lr: 0.000103\n",
      "2019-10-30 22:25:35,219 Epoch  23 Step:   208600 Batch Loss:     1.187647 Tokens per Sec:    17153, Lr: 0.000103\n",
      "2019-10-30 22:25:50,323 Epoch  23 Step:   208700 Batch Loss:     1.254962 Tokens per Sec:    17297, Lr: 0.000103\n",
      "2019-10-30 22:26:05,397 Epoch  23 Step:   208800 Batch Loss:     1.156076 Tokens per Sec:    17005, Lr: 0.000103\n",
      "2019-10-30 22:26:20,390 Epoch  23 Step:   208900 Batch Loss:     1.149101 Tokens per Sec:    17223, Lr: 0.000103\n",
      "2019-10-30 22:26:35,328 Epoch  23 Step:   209000 Batch Loss:     1.177355 Tokens per Sec:    16803, Lr: 0.000103\n",
      "2019-10-30 22:27:19,241 Example #0\n",
      "2019-10-30 22:27:19,241 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:27:19,241 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:27:19,241 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:27:19,241 Example #1\n",
      "2019-10-30 22:27:19,241 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:27:19,241 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:27:19,241 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:27:19,242 Example #2\n",
      "2019-10-30 22:27:19,242 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:27:19,242 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:27:19,242 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 22:27:19,242 Example #3\n",
      "2019-10-30 22:27:19,242 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:27:19,242 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:27:19,242 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 22:27:19,242 Validation result at epoch  23, step   209000: bleu:  37.36, loss: 29194.0352, ppl:   2.8928, duration: 43.9132s\n",
      "2019-10-30 22:27:34,070 Epoch  23 Step:   209100 Batch Loss:     1.029445 Tokens per Sec:    17109, Lr: 0.000103\n",
      "2019-10-30 22:27:49,129 Epoch  23 Step:   209200 Batch Loss:     1.209657 Tokens per Sec:    17323, Lr: 0.000103\n",
      "2019-10-30 22:28:04,307 Epoch  23 Step:   209300 Batch Loss:     1.202495 Tokens per Sec:    16739, Lr: 0.000103\n",
      "2019-10-30 22:28:19,623 Epoch  23 Step:   209400 Batch Loss:     1.207912 Tokens per Sec:    16857, Lr: 0.000103\n",
      "2019-10-30 22:28:34,609 Epoch  23 Step:   209500 Batch Loss:     1.332734 Tokens per Sec:    17248, Lr: 0.000103\n",
      "2019-10-30 22:28:49,520 Epoch  23 Step:   209600 Batch Loss:     1.265433 Tokens per Sec:    17162, Lr: 0.000103\n",
      "2019-10-30 22:29:04,596 Epoch  23 Step:   209700 Batch Loss:     1.353736 Tokens per Sec:    17142, Lr: 0.000103\n",
      "2019-10-30 22:29:19,464 Epoch  23 Step:   209800 Batch Loss:     1.287368 Tokens per Sec:    16758, Lr: 0.000103\n",
      "2019-10-30 22:29:34,524 Epoch  23 Step:   209900 Batch Loss:     1.174554 Tokens per Sec:    16982, Lr: 0.000103\n",
      "2019-10-30 22:29:49,653 Epoch  23 Step:   210000 Batch Loss:     1.145511 Tokens per Sec:    17245, Lr: 0.000103\n",
      "2019-10-30 22:30:33,712 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:30:33,712 Saving new checkpoint.\n",
      "2019-10-30 22:30:33,854 Example #0\n",
      "2019-10-30 22:30:33,855 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:30:33,855 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:30:33,855 \tHypothesis: Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:30:33,855 Example #1\n",
      "2019-10-30 22:30:33,855 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:30:33,855 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:30:33,855 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:30:33,855 Example #2\n",
      "2019-10-30 22:30:33,855 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:30:33,855 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:30:33,855 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:30:33,855 Example #3\n",
      "2019-10-30 22:30:33,855 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:30:33,855 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:30:33,855 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-30 22:30:33,855 Validation result at epoch  23, step   210000: bleu:  37.40, loss: 29146.2598, ppl:   2.8878, duration: 44.2014s\n",
      "2019-10-30 22:30:49,002 Epoch  23 Step:   210100 Batch Loss:     1.125130 Tokens per Sec:    16973, Lr: 0.000103\n",
      "2019-10-30 22:31:04,126 Epoch  23 Step:   210200 Batch Loss:     1.264107 Tokens per Sec:    17121, Lr: 0.000103\n",
      "2019-10-30 22:31:19,099 Epoch  23 Step:   210300 Batch Loss:     1.095070 Tokens per Sec:    17260, Lr: 0.000103\n",
      "2019-10-30 22:31:34,174 Epoch  23 Step:   210400 Batch Loss:     1.119464 Tokens per Sec:    16989, Lr: 0.000103\n",
      "2019-10-30 22:31:49,304 Epoch  23 Step:   210500 Batch Loss:     1.181006 Tokens per Sec:    17408, Lr: 0.000103\n",
      "2019-10-30 22:32:04,097 Epoch  23 Step:   210600 Batch Loss:     1.214929 Tokens per Sec:    16964, Lr: 0.000103\n",
      "2019-10-30 22:32:18,920 Epoch  23 Step:   210700 Batch Loss:     1.059378 Tokens per Sec:    17120, Lr: 0.000103\n",
      "2019-10-30 22:32:33,634 Epoch  23 Step:   210800 Batch Loss:     1.245999 Tokens per Sec:    17063, Lr: 0.000103\n",
      "2019-10-30 22:32:48,578 Epoch  23 Step:   210900 Batch Loss:     1.235661 Tokens per Sec:    17068, Lr: 0.000103\n",
      "2019-10-30 22:33:03,595 Epoch  23 Step:   211000 Batch Loss:     1.227435 Tokens per Sec:    17063, Lr: 0.000103\n",
      "2019-10-30 22:33:48,001 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:33:48,001 Saving new checkpoint.\n",
      "2019-10-30 22:33:48,150 Example #0\n",
      "2019-10-30 22:33:48,150 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:33:48,150 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:33:48,150 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:33:48,150 Example #1\n",
      "2019-10-30 22:33:48,150 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:33:48,150 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:33:48,150 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:33:48,150 Example #2\n",
      "2019-10-30 22:33:48,150 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:33:48,150 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:33:48,150 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:33:48,150 Example #3\n",
      "2019-10-30 22:33:48,151 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:33:48,151 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:33:48,151 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 22:33:48,151 Validation result at epoch  23, step   211000: bleu:  37.01, loss: 29122.3105, ppl:   2.8852, duration: 44.5550s\n",
      "2019-10-30 22:34:03,200 Epoch  23 Step:   211100 Batch Loss:     1.170426 Tokens per Sec:    16797, Lr: 0.000103\n",
      "2019-10-30 22:34:18,035 Epoch  23 Step:   211200 Batch Loss:     1.127115 Tokens per Sec:    17343, Lr: 0.000103\n",
      "2019-10-30 22:34:32,934 Epoch  23 Step:   211300 Batch Loss:     1.171736 Tokens per Sec:    17147, Lr: 0.000103\n",
      "2019-10-30 22:34:47,899 Epoch  23 Step:   211400 Batch Loss:     1.220840 Tokens per Sec:    16982, Lr: 0.000103\n",
      "2019-10-30 22:35:02,839 Epoch  23 Step:   211500 Batch Loss:     1.018752 Tokens per Sec:    17209, Lr: 0.000103\n",
      "2019-10-30 22:35:17,891 Epoch  23 Step:   211600 Batch Loss:     1.240115 Tokens per Sec:    17141, Lr: 0.000103\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 22:35:33,006 Epoch  23 Step:   211700 Batch Loss:     1.113848 Tokens per Sec:    16792, Lr: 0.000103\n",
      "2019-10-30 22:35:47,928 Epoch  23 Step:   211800 Batch Loss:     1.168357 Tokens per Sec:    16908, Lr: 0.000103\n",
      "2019-10-30 22:36:02,768 Epoch  23 Step:   211900 Batch Loss:     1.127887 Tokens per Sec:    16713, Lr: 0.000103\n",
      "2019-10-30 22:36:17,735 Epoch  23 Step:   212000 Batch Loss:     1.349949 Tokens per Sec:    17033, Lr: 0.000103\n",
      "2019-10-30 22:37:02,033 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:37:02,033 Saving new checkpoint.\n",
      "2019-10-30 22:37:02,182 Example #0\n",
      "2019-10-30 22:37:02,182 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:37:02,182 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:37:02,182 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:37:02,182 Example #1\n",
      "2019-10-30 22:37:02,183 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:37:02,183 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:37:02,183 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:37:02,183 Example #2\n",
      "2019-10-30 22:37:02,183 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:37:02,183 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:37:02,183 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:37:02,183 Example #3\n",
      "2019-10-30 22:37:02,183 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:37:02,183 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:37:02,183 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:37:02,183 Validation result at epoch  23, step   212000: bleu:  37.23, loss: 29096.9336, ppl:   2.8826, duration: 44.4475s\n",
      "2019-10-30 22:37:17,330 Epoch  23 Step:   212100 Batch Loss:     1.067303 Tokens per Sec:    16903, Lr: 0.000103\n",
      "2019-10-30 22:37:32,596 Epoch  23 Step:   212200 Batch Loss:     1.278064 Tokens per Sec:    17470, Lr: 0.000103\n",
      "2019-10-30 22:37:47,563 Epoch  23 Step:   212300 Batch Loss:     1.372585 Tokens per Sec:    17140, Lr: 0.000103\n",
      "2019-10-30 22:38:02,577 Epoch  23 Step:   212400 Batch Loss:     1.162575 Tokens per Sec:    17281, Lr: 0.000103\n",
      "2019-10-30 22:38:17,692 Epoch  23 Step:   212500 Batch Loss:     1.278782 Tokens per Sec:    17580, Lr: 0.000103\n",
      "2019-10-30 22:38:32,938 Epoch  23 Step:   212600 Batch Loss:     1.170267 Tokens per Sec:    16906, Lr: 0.000103\n",
      "2019-10-30 22:38:47,967 Epoch  23 Step:   212700 Batch Loss:     1.245395 Tokens per Sec:    16805, Lr: 0.000103\n",
      "2019-10-30 22:39:03,000 Epoch  23 Step:   212800 Batch Loss:     1.171780 Tokens per Sec:    17212, Lr: 0.000103\n",
      "2019-10-30 22:39:17,958 Epoch  23 Step:   212900 Batch Loss:     1.038269 Tokens per Sec:    17143, Lr: 0.000103\n",
      "2019-10-30 22:39:32,970 Epoch  23 Step:   213000 Batch Loss:     1.030711 Tokens per Sec:    17070, Lr: 0.000103\n",
      "2019-10-30 22:40:17,019 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:40:17,019 Saving new checkpoint.\n",
      "2019-10-30 22:40:17,169 Example #0\n",
      "2019-10-30 22:40:17,169 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:40:17,169 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:40:17,169 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:40:17,169 Example #1\n",
      "2019-10-30 22:40:17,169 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:40:17,169 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:40:17,169 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:40:17,170 Example #2\n",
      "2019-10-30 22:40:17,170 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:40:17,170 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:40:17,170 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:40:17,170 Example #3\n",
      "2019-10-30 22:40:17,170 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:40:17,170 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:40:17,170 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:40:17,170 Validation result at epoch  23, step   213000: bleu:  36.99, loss: 29072.6641, ppl:   2.8800, duration: 44.1992s\n",
      "2019-10-30 22:40:31,999 Epoch  23 Step:   213100 Batch Loss:     1.116379 Tokens per Sec:    17238, Lr: 0.000103\n",
      "2019-10-30 22:40:46,855 Epoch  23 Step:   213200 Batch Loss:     1.012223 Tokens per Sec:    17391, Lr: 0.000103\n",
      "2019-10-30 22:41:01,672 Epoch  23 Step:   213300 Batch Loss:     1.406735 Tokens per Sec:    17508, Lr: 0.000103\n",
      "2019-10-30 22:41:16,559 Epoch  23 Step:   213400 Batch Loss:     1.151498 Tokens per Sec:    17498, Lr: 0.000103\n",
      "2019-10-30 22:41:31,822 Epoch  23 Step:   213500 Batch Loss:     1.169102 Tokens per Sec:    16675, Lr: 0.000103\n",
      "2019-10-30 22:41:46,809 Epoch  23 Step:   213600 Batch Loss:     1.131881 Tokens per Sec:    16855, Lr: 0.000103\n",
      "2019-10-30 22:42:01,676 Epoch  23 Step:   213700 Batch Loss:     1.087751 Tokens per Sec:    17225, Lr: 0.000103\n",
      "2019-10-30 22:42:16,560 Epoch  23 Step:   213800 Batch Loss:     1.265628 Tokens per Sec:    17406, Lr: 0.000103\n",
      "2019-10-30 22:42:31,453 Epoch  23 Step:   213900 Batch Loss:     1.126278 Tokens per Sec:    17425, Lr: 0.000103\n",
      "2019-10-30 22:42:46,390 Epoch  23 Step:   214000 Batch Loss:     1.267327 Tokens per Sec:    17272, Lr: 0.000103\n",
      "2019-10-30 22:43:30,587 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:43:30,587 Saving new checkpoint.\n",
      "2019-10-30 22:43:30,735 Example #0\n",
      "2019-10-30 22:43:30,735 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:43:30,735 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:43:30,735 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:43:30,735 Example #1\n",
      "2019-10-30 22:43:30,736 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:43:30,736 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:43:30,736 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:43:30,736 Example #2\n",
      "2019-10-30 22:43:30,736 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:43:30,736 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:43:30,736 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-30 22:43:30,736 Example #3\n",
      "2019-10-30 22:43:30,736 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:43:30,736 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:43:30,736 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-30 22:43:30,736 Validation result at epoch  23, step   214000: bleu:  37.22, loss: 28987.4570, ppl:   2.8711, duration: 44.3457s\n",
      "2019-10-30 22:43:46,100 Epoch  23 Step:   214100 Batch Loss:     1.307872 Tokens per Sec:    17007, Lr: 0.000103\n",
      "2019-10-30 22:44:01,115 Epoch  23 Step:   214200 Batch Loss:     1.085981 Tokens per Sec:    16986, Lr: 0.000103\n",
      "2019-10-30 22:44:16,338 Epoch  23 Step:   214300 Batch Loss:     1.176683 Tokens per Sec:    17061, Lr: 0.000103\n",
      "2019-10-30 22:44:31,367 Epoch  23 Step:   214400 Batch Loss:     1.326839 Tokens per Sec:    17419, Lr: 0.000103\n",
      "2019-10-30 22:44:46,346 Epoch  23 Step:   214500 Batch Loss:     1.042389 Tokens per Sec:    17098, Lr: 0.000103\n",
      "2019-10-30 22:45:01,223 Epoch  23 Step:   214600 Batch Loss:     1.088935 Tokens per Sec:    17243, Lr: 0.000103\n",
      "2019-10-30 22:45:16,101 Epoch  23 Step:   214700 Batch Loss:     1.040966 Tokens per Sec:    17455, Lr: 0.000103\n",
      "2019-10-30 22:45:30,999 Epoch  23 Step:   214800 Batch Loss:     1.222209 Tokens per Sec:    17081, Lr: 0.000103\n",
      "2019-10-30 22:45:45,796 Epoch  23 Step:   214900 Batch Loss:     1.038229 Tokens per Sec:    17149, Lr: 0.000103\n",
      "2019-10-30 22:46:00,851 Epoch  23 Step:   215000 Batch Loss:     1.179031 Tokens per Sec:    17300, Lr: 0.000103\n",
      "2019-10-30 22:46:45,113 Example #0\n",
      "2019-10-30 22:46:45,114 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:46:45,114 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:46:45,114 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:46:45,114 Example #1\n",
      "2019-10-30 22:46:45,114 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:46:45,114 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:46:45,114 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 22:46:45,114 Example #2\n",
      "2019-10-30 22:46:45,114 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:46:45,114 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:46:45,114 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:46:45,114 Example #3\n",
      "2019-10-30 22:46:45,115 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:46:45,115 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:46:45,115 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-30 22:46:45,115 Validation result at epoch  23, step   215000: bleu:  37.19, loss: 29014.0879, ppl:   2.8739, duration: 44.2634s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 22:46:59,953 Epoch  23 Step:   215100 Batch Loss:     0.993606 Tokens per Sec:    17065, Lr: 0.000103\n",
      "2019-10-30 22:47:14,939 Epoch  23 Step:   215200 Batch Loss:     1.164497 Tokens per Sec:    17272, Lr: 0.000103\n",
      "2019-10-30 22:47:29,921 Epoch  23 Step:   215300 Batch Loss:     1.060755 Tokens per Sec:    17279, Lr: 0.000103\n",
      "2019-10-30 22:47:44,938 Epoch  23 Step:   215400 Batch Loss:     1.309820 Tokens per Sec:    16877, Lr: 0.000103\n",
      "2019-10-30 22:47:59,584 Epoch  23 Step:   215500 Batch Loss:     1.164031 Tokens per Sec:    16857, Lr: 0.000103\n",
      "2019-10-30 22:48:14,454 Epoch  23 Step:   215600 Batch Loss:     1.284957 Tokens per Sec:    17427, Lr: 0.000103\n",
      "2019-10-30 22:48:29,376 Epoch  23 Step:   215700 Batch Loss:     1.014837 Tokens per Sec:    17061, Lr: 0.000103\n",
      "2019-10-30 22:48:44,435 Epoch  23 Step:   215800 Batch Loss:     1.244782 Tokens per Sec:    17469, Lr: 0.000103\n",
      "2019-10-30 22:48:49,020 Epoch  23: total training loss 11080.33\n",
      "2019-10-30 22:48:49,021 EPOCH 24\n",
      "2019-10-30 22:49:00,783 Epoch  24 Step:   215900 Batch Loss:     1.044441 Tokens per Sec:    15555, Lr: 0.000103\n",
      "2019-10-30 22:49:15,878 Epoch  24 Step:   216000 Batch Loss:     1.248989 Tokens per Sec:    17252, Lr: 0.000103\n",
      "2019-10-30 22:49:59,812 Example #0\n",
      "2019-10-30 22:49:59,812 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:49:59,812 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:49:59,812 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:49:59,812 Example #1\n",
      "2019-10-30 22:49:59,812 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:49:59,812 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:49:59,812 \tHypothesis: Biblia haidhuru sifa .\n",
      "2019-10-30 22:49:59,812 Example #2\n",
      "2019-10-30 22:49:59,812 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:49:59,813 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:49:59,813 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:49:59,813 Example #3\n",
      "2019-10-30 22:49:59,813 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:49:59,813 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:49:59,813 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 22:49:59,813 Validation result at epoch  24, step   216000: bleu:  37.55, loss: 29029.6797, ppl:   2.8755, duration: 43.9339s\n",
      "2019-10-30 22:50:14,711 Epoch  24 Step:   216100 Batch Loss:     0.983770 Tokens per Sec:    16983, Lr: 0.000103\n",
      "2019-10-30 22:50:29,425 Epoch  24 Step:   216200 Batch Loss:     1.116418 Tokens per Sec:    17332, Lr: 0.000103\n",
      "2019-10-30 22:50:44,305 Epoch  24 Step:   216300 Batch Loss:     1.214296 Tokens per Sec:    17181, Lr: 0.000103\n",
      "2019-10-30 22:50:59,380 Epoch  24 Step:   216400 Batch Loss:     1.214550 Tokens per Sec:    17148, Lr: 0.000103\n",
      "2019-10-30 22:51:14,187 Epoch  24 Step:   216500 Batch Loss:     1.209974 Tokens per Sec:    17159, Lr: 0.000103\n",
      "2019-10-30 22:51:29,239 Epoch  24 Step:   216600 Batch Loss:     1.194159 Tokens per Sec:    16873, Lr: 0.000103\n",
      "2019-10-30 22:51:44,179 Epoch  24 Step:   216700 Batch Loss:     1.138254 Tokens per Sec:    17219, Lr: 0.000103\n",
      "2019-10-30 22:51:59,050 Epoch  24 Step:   216800 Batch Loss:     1.129535 Tokens per Sec:    17162, Lr: 0.000103\n",
      "2019-10-30 22:52:14,206 Epoch  24 Step:   216900 Batch Loss:     1.087114 Tokens per Sec:    16949, Lr: 0.000103\n",
      "2019-10-30 22:52:29,262 Epoch  24 Step:   217000 Batch Loss:     1.256919 Tokens per Sec:    17283, Lr: 0.000103\n",
      "2019-10-30 22:53:13,206 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:53:13,206 Saving new checkpoint.\n",
      "2019-10-30 22:53:13,347 Example #0\n",
      "2019-10-30 22:53:13,347 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:53:13,347 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:53:13,347 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:53:13,347 Example #1\n",
      "2019-10-30 22:53:13,347 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:53:13,347 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:53:13,347 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 22:53:13,347 Example #2\n",
      "2019-10-30 22:53:13,347 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:53:13,347 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:53:13,347 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 22:53:13,347 Example #3\n",
      "2019-10-30 22:53:13,348 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:53:13,348 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:53:13,348 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 22:53:13,348 Validation result at epoch  24, step   217000: bleu:  37.04, loss: 28986.4980, ppl:   2.8710, duration: 44.0849s\n",
      "2019-10-30 22:53:28,360 Epoch  24 Step:   217100 Batch Loss:     1.193440 Tokens per Sec:    17437, Lr: 0.000103\n",
      "2019-10-30 22:53:43,222 Epoch  24 Step:   217200 Batch Loss:     1.210952 Tokens per Sec:    16994, Lr: 0.000103\n",
      "2019-10-30 22:53:58,356 Epoch  24 Step:   217300 Batch Loss:     1.243709 Tokens per Sec:    16963, Lr: 0.000103\n",
      "2019-10-30 22:54:13,285 Epoch  24 Step:   217400 Batch Loss:     1.171359 Tokens per Sec:    16769, Lr: 0.000103\n",
      "2019-10-30 22:54:28,323 Epoch  24 Step:   217500 Batch Loss:     1.106835 Tokens per Sec:    17182, Lr: 0.000103\n",
      "2019-10-30 22:54:43,220 Epoch  24 Step:   217600 Batch Loss:     1.116722 Tokens per Sec:    17033, Lr: 0.000103\n",
      "2019-10-30 22:54:58,259 Epoch  24 Step:   217700 Batch Loss:     1.221925 Tokens per Sec:    17054, Lr: 0.000103\n",
      "2019-10-30 22:55:13,170 Epoch  24 Step:   217800 Batch Loss:     1.204191 Tokens per Sec:    17153, Lr: 0.000103\n",
      "2019-10-30 22:55:28,296 Epoch  24 Step:   217900 Batch Loss:     1.086527 Tokens per Sec:    17351, Lr: 0.000103\n",
      "2019-10-30 22:55:43,191 Epoch  24 Step:   218000 Batch Loss:     1.261709 Tokens per Sec:    17155, Lr: 0.000103\n",
      "2019-10-30 22:56:27,191 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:56:27,191 Saving new checkpoint.\n",
      "2019-10-30 22:56:27,376 Example #0\n",
      "2019-10-30 22:56:27,377 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:56:27,377 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:56:27,377 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:56:27,377 Example #1\n",
      "2019-10-30 22:56:27,377 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:56:27,377 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:56:27,377 \tHypothesis: Biblia haidumu sifa ya kukubali sifa .\n",
      "2019-10-30 22:56:27,377 Example #2\n",
      "2019-10-30 22:56:27,377 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:56:27,377 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:56:27,377 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 22:56:27,377 Example #3\n",
      "2019-10-30 22:56:27,377 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:56:27,377 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:56:27,377 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-30 22:56:27,377 Validation result at epoch  24, step   218000: bleu:  37.67, loss: 28982.6270, ppl:   2.8706, duration: 44.1861s\n",
      "2019-10-30 22:56:42,447 Epoch  24 Step:   218100 Batch Loss:     1.187714 Tokens per Sec:    17143, Lr: 0.000103\n",
      "2019-10-30 22:56:57,416 Epoch  24 Step:   218200 Batch Loss:     1.078149 Tokens per Sec:    17048, Lr: 0.000103\n",
      "2019-10-30 22:57:12,473 Epoch  24 Step:   218300 Batch Loss:     1.025550 Tokens per Sec:    16828, Lr: 0.000103\n",
      "2019-10-30 22:57:27,647 Epoch  24 Step:   218400 Batch Loss:     1.278171 Tokens per Sec:    17150, Lr: 0.000103\n",
      "2019-10-30 22:57:42,488 Epoch  24 Step:   218500 Batch Loss:     1.327098 Tokens per Sec:    17156, Lr: 0.000103\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 22:57:57,391 Epoch  24 Step:   218600 Batch Loss:     1.242478 Tokens per Sec:    17312, Lr: 0.000103\n",
      "2019-10-30 22:58:12,395 Epoch  24 Step:   218700 Batch Loss:     1.134521 Tokens per Sec:    17044, Lr: 0.000103\n",
      "2019-10-30 22:58:27,428 Epoch  24 Step:   218800 Batch Loss:     1.255663 Tokens per Sec:    16661, Lr: 0.000103\n",
      "2019-10-30 22:58:42,555 Epoch  24 Step:   218900 Batch Loss:     1.245463 Tokens per Sec:    17124, Lr: 0.000103\n",
      "2019-10-30 22:58:57,556 Epoch  24 Step:   219000 Batch Loss:     1.134705 Tokens per Sec:    17219, Lr: 0.000103\n",
      "2019-10-30 22:59:41,398 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 22:59:41,398 Saving new checkpoint.\n",
      "2019-10-30 22:59:41,538 Example #0\n",
      "2019-10-30 22:59:41,538 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 22:59:41,538 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 22:59:41,538 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 22:59:41,538 Example #1\n",
      "2019-10-30 22:59:41,538 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 22:59:41,538 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 22:59:41,538 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali .\n",
      "2019-10-30 22:59:41,538 Example #2\n",
      "2019-10-30 22:59:41,538 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 22:59:41,538 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 22:59:41,538 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 22:59:41,538 Example #3\n",
      "2019-10-30 22:59:41,538 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 22:59:41,538 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 22:59:41,539 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 22:59:41,539 Validation result at epoch  24, step   219000: bleu:  37.35, loss: 28973.8184, ppl:   2.8697, duration: 43.9820s\n",
      "2019-10-30 22:59:56,587 Epoch  24 Step:   219100 Batch Loss:     1.101646 Tokens per Sec:    17068, Lr: 0.000103\n",
      "2019-10-30 23:00:11,525 Epoch  24 Step:   219200 Batch Loss:     1.300903 Tokens per Sec:    17376, Lr: 0.000103\n",
      "2019-10-30 23:00:26,361 Epoch  24 Step:   219300 Batch Loss:     1.348914 Tokens per Sec:    17264, Lr: 0.000103\n",
      "2019-10-30 23:00:41,276 Epoch  24 Step:   219400 Batch Loss:     1.346097 Tokens per Sec:    16920, Lr: 0.000103\n",
      "2019-10-30 23:00:56,130 Epoch  24 Step:   219500 Batch Loss:     1.286721 Tokens per Sec:    17476, Lr: 0.000103\n",
      "2019-10-30 23:01:11,258 Epoch  24 Step:   219600 Batch Loss:     1.050594 Tokens per Sec:    16922, Lr: 0.000103\n",
      "2019-10-30 23:01:26,200 Epoch  24 Step:   219700 Batch Loss:     1.199667 Tokens per Sec:    17349, Lr: 0.000103\n",
      "2019-10-30 23:01:41,088 Epoch  24 Step:   219800 Batch Loss:     1.337210 Tokens per Sec:    17116, Lr: 0.000103\n",
      "2019-10-30 23:01:55,970 Epoch  24 Step:   219900 Batch Loss:     1.262029 Tokens per Sec:    17204, Lr: 0.000103\n",
      "2019-10-30 23:02:11,010 Epoch  24 Step:   220000 Batch Loss:     1.150214 Tokens per Sec:    16989, Lr: 0.000103\n",
      "2019-10-30 23:02:54,898 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:02:54,899 Saving new checkpoint.\n",
      "2019-10-30 23:02:55,041 Example #0\n",
      "2019-10-30 23:02:55,041 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:02:55,042 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:02:55,042 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:02:55,042 Example #1\n",
      "2019-10-30 23:02:55,042 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:02:55,042 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:02:55,042 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:02:55,042 Example #2\n",
      "2019-10-30 23:02:55,042 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:02:55,042 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:02:55,042 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 23:02:55,042 Example #3\n",
      "2019-10-30 23:02:55,042 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:02:55,042 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:02:55,042 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 23:02:55,042 Validation result at epoch  24, step   220000: bleu:  37.48, loss: 28919.0605, ppl:   2.8640, duration: 44.0321s\n",
      "2019-10-30 23:03:10,002 Epoch  24 Step:   220100 Batch Loss:     1.227959 Tokens per Sec:    17086, Lr: 0.000103\n",
      "2019-10-30 23:03:24,964 Epoch  24 Step:   220200 Batch Loss:     1.325984 Tokens per Sec:    17094, Lr: 0.000103\n",
      "2019-10-30 23:03:39,994 Epoch  24 Step:   220300 Batch Loss:     1.177501 Tokens per Sec:    17364, Lr: 0.000103\n",
      "2019-10-30 23:03:55,051 Epoch  24 Step:   220400 Batch Loss:     1.142296 Tokens per Sec:    17403, Lr: 0.000103\n",
      "2019-10-30 23:04:09,851 Epoch  24 Step:   220500 Batch Loss:     0.945962 Tokens per Sec:    17248, Lr: 0.000103\n",
      "2019-10-30 23:04:24,741 Epoch  24 Step:   220600 Batch Loss:     1.063171 Tokens per Sec:    17354, Lr: 0.000103\n",
      "2019-10-30 23:04:39,801 Epoch  24 Step:   220700 Batch Loss:     1.287006 Tokens per Sec:    17263, Lr: 0.000103\n",
      "2019-10-30 23:04:54,681 Epoch  24 Step:   220800 Batch Loss:     1.146118 Tokens per Sec:    17102, Lr: 0.000103\n",
      "2019-10-30 23:05:09,696 Epoch  24 Step:   220900 Batch Loss:     1.285082 Tokens per Sec:    17365, Lr: 0.000103\n",
      "2019-10-30 23:05:24,820 Epoch  24 Step:   221000 Batch Loss:     1.138413 Tokens per Sec:    17032, Lr: 0.000103\n",
      "2019-10-30 23:06:08,907 Example #0\n",
      "2019-10-30 23:06:08,907 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:06:08,907 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:06:08,907 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:06:08,907 Example #1\n",
      "2019-10-30 23:06:08,908 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:06:08,908 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:06:08,908 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:06:08,908 Example #2\n",
      "2019-10-30 23:06:08,908 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:06:08,908 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:06:08,908 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:06:08,908 Example #3\n",
      "2019-10-30 23:06:08,908 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:06:08,908 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:06:08,908 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 23:06:08,908 Validation result at epoch  24, step   221000: bleu:  37.43, loss: 28956.2500, ppl:   2.8679, duration: 44.0880s\n",
      "2019-10-30 23:06:23,979 Epoch  24 Step:   221100 Batch Loss:     1.148534 Tokens per Sec:    16701, Lr: 0.000103\n",
      "2019-10-30 23:06:38,945 Epoch  24 Step:   221200 Batch Loss:     1.063757 Tokens per Sec:    17057, Lr: 0.000103\n",
      "2019-10-30 23:06:53,934 Epoch  24 Step:   221300 Batch Loss:     1.108780 Tokens per Sec:    17369, Lr: 0.000103\n",
      "2019-10-30 23:07:08,980 Epoch  24 Step:   221400 Batch Loss:     1.060498 Tokens per Sec:    17586, Lr: 0.000103\n",
      "2019-10-30 23:07:24,058 Epoch  24 Step:   221500 Batch Loss:     1.100345 Tokens per Sec:    17223, Lr: 0.000103\n",
      "2019-10-30 23:07:39,224 Epoch  24 Step:   221600 Batch Loss:     1.304268 Tokens per Sec:    17257, Lr: 0.000103\n",
      "2019-10-30 23:07:54,090 Epoch  24 Step:   221700 Batch Loss:     1.167812 Tokens per Sec:    17088, Lr: 0.000103\n",
      "2019-10-30 23:08:09,137 Epoch  24 Step:   221800 Batch Loss:     1.184510 Tokens per Sec:    17159, Lr: 0.000103\n",
      "2019-10-30 23:08:23,869 Epoch  24 Step:   221900 Batch Loss:     1.101540 Tokens per Sec:    17009, Lr: 0.000103\n",
      "2019-10-30 23:08:38,674 Epoch  24 Step:   222000 Batch Loss:     1.276125 Tokens per Sec:    17317, Lr: 0.000103\n",
      "2019-10-30 23:09:22,708 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:09:22,708 Saving new checkpoint.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 23:09:22,848 Example #0\n",
      "2019-10-30 23:09:22,848 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:09:22,848 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:09:22,848 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:09:22,848 Example #1\n",
      "2019-10-30 23:09:22,848 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:09:22,848 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:09:22,848 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:09:22,848 Example #2\n",
      "2019-10-30 23:09:22,848 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:09:22,848 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:09:22,848 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:09:22,849 Example #3\n",
      "2019-10-30 23:09:22,849 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:09:22,849 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:09:22,849 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 23:09:22,849 Validation result at epoch  24, step   222000: bleu:  37.58, loss: 28910.6953, ppl:   2.8631, duration: 44.1740s\n",
      "2019-10-30 23:09:37,728 Epoch  24 Step:   222100 Batch Loss:     1.108926 Tokens per Sec:    17171, Lr: 0.000103\n",
      "2019-10-30 23:09:52,626 Epoch  24 Step:   222200 Batch Loss:     1.111209 Tokens per Sec:    17132, Lr: 0.000103\n",
      "2019-10-30 23:10:07,665 Epoch  24 Step:   222300 Batch Loss:     1.193402 Tokens per Sec:    17597, Lr: 0.000103\n",
      "2019-10-30 23:10:22,489 Epoch  24 Step:   222400 Batch Loss:     1.143304 Tokens per Sec:    16861, Lr: 0.000103\n",
      "2019-10-30 23:10:37,382 Epoch  24 Step:   222500 Batch Loss:     1.194979 Tokens per Sec:    17223, Lr: 0.000103\n",
      "2019-10-30 23:10:52,636 Epoch  24 Step:   222600 Batch Loss:     1.286965 Tokens per Sec:    17134, Lr: 0.000103\n",
      "2019-10-30 23:11:07,555 Epoch  24 Step:   222700 Batch Loss:     1.159641 Tokens per Sec:    16757, Lr: 0.000103\n",
      "2019-10-30 23:11:22,709 Epoch  24 Step:   222800 Batch Loss:     1.274424 Tokens per Sec:    17039, Lr: 0.000103\n",
      "2019-10-30 23:11:37,835 Epoch  24 Step:   222900 Batch Loss:     1.211628 Tokens per Sec:    17204, Lr: 0.000103\n",
      "2019-10-30 23:11:52,762 Epoch  24 Step:   223000 Batch Loss:     1.085845 Tokens per Sec:    17236, Lr: 0.000103\n",
      "2019-10-30 23:12:36,735 Example #0\n",
      "2019-10-30 23:12:36,735 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:12:36,735 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:12:36,735 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:12:36,735 Example #1\n",
      "2019-10-30 23:12:36,735 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:12:36,735 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:12:36,735 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 23:12:36,735 Example #2\n",
      "2019-10-30 23:12:36,735 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:12:36,736 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:12:36,736 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:12:36,736 Example #3\n",
      "2019-10-30 23:12:36,736 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:12:36,736 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:12:36,736 \tHypothesis: Mikono ya makumbatio na busu .\n",
      "2019-10-30 23:12:36,736 Validation result at epoch  24, step   223000: bleu:  37.18, loss: 28949.1289, ppl:   2.8671, duration: 43.9736s\n",
      "2019-10-30 23:12:51,910 Epoch  24 Step:   223100 Batch Loss:     1.338304 Tokens per Sec:    17020, Lr: 0.000103\n",
      "2019-10-30 23:13:07,118 Epoch  24 Step:   223200 Batch Loss:     1.181213 Tokens per Sec:    17127, Lr: 0.000103\n",
      "2019-10-30 23:13:21,880 Epoch  24 Step:   223300 Batch Loss:     1.292854 Tokens per Sec:    17103, Lr: 0.000103\n",
      "2019-10-30 23:13:36,677 Epoch  24 Step:   223400 Batch Loss:     1.192058 Tokens per Sec:    17021, Lr: 0.000103\n",
      "2019-10-30 23:13:51,541 Epoch  24 Step:   223500 Batch Loss:     1.030520 Tokens per Sec:    17581, Lr: 0.000103\n",
      "2019-10-30 23:14:06,433 Epoch  24 Step:   223600 Batch Loss:     1.370828 Tokens per Sec:    17591, Lr: 0.000103\n",
      "2019-10-30 23:14:21,276 Epoch  24 Step:   223700 Batch Loss:     1.210033 Tokens per Sec:    17155, Lr: 0.000103\n",
      "2019-10-30 23:14:36,289 Epoch  24 Step:   223800 Batch Loss:     1.150324 Tokens per Sec:    17043, Lr: 0.000103\n",
      "2019-10-30 23:14:51,272 Epoch  24 Step:   223900 Batch Loss:     1.039054 Tokens per Sec:    17214, Lr: 0.000103\n",
      "2019-10-30 23:15:06,301 Epoch  24 Step:   224000 Batch Loss:     1.292062 Tokens per Sec:    17025, Lr: 0.000103\n",
      "2019-10-30 23:15:50,263 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:15:50,264 Saving new checkpoint.\n",
      "2019-10-30 23:15:50,405 Example #0\n",
      "2019-10-30 23:15:50,405 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:15:50,405 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:15:50,405 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:15:50,405 Example #1\n",
      "2019-10-30 23:15:50,406 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:15:50,406 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:15:50,406 \tHypothesis: Biblia haidumii sifa ya ustahili .\n",
      "2019-10-30 23:15:50,406 Example #2\n",
      "2019-10-30 23:15:50,406 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:15:50,406 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:15:50,406 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-30 23:15:50,406 Example #3\n",
      "2019-10-30 23:15:50,406 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:15:50,406 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:15:50,406 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 23:15:50,406 Validation result at epoch  24, step   224000: bleu:  37.59, loss: 28782.7227, ppl:   2.8498, duration: 44.1045s\n",
      "2019-10-30 23:16:05,364 Epoch  24 Step:   224100 Batch Loss:     1.209168 Tokens per Sec:    17360, Lr: 0.000103\n",
      "2019-10-30 23:16:20,199 Epoch  24 Step:   224200 Batch Loss:     1.208121 Tokens per Sec:    17463, Lr: 0.000103\n",
      "2019-10-30 23:16:35,086 Epoch  24 Step:   224300 Batch Loss:     1.278272 Tokens per Sec:    17243, Lr: 0.000103\n",
      "2019-10-30 23:16:50,228 Epoch  24 Step:   224400 Batch Loss:     1.229073 Tokens per Sec:    16949, Lr: 0.000103\n",
      "2019-10-30 23:17:05,136 Epoch  24 Step:   224500 Batch Loss:     1.095677 Tokens per Sec:    17551, Lr: 0.000103\n",
      "2019-10-30 23:17:19,966 Epoch  24 Step:   224600 Batch Loss:     1.156146 Tokens per Sec:    17174, Lr: 0.000103\n",
      "2019-10-30 23:17:35,048 Epoch  24 Step:   224700 Batch Loss:     1.186276 Tokens per Sec:    16916, Lr: 0.000103\n",
      "2019-10-30 23:17:50,296 Epoch  24 Step:   224800 Batch Loss:     1.292431 Tokens per Sec:    16971, Lr: 0.000103\n",
      "2019-10-30 23:18:05,162 Epoch  24 Step:   224900 Batch Loss:     1.188254 Tokens per Sec:    16986, Lr: 0.000103\n",
      "2019-10-30 23:18:20,274 Epoch  24 Step:   225000 Batch Loss:     1.123542 Tokens per Sec:    16745, Lr: 0.000103\n",
      "2019-10-30 23:19:04,154 Example #0\n",
      "2019-10-30 23:19:04,154 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:19:04,154 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:19:04,154 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:19:04,154 Example #1\n",
      "2019-10-30 23:19:04,154 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:19:04,154 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:19:04,154 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:19:04,154 Example #2\n",
      "2019-10-30 23:19:04,154 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:19:04,154 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:19:04,154 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 23:19:04,154 Example #3\n",
      "2019-10-30 23:19:04,154 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:19:04,154 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:19:04,155 \tHypothesis: Mikono ya makumbatio na busu .\n",
      "2019-10-30 23:19:04,155 Validation result at epoch  24, step   225000: bleu:  37.49, loss: 28923.2305, ppl:   2.8644, duration: 43.8802s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 23:19:19,228 Epoch  24 Step:   225100 Batch Loss:     1.244529 Tokens per Sec:    16940, Lr: 0.000103\n",
      "2019-10-30 23:19:34,174 Epoch  24 Step:   225200 Batch Loss:     1.376143 Tokens per Sec:    16980, Lr: 0.000103\n",
      "2019-10-30 23:19:37,012 Epoch  24: total training loss 11025.04\n",
      "2019-10-30 23:19:37,013 EPOCH 25\n",
      "2019-10-30 23:19:50,310 Epoch  25 Step:   225300 Batch Loss:     1.162172 Tokens per Sec:    15710, Lr: 0.000103\n",
      "2019-10-30 23:20:05,342 Epoch  25 Step:   225400 Batch Loss:     1.242838 Tokens per Sec:    17287, Lr: 0.000103\n",
      "2019-10-30 23:20:20,458 Epoch  25 Step:   225500 Batch Loss:     1.077971 Tokens per Sec:    17293, Lr: 0.000103\n",
      "2019-10-30 23:20:35,515 Epoch  25 Step:   225600 Batch Loss:     1.109849 Tokens per Sec:    17076, Lr: 0.000103\n",
      "2019-10-30 23:20:50,355 Epoch  25 Step:   225700 Batch Loss:     1.132669 Tokens per Sec:    17215, Lr: 0.000103\n",
      "2019-10-30 23:21:05,370 Epoch  25 Step:   225800 Batch Loss:     1.202077 Tokens per Sec:    17069, Lr: 0.000103\n",
      "2019-10-30 23:21:20,306 Epoch  25 Step:   225900 Batch Loss:     1.251729 Tokens per Sec:    16940, Lr: 0.000103\n",
      "2019-10-30 23:21:35,133 Epoch  25 Step:   226000 Batch Loss:     1.102649 Tokens per Sec:    17345, Lr: 0.000103\n",
      "2019-10-30 23:22:18,995 Example #0\n",
      "2019-10-30 23:22:18,995 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:22:18,995 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:22:18,995 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:22:18,995 Example #1\n",
      "2019-10-30 23:22:18,995 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:22:18,995 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:22:18,995 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:22:18,996 Example #2\n",
      "2019-10-30 23:22:18,996 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:22:18,996 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:22:18,996 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:22:18,996 Example #3\n",
      "2019-10-30 23:22:18,996 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:22:18,996 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:22:18,996 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 23:22:18,996 Validation result at epoch  25, step   226000: bleu:  37.75, loss: 28956.2246, ppl:   2.8679, duration: 43.8620s\n",
      "2019-10-30 23:22:33,848 Epoch  25 Step:   226100 Batch Loss:     1.146886 Tokens per Sec:    17303, Lr: 0.000103\n",
      "2019-10-30 23:22:48,899 Epoch  25 Step:   226200 Batch Loss:     1.099133 Tokens per Sec:    17300, Lr: 0.000103\n",
      "2019-10-30 23:23:03,967 Epoch  25 Step:   226300 Batch Loss:     1.131022 Tokens per Sec:    17248, Lr: 0.000103\n",
      "2019-10-30 23:23:19,028 Epoch  25 Step:   226400 Batch Loss:     1.230645 Tokens per Sec:    16961, Lr: 0.000103\n",
      "2019-10-30 23:23:34,149 Epoch  25 Step:   226500 Batch Loss:     1.243130 Tokens per Sec:    17105, Lr: 0.000103\n",
      "2019-10-30 23:23:49,166 Epoch  25 Step:   226600 Batch Loss:     1.146327 Tokens per Sec:    17184, Lr: 0.000103\n",
      "2019-10-30 23:24:04,170 Epoch  25 Step:   226700 Batch Loss:     1.255164 Tokens per Sec:    17040, Lr: 0.000103\n",
      "2019-10-30 23:24:19,103 Epoch  25 Step:   226800 Batch Loss:     1.191854 Tokens per Sec:    17163, Lr: 0.000103\n",
      "2019-10-30 23:24:33,825 Epoch  25 Step:   226900 Batch Loss:     1.298599 Tokens per Sec:    16844, Lr: 0.000103\n",
      "2019-10-30 23:24:48,901 Epoch  25 Step:   227000 Batch Loss:     1.181426 Tokens per Sec:    17325, Lr: 0.000103\n",
      "2019-10-30 23:25:32,964 Example #0\n",
      "2019-10-30 23:25:32,964 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:25:32,964 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:25:32,964 \tHypothesis: Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:25:32,964 Example #1\n",
      "2019-10-30 23:25:32,964 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:25:32,964 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:25:32,964 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:25:32,965 Example #2\n",
      "2019-10-30 23:25:32,965 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:25:32,965 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:25:32,965 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-30 23:25:32,965 Example #3\n",
      "2019-10-30 23:25:32,965 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:25:32,965 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:25:32,965 \tHypothesis: Mikono ya makumbatio na busu .\n",
      "2019-10-30 23:25:32,965 Validation result at epoch  25, step   227000: bleu:  37.44, loss: 28897.1895, ppl:   2.8617, duration: 44.0638s\n",
      "2019-10-30 23:25:47,965 Epoch  25 Step:   227100 Batch Loss:     1.029019 Tokens per Sec:    17327, Lr: 0.000103\n",
      "2019-10-30 23:26:03,008 Epoch  25 Step:   227200 Batch Loss:     1.077697 Tokens per Sec:    16958, Lr: 0.000103\n",
      "2019-10-30 23:26:18,208 Epoch  25 Step:   227300 Batch Loss:     1.212331 Tokens per Sec:    17134, Lr: 0.000103\n",
      "2019-10-30 23:26:33,343 Epoch  25 Step:   227400 Batch Loss:     1.140531 Tokens per Sec:    16968, Lr: 0.000103\n",
      "2019-10-30 23:26:48,129 Epoch  25 Step:   227500 Batch Loss:     1.070432 Tokens per Sec:    17093, Lr: 0.000103\n",
      "2019-10-30 23:27:03,162 Epoch  25 Step:   227600 Batch Loss:     1.043882 Tokens per Sec:    17043, Lr: 0.000103\n",
      "2019-10-30 23:27:17,997 Epoch  25 Step:   227700 Batch Loss:     1.315996 Tokens per Sec:    17135, Lr: 0.000103\n",
      "2019-10-30 23:27:32,759 Epoch  25 Step:   227800 Batch Loss:     1.215914 Tokens per Sec:    16856, Lr: 0.000103\n",
      "2019-10-30 23:27:47,691 Epoch  25 Step:   227900 Batch Loss:     1.068574 Tokens per Sec:    17347, Lr: 0.000103\n",
      "2019-10-30 23:28:02,651 Epoch  25 Step:   228000 Batch Loss:     1.141242 Tokens per Sec:    17201, Lr: 0.000103\n",
      "2019-10-30 23:28:46,888 Example #0\n",
      "2019-10-30 23:28:46,888 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:28:46,888 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:28:46,888 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:28:46,888 Example #1\n",
      "2019-10-30 23:28:46,888 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:28:46,888 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:28:46,888 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:28:46,888 Example #2\n",
      "2019-10-30 23:28:46,888 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:28:46,889 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:28:46,889 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-30 23:28:46,889 Example #3\n",
      "2019-10-30 23:28:46,889 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:28:46,889 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:28:46,889 \tHypothesis: Mikono ya makumbatio na busu .\n",
      "2019-10-30 23:28:46,889 Validation result at epoch  25, step   228000: bleu:  37.49, loss: 28834.1660, ppl:   2.8552, duration: 44.2376s\n",
      "2019-10-30 23:29:01,952 Epoch  25 Step:   228100 Batch Loss:     1.180339 Tokens per Sec:    16986, Lr: 0.000103\n",
      "2019-10-30 23:29:16,820 Epoch  25 Step:   228200 Batch Loss:     1.137846 Tokens per Sec:    17314, Lr: 0.000103\n",
      "2019-10-30 23:29:31,617 Epoch  25 Step:   228300 Batch Loss:     1.159522 Tokens per Sec:    17400, Lr: 0.000103\n",
      "2019-10-30 23:29:46,548 Epoch  25 Step:   228400 Batch Loss:     1.081443 Tokens per Sec:    17142, Lr: 0.000103\n",
      "2019-10-30 23:30:01,450 Epoch  25 Step:   228500 Batch Loss:     1.060598 Tokens per Sec:    17273, Lr: 0.000103\n",
      "2019-10-30 23:30:16,454 Epoch  25 Step:   228600 Batch Loss:     1.303155 Tokens per Sec:    17108, Lr: 0.000103\n",
      "2019-10-30 23:30:31,266 Epoch  25 Step:   228700 Batch Loss:     1.167198 Tokens per Sec:    16990, Lr: 0.000103\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 23:30:46,143 Epoch  25 Step:   228800 Batch Loss:     1.184487 Tokens per Sec:    17278, Lr: 0.000103\n",
      "2019-10-30 23:31:00,892 Epoch  25 Step:   228900 Batch Loss:     1.240280 Tokens per Sec:    17396, Lr: 0.000103\n",
      "2019-10-30 23:31:15,924 Epoch  25 Step:   229000 Batch Loss:     1.140849 Tokens per Sec:    17230, Lr: 0.000103\n",
      "2019-10-30 23:31:59,911 Example #0\n",
      "2019-10-30 23:31:59,911 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:31:59,911 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:31:59,911 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:31:59,911 Example #1\n",
      "2019-10-30 23:31:59,911 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:31:59,911 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:31:59,911 \tHypothesis: Biblia haitegemei sifa ya ustahili .\n",
      "2019-10-30 23:31:59,911 Example #2\n",
      "2019-10-30 23:31:59,911 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:31:59,911 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:31:59,911 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 23:31:59,911 Example #3\n",
      "2019-10-30 23:31:59,912 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:31:59,912 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:31:59,912 \tHypothesis: Mikono ya makumbatio na busu .\n",
      "2019-10-30 23:31:59,912 Validation result at epoch  25, step   229000: bleu:  37.72, loss: 28807.1953, ppl:   2.8524, duration: 43.9867s\n",
      "2019-10-30 23:32:14,688 Epoch  25 Step:   229100 Batch Loss:     1.165408 Tokens per Sec:    16892, Lr: 0.000103\n",
      "2019-10-30 23:32:29,360 Epoch  25 Step:   229200 Batch Loss:     1.248432 Tokens per Sec:    17179, Lr: 0.000103\n",
      "2019-10-30 23:32:44,537 Epoch  25 Step:   229300 Batch Loss:     1.164829 Tokens per Sec:    17155, Lr: 0.000103\n",
      "2019-10-30 23:32:59,419 Epoch  25 Step:   229400 Batch Loss:     1.109407 Tokens per Sec:    17105, Lr: 0.000103\n",
      "2019-10-30 23:33:14,405 Epoch  25 Step:   229500 Batch Loss:     1.038334 Tokens per Sec:    17035, Lr: 0.000103\n",
      "2019-10-30 23:33:29,326 Epoch  25 Step:   229600 Batch Loss:     1.302198 Tokens per Sec:    17266, Lr: 0.000103\n",
      "2019-10-30 23:33:44,251 Epoch  25 Step:   229700 Batch Loss:     1.302440 Tokens per Sec:    17318, Lr: 0.000103\n",
      "2019-10-30 23:33:59,153 Epoch  25 Step:   229800 Batch Loss:     1.138192 Tokens per Sec:    17098, Lr: 0.000103\n",
      "2019-10-30 23:34:14,084 Epoch  25 Step:   229900 Batch Loss:     1.162263 Tokens per Sec:    17119, Lr: 0.000103\n",
      "2019-10-30 23:34:28,979 Epoch  25 Step:   230000 Batch Loss:     1.252321 Tokens per Sec:    17085, Lr: 0.000103\n",
      "2019-10-30 23:35:12,914 Example #0\n",
      "2019-10-30 23:35:12,914 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:35:12,914 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:35:12,914 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:35:12,914 Example #1\n",
      "2019-10-30 23:35:12,914 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:35:12,915 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:35:12,915 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:35:12,915 Example #2\n",
      "2019-10-30 23:35:12,915 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:35:12,915 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:35:12,915 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 23:35:12,915 Example #3\n",
      "2019-10-30 23:35:12,915 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:35:12,915 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:35:12,915 \tHypothesis: Mikono ya makumbatio na busu .\n",
      "2019-10-30 23:35:12,915 Validation result at epoch  25, step   230000: bleu:  37.66, loss: 28813.5352, ppl:   2.8530, duration: 43.9357s\n",
      "2019-10-30 23:35:27,843 Epoch  25 Step:   230100 Batch Loss:     1.251492 Tokens per Sec:    17084, Lr: 0.000072\n",
      "2019-10-30 23:35:42,857 Epoch  25 Step:   230200 Batch Loss:     1.188222 Tokens per Sec:    17196, Lr: 0.000072\n",
      "2019-10-30 23:35:57,694 Epoch  25 Step:   230300 Batch Loss:     1.184766 Tokens per Sec:    17154, Lr: 0.000072\n",
      "2019-10-30 23:36:12,726 Epoch  25 Step:   230400 Batch Loss:     1.142142 Tokens per Sec:    17415, Lr: 0.000072\n",
      "2019-10-30 23:36:27,568 Epoch  25 Step:   230500 Batch Loss:     1.102959 Tokens per Sec:    17390, Lr: 0.000072\n",
      "2019-10-30 23:36:42,663 Epoch  25 Step:   230600 Batch Loss:     1.301738 Tokens per Sec:    17098, Lr: 0.000072\n",
      "2019-10-30 23:36:57,626 Epoch  25 Step:   230700 Batch Loss:     1.223400 Tokens per Sec:    17380, Lr: 0.000072\n",
      "2019-10-30 23:37:12,715 Epoch  25 Step:   230800 Batch Loss:     1.184022 Tokens per Sec:    17532, Lr: 0.000072\n",
      "2019-10-30 23:37:27,647 Epoch  25 Step:   230900 Batch Loss:     1.133994 Tokens per Sec:    17075, Lr: 0.000072\n",
      "2019-10-30 23:37:42,813 Epoch  25 Step:   231000 Batch Loss:     1.108571 Tokens per Sec:    16974, Lr: 0.000072\n",
      "2019-10-30 23:38:26,730 Example #0\n",
      "2019-10-30 23:38:26,730 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:38:26,730 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:38:26,730 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:38:26,730 Example #1\n",
      "2019-10-30 23:38:26,730 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:38:26,730 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:38:26,730 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:38:26,730 Example #2\n",
      "2019-10-30 23:38:26,731 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:38:26,731 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:38:26,731 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:38:26,731 Example #3\n",
      "2019-10-30 23:38:26,731 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:38:26,731 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:38:26,731 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 23:38:26,731 Validation result at epoch  25, step   231000: bleu:  37.65, loss: 28809.1523, ppl:   2.8526, duration: 43.9174s\n",
      "2019-10-30 23:38:41,539 Epoch  25 Step:   231100 Batch Loss:     1.130209 Tokens per Sec:    17061, Lr: 0.000072\n",
      "2019-10-30 23:38:56,367 Epoch  25 Step:   231200 Batch Loss:     1.065454 Tokens per Sec:    17410, Lr: 0.000072\n",
      "2019-10-30 23:39:11,174 Epoch  25 Step:   231300 Batch Loss:     1.744873 Tokens per Sec:    17424, Lr: 0.000072\n",
      "2019-10-30 23:39:26,184 Epoch  25 Step:   231400 Batch Loss:     1.308987 Tokens per Sec:    17135, Lr: 0.000072\n",
      "2019-10-30 23:39:41,360 Epoch  25 Step:   231500 Batch Loss:     1.103549 Tokens per Sec:    16910, Lr: 0.000072\n",
      "2019-10-30 23:39:56,489 Epoch  25 Step:   231600 Batch Loss:     1.150401 Tokens per Sec:    17573, Lr: 0.000072\n",
      "2019-10-30 23:40:11,429 Epoch  25 Step:   231700 Batch Loss:     1.252180 Tokens per Sec:    17002, Lr: 0.000072\n",
      "2019-10-30 23:40:26,440 Epoch  25 Step:   231800 Batch Loss:     1.185528 Tokens per Sec:    17220, Lr: 0.000072\n",
      "2019-10-30 23:40:41,261 Epoch  25 Step:   231900 Batch Loss:     1.202942 Tokens per Sec:    17222, Lr: 0.000072\n",
      "2019-10-30 23:40:56,267 Epoch  25 Step:   232000 Batch Loss:     1.111093 Tokens per Sec:    17235, Lr: 0.000072\n",
      "2019-10-30 23:41:40,434 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:41:40,434 Saving new checkpoint.\n",
      "2019-10-30 23:41:40,585 Example #0\n",
      "2019-10-30 23:41:40,585 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:41:40,585 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:41:40,585 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:41:40,585 Example #1\n",
      "2019-10-30 23:41:40,585 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:41:40,586 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:41:40,586 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:41:40,586 Example #2\n",
      "2019-10-30 23:41:40,586 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:41:40,586 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:41:40,586 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kufanyaje ?\n",
      "2019-10-30 23:41:40,586 Example #3\n",
      "2019-10-30 23:41:40,586 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:41:40,586 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:41:40,586 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-30 23:41:40,586 Validation result at epoch  25, step   232000: bleu:  37.52, loss: 28742.6074, ppl:   2.8457, duration: 44.3186s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 23:41:55,660 Epoch  25 Step:   232100 Batch Loss:     0.928769 Tokens per Sec:    16970, Lr: 0.000072\n",
      "2019-10-30 23:42:10,492 Epoch  25 Step:   232200 Batch Loss:     1.231715 Tokens per Sec:    17495, Lr: 0.000072\n",
      "2019-10-30 23:42:25,266 Epoch  25 Step:   232300 Batch Loss:     1.126655 Tokens per Sec:    17318, Lr: 0.000072\n",
      "2019-10-30 23:42:40,297 Epoch  25 Step:   232400 Batch Loss:     0.992984 Tokens per Sec:    17501, Lr: 0.000072\n",
      "2019-10-30 23:42:55,034 Epoch  25 Step:   232500 Batch Loss:     1.296237 Tokens per Sec:    17250, Lr: 0.000072\n",
      "2019-10-30 23:43:09,845 Epoch  25 Step:   232600 Batch Loss:     1.454517 Tokens per Sec:    17232, Lr: 0.000072\n",
      "2019-10-30 23:43:25,046 Epoch  25 Step:   232700 Batch Loss:     1.153860 Tokens per Sec:    17313, Lr: 0.000072\n",
      "2019-10-30 23:43:39,911 Epoch  25 Step:   232800 Batch Loss:     1.171191 Tokens per Sec:    17151, Lr: 0.000072\n",
      "2019-10-30 23:43:54,882 Epoch  25 Step:   232900 Batch Loss:     1.264009 Tokens per Sec:    17191, Lr: 0.000072\n",
      "2019-10-30 23:44:09,813 Epoch  25 Step:   233000 Batch Loss:     1.154605 Tokens per Sec:    16929, Lr: 0.000072\n",
      "2019-10-30 23:44:53,883 Example #0\n",
      "2019-10-30 23:44:53,883 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:44:53,883 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:44:53,883 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:44:53,883 Example #1\n",
      "2019-10-30 23:44:53,883 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:44:53,883 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:44:53,883 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:44:53,883 Example #2\n",
      "2019-10-30 23:44:53,884 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:44:53,884 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:44:53,884 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 23:44:53,884 Example #3\n",
      "2019-10-30 23:44:53,884 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:44:53,884 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:44:53,884 \tHypothesis: Mabusu ya makumbatio na mabusu .\n",
      "2019-10-30 23:44:53,884 Validation result at epoch  25, step   233000: bleu:  37.71, loss: 28759.6699, ppl:   2.8474, duration: 44.0704s\n",
      "2019-10-30 23:45:08,743 Epoch  25 Step:   233100 Batch Loss:     1.093122 Tokens per Sec:    17183, Lr: 0.000072\n",
      "2019-10-30 23:45:23,675 Epoch  25 Step:   233200 Batch Loss:     1.070438 Tokens per Sec:    17589, Lr: 0.000072\n",
      "2019-10-30 23:45:38,698 Epoch  25 Step:   233300 Batch Loss:     1.191255 Tokens per Sec:    16881, Lr: 0.000072\n",
      "2019-10-30 23:45:54,038 Epoch  25 Step:   233400 Batch Loss:     1.095269 Tokens per Sec:    17048, Lr: 0.000072\n",
      "2019-10-30 23:46:08,913 Epoch  25 Step:   233500 Batch Loss:     1.068863 Tokens per Sec:    17138, Lr: 0.000072\n",
      "2019-10-30 23:46:23,757 Epoch  25 Step:   233600 Batch Loss:     1.171800 Tokens per Sec:    17246, Lr: 0.000072\n",
      "2019-10-30 23:46:38,731 Epoch  25 Step:   233700 Batch Loss:     1.068693 Tokens per Sec:    17137, Lr: 0.000072\n",
      "2019-10-30 23:46:53,699 Epoch  25 Step:   233800 Batch Loss:     1.061432 Tokens per Sec:    17110, Lr: 0.000072\n",
      "2019-10-30 23:47:08,490 Epoch  25 Step:   233900 Batch Loss:     1.214776 Tokens per Sec:    17251, Lr: 0.000072\n",
      "2019-10-30 23:47:23,347 Epoch  25 Step:   234000 Batch Loss:     1.156534 Tokens per Sec:    16891, Lr: 0.000072\n",
      "2019-10-30 23:48:07,156 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:48:07,156 Saving new checkpoint.\n",
      "2019-10-30 23:48:07,297 Example #0\n",
      "2019-10-30 23:48:07,297 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:48:07,297 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:48:07,297 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:48:07,297 Example #1\n",
      "2019-10-30 23:48:07,297 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:48:07,297 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:48:07,297 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:48:07,297 Example #2\n",
      "2019-10-30 23:48:07,297 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:48:07,297 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:48:07,298 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:48:07,298 Example #3\n",
      "2019-10-30 23:48:07,298 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:48:07,298 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:48:07,298 \tHypothesis: Vitu vya kukumbatia na kubusu .\n",
      "2019-10-30 23:48:07,298 Validation result at epoch  25, step   234000: bleu:  37.55, loss: 28694.1289, ppl:   2.8406, duration: 43.9503s\n",
      "2019-10-30 23:48:22,198 Epoch  25 Step:   234100 Batch Loss:     1.043693 Tokens per Sec:    17233, Lr: 0.000072\n",
      "2019-10-30 23:48:37,034 Epoch  25 Step:   234200 Batch Loss:     1.150097 Tokens per Sec:    17241, Lr: 0.000072\n",
      "2019-10-30 23:48:51,798 Epoch  25 Step:   234300 Batch Loss:     1.308511 Tokens per Sec:    16810, Lr: 0.000072\n",
      "2019-10-30 23:49:06,545 Epoch  25 Step:   234400 Batch Loss:     1.231544 Tokens per Sec:    17341, Lr: 0.000072\n",
      "2019-10-30 23:49:21,434 Epoch  25 Step:   234500 Batch Loss:     1.181180 Tokens per Sec:    17192, Lr: 0.000072\n",
      "2019-10-30 23:49:36,345 Epoch  25 Step:   234600 Batch Loss:     1.185562 Tokens per Sec:    17108, Lr: 0.000072\n",
      "2019-10-30 23:49:38,550 Epoch  25: total training loss 10983.08\n",
      "2019-10-30 23:49:38,550 EPOCH 26\n",
      "2019-10-30 23:49:52,324 Epoch  26 Step:   234700 Batch Loss:     1.029702 Tokens per Sec:    15565, Lr: 0.000072\n",
      "2019-10-30 23:50:07,184 Epoch  26 Step:   234800 Batch Loss:     1.136946 Tokens per Sec:    17284, Lr: 0.000072\n",
      "2019-10-30 23:50:22,033 Epoch  26 Step:   234900 Batch Loss:     1.123218 Tokens per Sec:    17230, Lr: 0.000072\n",
      "2019-10-30 23:50:36,833 Epoch  26 Step:   235000 Batch Loss:     1.097915 Tokens per Sec:    17547, Lr: 0.000072\n",
      "2019-10-30 23:51:20,844 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:51:20,844 Saving new checkpoint.\n",
      "2019-10-30 23:51:20,984 Example #0\n",
      "2019-10-30 23:51:20,984 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:51:20,984 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:51:20,984 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:51:20,984 Example #1\n",
      "2019-10-30 23:51:20,984 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:51:20,985 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:51:20,985 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:51:20,985 Example #2\n",
      "2019-10-30 23:51:20,985 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:51:20,985 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:51:20,985 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-30 23:51:20,985 Example #3\n",
      "2019-10-30 23:51:20,985 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:51:20,985 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:51:20,985 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-30 23:51:20,985 Validation result at epoch  26, step   235000: bleu:  37.59, loss: 28671.1777, ppl:   2.8383, duration: 44.1520s\n",
      "2019-10-30 23:51:36,036 Epoch  26 Step:   235100 Batch Loss:     1.149808 Tokens per Sec:    16989, Lr: 0.000072\n",
      "2019-10-30 23:51:51,020 Epoch  26 Step:   235200 Batch Loss:     1.170120 Tokens per Sec:    16974, Lr: 0.000072\n",
      "2019-10-30 23:52:06,052 Epoch  26 Step:   235300 Batch Loss:     1.046078 Tokens per Sec:    16695, Lr: 0.000072\n",
      "2019-10-30 23:52:21,080 Epoch  26 Step:   235400 Batch Loss:     1.140181 Tokens per Sec:    17330, Lr: 0.000072\n",
      "2019-10-30 23:52:36,058 Epoch  26 Step:   235500 Batch Loss:     1.153160 Tokens per Sec:    17325, Lr: 0.000072\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-30 23:52:51,039 Epoch  26 Step:   235600 Batch Loss:     1.065120 Tokens per Sec:    17342, Lr: 0.000072\n",
      "2019-10-30 23:53:06,146 Epoch  26 Step:   235700 Batch Loss:     1.021642 Tokens per Sec:    16741, Lr: 0.000072\n",
      "2019-10-30 23:53:20,917 Epoch  26 Step:   235800 Batch Loss:     1.140887 Tokens per Sec:    17039, Lr: 0.000072\n",
      "2019-10-30 23:53:35,911 Epoch  26 Step:   235900 Batch Loss:     1.153027 Tokens per Sec:    17450, Lr: 0.000072\n",
      "2019-10-30 23:53:50,713 Epoch  26 Step:   236000 Batch Loss:     1.065090 Tokens per Sec:    17223, Lr: 0.000072\n",
      "2019-10-30 23:54:34,940 Example #0\n",
      "2019-10-30 23:54:34,940 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:54:34,940 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:54:34,940 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:54:34,940 Example #1\n",
      "2019-10-30 23:54:34,940 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:54:34,940 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:54:34,940 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:54:34,941 Example #2\n",
      "2019-10-30 23:54:34,941 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:54:34,941 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:54:34,941 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-30 23:54:34,941 Example #3\n",
      "2019-10-30 23:54:34,941 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:54:34,941 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:54:34,941 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-30 23:54:34,941 Validation result at epoch  26, step   236000: bleu:  37.95, loss: 28682.8848, ppl:   2.8395, duration: 44.2277s\n",
      "2019-10-30 23:54:50,057 Epoch  26 Step:   236100 Batch Loss:     1.110985 Tokens per Sec:    17155, Lr: 0.000072\n",
      "2019-10-30 23:55:04,874 Epoch  26 Step:   236200 Batch Loss:     1.205312 Tokens per Sec:    17324, Lr: 0.000072\n",
      "2019-10-30 23:55:19,749 Epoch  26 Step:   236300 Batch Loss:     1.292860 Tokens per Sec:    17291, Lr: 0.000072\n",
      "2019-10-30 23:55:34,568 Epoch  26 Step:   236400 Batch Loss:     0.981824 Tokens per Sec:    17625, Lr: 0.000072\n",
      "2019-10-30 23:55:49,577 Epoch  26 Step:   236500 Batch Loss:     1.131276 Tokens per Sec:    17484, Lr: 0.000072\n",
      "2019-10-30 23:56:04,391 Epoch  26 Step:   236600 Batch Loss:     1.131294 Tokens per Sec:    16946, Lr: 0.000072\n",
      "2019-10-30 23:56:19,438 Epoch  26 Step:   236700 Batch Loss:     1.270813 Tokens per Sec:    17309, Lr: 0.000072\n",
      "2019-10-30 23:56:34,302 Epoch  26 Step:   236800 Batch Loss:     1.310431 Tokens per Sec:    17167, Lr: 0.000072\n",
      "2019-10-30 23:56:49,166 Epoch  26 Step:   236900 Batch Loss:     1.271472 Tokens per Sec:    17115, Lr: 0.000072\n",
      "2019-10-30 23:57:04,169 Epoch  26 Step:   237000 Batch Loss:     1.333572 Tokens per Sec:    17191, Lr: 0.000072\n",
      "2019-10-30 23:57:48,033 Hooray! New best validation result [ppl]!\n",
      "2019-10-30 23:57:48,033 Saving new checkpoint.\n",
      "2019-10-30 23:57:48,176 Example #0\n",
      "2019-10-30 23:57:48,176 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-30 23:57:48,176 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-30 23:57:48,176 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-30 23:57:48,176 Example #1\n",
      "2019-10-30 23:57:48,176 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-30 23:57:48,176 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-30 23:57:48,176 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-30 23:57:48,176 Example #2\n",
      "2019-10-30 23:57:48,176 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-30 23:57:48,176 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-30 23:57:48,176 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-30 23:57:48,176 Example #3\n",
      "2019-10-30 23:57:48,177 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-30 23:57:48,177 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-30 23:57:48,177 \tHypothesis: Mabusu ya makumbatio na mabusu .\n",
      "2019-10-30 23:57:48,177 Validation result at epoch  26, step   237000: bleu:  37.52, loss: 28648.0879, ppl:   2.8359, duration: 44.0073s\n",
      "2019-10-30 23:58:02,857 Epoch  26 Step:   237100 Batch Loss:     1.240912 Tokens per Sec:    17094, Lr: 0.000072\n",
      "2019-10-30 23:58:17,723 Epoch  26 Step:   237200 Batch Loss:     1.178090 Tokens per Sec:    17440, Lr: 0.000072\n",
      "2019-10-30 23:58:32,524 Epoch  26 Step:   237300 Batch Loss:     1.233021 Tokens per Sec:    17354, Lr: 0.000072\n",
      "2019-10-30 23:58:47,508 Epoch  26 Step:   237400 Batch Loss:     1.236149 Tokens per Sec:    17476, Lr: 0.000072\n",
      "2019-10-30 23:59:02,656 Epoch  26 Step:   237500 Batch Loss:     1.134192 Tokens per Sec:    16934, Lr: 0.000072\n",
      "2019-10-30 23:59:17,656 Epoch  26 Step:   237600 Batch Loss:     0.953981 Tokens per Sec:    17038, Lr: 0.000072\n",
      "2019-10-30 23:59:32,610 Epoch  26 Step:   237700 Batch Loss:     1.241992 Tokens per Sec:    17235, Lr: 0.000072\n",
      "2019-10-30 23:59:47,580 Epoch  26 Step:   237800 Batch Loss:     1.296279 Tokens per Sec:    17287, Lr: 0.000072\n",
      "2019-10-31 00:00:02,734 Epoch  26 Step:   237900 Batch Loss:     1.051884 Tokens per Sec:    17224, Lr: 0.000072\n",
      "2019-10-31 00:00:17,584 Epoch  26 Step:   238000 Batch Loss:     1.198855 Tokens per Sec:    16877, Lr: 0.000072\n",
      "2019-10-31 00:01:01,545 Example #0\n",
      "2019-10-31 00:01:01,545 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:01:01,545 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:01:01,545 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:01:01,546 Example #1\n",
      "2019-10-31 00:01:01,546 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:01:01,546 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:01:01,546 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:01:01,546 Example #2\n",
      "2019-10-31 00:01:01,546 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:01:01,546 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:01:01,546 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-31 00:01:01,546 Example #3\n",
      "2019-10-31 00:01:01,546 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:01:01,546 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:01:01,546 \tHypothesis: Mabusu na makumbatio .\n",
      "2019-10-31 00:01:01,546 Validation result at epoch  26, step   238000: bleu:  37.87, loss: 28714.4688, ppl:   2.8427, duration: 43.9619s\n",
      "2019-10-31 00:01:16,381 Epoch  26 Step:   238100 Batch Loss:     1.124848 Tokens per Sec:    17034, Lr: 0.000072\n",
      "2019-10-31 00:01:31,542 Epoch  26 Step:   238200 Batch Loss:     1.097758 Tokens per Sec:    17038, Lr: 0.000072\n",
      "2019-10-31 00:01:46,431 Epoch  26 Step:   238300 Batch Loss:     1.185127 Tokens per Sec:    16818, Lr: 0.000072\n",
      "2019-10-31 00:02:01,606 Epoch  26 Step:   238400 Batch Loss:     1.227138 Tokens per Sec:    17062, Lr: 0.000072\n",
      "2019-10-31 00:02:16,583 Epoch  26 Step:   238500 Batch Loss:     1.006776 Tokens per Sec:    17555, Lr: 0.000072\n",
      "2019-10-31 00:02:31,414 Epoch  26 Step:   238600 Batch Loss:     1.177802 Tokens per Sec:    17331, Lr: 0.000072\n",
      "2019-10-31 00:02:46,358 Epoch  26 Step:   238700 Batch Loss:     1.162763 Tokens per Sec:    16997, Lr: 0.000072\n",
      "2019-10-31 00:03:01,524 Epoch  26 Step:   238800 Batch Loss:     1.051676 Tokens per Sec:    17146, Lr: 0.000072\n",
      "2019-10-31 00:03:16,493 Epoch  26 Step:   238900 Batch Loss:     1.013886 Tokens per Sec:    17178, Lr: 0.000072\n",
      "2019-10-31 00:03:31,356 Epoch  26 Step:   239000 Batch Loss:     1.139684 Tokens per Sec:    17166, Lr: 0.000072\n",
      "2019-10-31 00:04:15,229 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 00:04:15,229 Saving new checkpoint.\n",
      "2019-10-31 00:04:15,370 Example #0\n",
      "2019-10-31 00:04:15,370 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:04:15,370 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:04:15,370 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:04:15,370 Example #1\n",
      "2019-10-31 00:04:15,370 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:04:15,370 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:04:15,370 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:04:15,370 Example #2\n",
      "2019-10-31 00:04:15,370 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:04:15,370 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:04:15,370 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:04:15,371 Example #3\n",
      "2019-10-31 00:04:15,371 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:04:15,371 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:04:15,371 \tHypothesis: Miguu ya makumbatio na mabusu .\n",
      "2019-10-31 00:04:15,371 Validation result at epoch  26, step   239000: bleu:  38.00, loss: 28606.3926, ppl:   2.8316, duration: 44.0146s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 00:04:30,416 Epoch  26 Step:   239100 Batch Loss:     1.033851 Tokens per Sec:    17213, Lr: 0.000072\n",
      "2019-10-31 00:04:45,470 Epoch  26 Step:   239200 Batch Loss:     1.031900 Tokens per Sec:    17260, Lr: 0.000072\n",
      "2019-10-31 00:05:00,336 Epoch  26 Step:   239300 Batch Loss:     1.285538 Tokens per Sec:    16907, Lr: 0.000072\n",
      "2019-10-31 00:05:15,550 Epoch  26 Step:   239400 Batch Loss:     1.231579 Tokens per Sec:    17405, Lr: 0.000072\n",
      "2019-10-31 00:05:30,496 Epoch  26 Step:   239500 Batch Loss:     1.116913 Tokens per Sec:    17105, Lr: 0.000072\n",
      "2019-10-31 00:05:45,628 Epoch  26 Step:   239600 Batch Loss:     1.117622 Tokens per Sec:    17628, Lr: 0.000072\n",
      "2019-10-31 00:06:00,806 Epoch  26 Step:   239700 Batch Loss:     1.099376 Tokens per Sec:    17178, Lr: 0.000072\n",
      "2019-10-31 00:06:15,914 Epoch  26 Step:   239800 Batch Loss:     1.218608 Tokens per Sec:    16852, Lr: 0.000072\n",
      "2019-10-31 00:06:30,908 Epoch  26 Step:   239900 Batch Loss:     1.292508 Tokens per Sec:    17061, Lr: 0.000072\n",
      "2019-10-31 00:06:45,939 Epoch  26 Step:   240000 Batch Loss:     1.064610 Tokens per Sec:    17512, Lr: 0.000072\n",
      "2019-10-31 00:07:29,823 Example #0\n",
      "2019-10-31 00:07:29,823 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:07:29,823 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:07:29,823 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:07:29,823 Example #1\n",
      "2019-10-31 00:07:29,824 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:07:29,824 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:07:29,824 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:07:29,824 Example #2\n",
      "2019-10-31 00:07:29,824 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:07:29,824 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:07:29,824 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-31 00:07:29,824 Example #3\n",
      "2019-10-31 00:07:29,824 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:07:29,824 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:07:29,824 \tHypothesis: Vitu vya kukumbatia na kubusu .\n",
      "2019-10-31 00:07:29,824 Validation result at epoch  26, step   240000: bleu:  37.77, loss: 28655.8184, ppl:   2.8367, duration: 43.8848s\n",
      "2019-10-31 00:07:44,915 Epoch  26 Step:   240100 Batch Loss:     1.335562 Tokens per Sec:    16938, Lr: 0.000072\n",
      "2019-10-31 00:08:00,065 Epoch  26 Step:   240200 Batch Loss:     0.995848 Tokens per Sec:    17114, Lr: 0.000072\n",
      "2019-10-31 00:08:15,035 Epoch  26 Step:   240300 Batch Loss:     1.120932 Tokens per Sec:    16990, Lr: 0.000072\n",
      "2019-10-31 00:08:30,002 Epoch  26 Step:   240400 Batch Loss:     1.099894 Tokens per Sec:    16964, Lr: 0.000072\n",
      "2019-10-31 00:08:45,071 Epoch  26 Step:   240500 Batch Loss:     1.037611 Tokens per Sec:    17220, Lr: 0.000072\n",
      "2019-10-31 00:09:00,116 Epoch  26 Step:   240600 Batch Loss:     1.131181 Tokens per Sec:    17346, Lr: 0.000072\n",
      "2019-10-31 00:09:14,947 Epoch  26 Step:   240700 Batch Loss:     1.086585 Tokens per Sec:    17441, Lr: 0.000072\n",
      "2019-10-31 00:09:29,711 Epoch  26 Step:   240800 Batch Loss:     1.220887 Tokens per Sec:    17205, Lr: 0.000072\n",
      "2019-10-31 00:09:45,028 Epoch  26 Step:   240900 Batch Loss:     1.273428 Tokens per Sec:    17083, Lr: 0.000072\n",
      "2019-10-31 00:10:00,115 Epoch  26 Step:   241000 Batch Loss:     1.192528 Tokens per Sec:    17207, Lr: 0.000072\n",
      "2019-10-31 00:10:43,935 Example #0\n",
      "2019-10-31 00:10:43,936 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:10:43,936 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:10:43,936 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:10:43,936 Example #1\n",
      "2019-10-31 00:10:43,936 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:10:43,936 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:10:43,936 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:10:43,936 Example #2\n",
      "2019-10-31 00:10:43,936 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:10:43,936 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:10:43,936 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 00:10:43,936 Example #3\n",
      "2019-10-31 00:10:43,936 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:10:43,936 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:10:43,936 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:10:43,937 Validation result at epoch  26, step   241000: bleu:  37.76, loss: 28657.2031, ppl:   2.8368, duration: 43.8211s\n",
      "2019-10-31 00:10:58,849 Epoch  26 Step:   241100 Batch Loss:     1.180908 Tokens per Sec:    17106, Lr: 0.000072\n",
      "2019-10-31 00:11:13,913 Epoch  26 Step:   241200 Batch Loss:     1.122508 Tokens per Sec:    17103, Lr: 0.000072\n",
      "2019-10-31 00:11:29,145 Epoch  26 Step:   241300 Batch Loss:     1.298967 Tokens per Sec:    17157, Lr: 0.000072\n",
      "2019-10-31 00:11:44,369 Epoch  26 Step:   241400 Batch Loss:     1.124199 Tokens per Sec:    17264, Lr: 0.000072\n",
      "2019-10-31 00:11:59,133 Epoch  26 Step:   241500 Batch Loss:     1.184859 Tokens per Sec:    16835, Lr: 0.000072\n",
      "2019-10-31 00:12:14,080 Epoch  26 Step:   241600 Batch Loss:     1.165008 Tokens per Sec:    17101, Lr: 0.000072\n",
      "2019-10-31 00:12:28,829 Epoch  26 Step:   241700 Batch Loss:     1.430762 Tokens per Sec:    17199, Lr: 0.000072\n",
      "2019-10-31 00:12:43,758 Epoch  26 Step:   241800 Batch Loss:     1.254050 Tokens per Sec:    17276, Lr: 0.000072\n",
      "2019-10-31 00:12:58,572 Epoch  26 Step:   241900 Batch Loss:     1.336314 Tokens per Sec:    16875, Lr: 0.000072\n",
      "2019-10-31 00:13:13,389 Epoch  26 Step:   242000 Batch Loss:     1.196184 Tokens per Sec:    17035, Lr: 0.000072\n",
      "2019-10-31 00:13:57,357 Example #0\n",
      "2019-10-31 00:13:57,357 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:13:57,357 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:13:57,357 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:13:57,357 Example #1\n",
      "2019-10-31 00:13:57,357 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:13:57,357 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:13:57,357 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:13:57,357 Example #2\n",
      "2019-10-31 00:13:57,357 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:13:57,357 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:13:57,358 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 00:13:57,358 Example #3\n",
      "2019-10-31 00:13:57,358 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:13:57,358 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:13:57,358 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:13:57,358 Validation result at epoch  26, step   242000: bleu:  37.86, loss: 28689.3770, ppl:   2.8402, duration: 43.9685s\n",
      "2019-10-31 00:14:12,200 Epoch  26 Step:   242100 Batch Loss:     1.221675 Tokens per Sec:    17162, Lr: 0.000072\n",
      "2019-10-31 00:14:26,903 Epoch  26 Step:   242200 Batch Loss:     1.139287 Tokens per Sec:    17408, Lr: 0.000072\n",
      "2019-10-31 00:14:41,885 Epoch  26 Step:   242300 Batch Loss:     1.203787 Tokens per Sec:    17281, Lr: 0.000072\n",
      "2019-10-31 00:14:56,928 Epoch  26 Step:   242400 Batch Loss:     1.196503 Tokens per Sec:    17198, Lr: 0.000072\n",
      "2019-10-31 00:15:11,878 Epoch  26 Step:   242500 Batch Loss:     1.132742 Tokens per Sec:    16773, Lr: 0.000072\n",
      "2019-10-31 00:15:26,826 Epoch  26 Step:   242600 Batch Loss:     1.034801 Tokens per Sec:    17541, Lr: 0.000072\n",
      "2019-10-31 00:15:41,502 Epoch  26 Step:   242700 Batch Loss:     1.062282 Tokens per Sec:    17481, Lr: 0.000072\n",
      "2019-10-31 00:15:56,360 Epoch  26 Step:   242800 Batch Loss:     1.315359 Tokens per Sec:    17249, Lr: 0.000072\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 00:16:11,470 Epoch  26 Step:   242900 Batch Loss:     1.212824 Tokens per Sec:    17058, Lr: 0.000072\n",
      "2019-10-31 00:16:26,521 Epoch  26 Step:   243000 Batch Loss:     1.183514 Tokens per Sec:    17199, Lr: 0.000072\n",
      "2019-10-31 00:17:10,324 Example #0\n",
      "2019-10-31 00:17:10,324 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:17:10,324 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:17:10,324 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:17:10,324 Example #1\n",
      "2019-10-31 00:17:10,324 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:17:10,324 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:17:10,324 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:17:10,324 Example #2\n",
      "2019-10-31 00:17:10,325 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:17:10,325 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:17:10,325 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:17:10,325 Example #3\n",
      "2019-10-31 00:17:10,325 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:17:10,325 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:17:10,325 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:17:10,325 Validation result at epoch  26, step   243000: bleu:  37.63, loss: 28619.0547, ppl:   2.8329, duration: 43.8035s\n",
      "2019-10-31 00:17:25,271 Epoch  26 Step:   243100 Batch Loss:     1.193416 Tokens per Sec:    17064, Lr: 0.000072\n",
      "2019-10-31 00:17:40,112 Epoch  26 Step:   243200 Batch Loss:     1.142869 Tokens per Sec:    17495, Lr: 0.000072\n",
      "2019-10-31 00:17:54,737 Epoch  26 Step:   243300 Batch Loss:     1.145380 Tokens per Sec:    17063, Lr: 0.000072\n",
      "2019-10-31 00:18:09,752 Epoch  26 Step:   243400 Batch Loss:     1.211417 Tokens per Sec:    17420, Lr: 0.000072\n",
      "2019-10-31 00:18:24,675 Epoch  26 Step:   243500 Batch Loss:     1.004508 Tokens per Sec:    16825, Lr: 0.000072\n",
      "2019-10-31 00:18:39,639 Epoch  26 Step:   243600 Batch Loss:     1.123438 Tokens per Sec:    17031, Lr: 0.000072\n",
      "2019-10-31 00:18:54,714 Epoch  26 Step:   243700 Batch Loss:     1.215536 Tokens per Sec:    17304, Lr: 0.000072\n",
      "2019-10-31 00:19:09,418 Epoch  26 Step:   243800 Batch Loss:     1.287508 Tokens per Sec:    17029, Lr: 0.000072\n",
      "2019-10-31 00:19:24,370 Epoch  26 Step:   243900 Batch Loss:     1.055588 Tokens per Sec:    17524, Lr: 0.000072\n",
      "2019-10-31 00:19:39,177 Epoch  26: total training loss 10899.70\n",
      "2019-10-31 00:19:39,178 EPOCH 27\n",
      "2019-10-31 00:19:40,518 Epoch  27 Step:   244000 Batch Loss:     1.147207 Tokens per Sec:     2227, Lr: 0.000072\n",
      "2019-10-31 00:20:24,440 Example #0\n",
      "2019-10-31 00:20:24,440 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:20:24,440 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:20:24,440 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:20:24,440 Example #1\n",
      "2019-10-31 00:20:24,440 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:20:24,440 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:20:24,440 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:20:24,440 Example #2\n",
      "2019-10-31 00:20:24,440 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:20:24,440 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:20:24,440 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 00:20:24,441 Example #3\n",
      "2019-10-31 00:20:24,441 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:20:24,441 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:20:24,441 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:20:24,441 Validation result at epoch  27, step   244000: bleu:  37.56, loss: 28617.8203, ppl:   2.8328, duration: 43.9221s\n",
      "2019-10-31 00:20:39,594 Epoch  27 Step:   244100 Batch Loss:     1.084919 Tokens per Sec:    17244, Lr: 0.000072\n",
      "2019-10-31 00:20:54,651 Epoch  27 Step:   244200 Batch Loss:     1.065566 Tokens per Sec:    17325, Lr: 0.000072\n",
      "2019-10-31 00:21:09,470 Epoch  27 Step:   244300 Batch Loss:     1.083605 Tokens per Sec:    17392, Lr: 0.000072\n",
      "2019-10-31 00:21:24,314 Epoch  27 Step:   244400 Batch Loss:     1.171448 Tokens per Sec:    17389, Lr: 0.000072\n",
      "2019-10-31 00:21:39,659 Epoch  27 Step:   244500 Batch Loss:     1.066575 Tokens per Sec:    17344, Lr: 0.000072\n",
      "2019-10-31 00:21:54,800 Epoch  27 Step:   244600 Batch Loss:     1.051282 Tokens per Sec:    17130, Lr: 0.000072\n",
      "2019-10-31 00:22:09,658 Epoch  27 Step:   244700 Batch Loss:     1.205762 Tokens per Sec:    16818, Lr: 0.000072\n",
      "2019-10-31 00:22:24,733 Epoch  27 Step:   244800 Batch Loss:     1.157849 Tokens per Sec:    17137, Lr: 0.000072\n",
      "2019-10-31 00:22:39,670 Epoch  27 Step:   244900 Batch Loss:     1.030527 Tokens per Sec:    17215, Lr: 0.000072\n",
      "2019-10-31 00:22:54,548 Epoch  27 Step:   245000 Batch Loss:     1.145550 Tokens per Sec:    17437, Lr: 0.000072\n",
      "2019-10-31 00:23:38,331 Example #0\n",
      "2019-10-31 00:23:38,331 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:23:38,331 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:23:38,331 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:23:38,331 Example #1\n",
      "2019-10-31 00:23:38,331 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:23:38,331 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:23:38,331 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:23:38,331 Example #2\n",
      "2019-10-31 00:23:38,332 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:23:38,332 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:23:38,332 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 00:23:38,332 Example #3\n",
      "2019-10-31 00:23:38,332 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:23:38,332 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:23:38,332 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:23:38,332 Validation result at epoch  27, step   245000: bleu:  37.99, loss: 28618.3945, ppl:   2.8328, duration: 43.7839s\n",
      "2019-10-31 00:23:53,033 Epoch  27 Step:   245100 Batch Loss:     1.085828 Tokens per Sec:    17312, Lr: 0.000050\n",
      "2019-10-31 00:24:07,918 Epoch  27 Step:   245200 Batch Loss:     1.233145 Tokens per Sec:    17200, Lr: 0.000050\n",
      "2019-10-31 00:24:22,761 Epoch  27 Step:   245300 Batch Loss:     1.192829 Tokens per Sec:    17296, Lr: 0.000050\n",
      "2019-10-31 00:24:37,620 Epoch  27 Step:   245400 Batch Loss:     1.237584 Tokens per Sec:    17483, Lr: 0.000050\n",
      "2019-10-31 00:24:52,184 Epoch  27 Step:   245500 Batch Loss:     1.221200 Tokens per Sec:    17150, Lr: 0.000050\n",
      "2019-10-31 00:25:07,311 Epoch  27 Step:   245600 Batch Loss:     1.078106 Tokens per Sec:    17246, Lr: 0.000050\n",
      "2019-10-31 00:25:22,242 Epoch  27 Step:   245700 Batch Loss:     1.332248 Tokens per Sec:    17140, Lr: 0.000050\n",
      "2019-10-31 00:25:37,387 Epoch  27 Step:   245800 Batch Loss:     1.208727 Tokens per Sec:    17101, Lr: 0.000050\n",
      "2019-10-31 00:25:52,349 Epoch  27 Step:   245900 Batch Loss:     1.192972 Tokens per Sec:    16981, Lr: 0.000050\n",
      "2019-10-31 00:26:07,545 Epoch  27 Step:   246000 Batch Loss:     1.049751 Tokens per Sec:    17185, Lr: 0.000050\n",
      "2019-10-31 00:26:51,521 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 00:26:51,521 Saving new checkpoint.\n",
      "2019-10-31 00:26:51,704 Example #0\n",
      "2019-10-31 00:26:51,704 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:26:51,704 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:26:51,705 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:26:51,705 Example #1\n",
      "2019-10-31 00:26:51,705 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:26:51,705 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:26:51,705 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:26:51,705 Example #2\n",
      "2019-10-31 00:26:51,705 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:26:51,705 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:26:51,705 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 00:26:51,705 Example #3\n",
      "2019-10-31 00:26:51,705 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:26:51,705 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:26:51,705 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 00:26:51,705 Validation result at epoch  27, step   246000: bleu:  37.68, loss: 28587.6289, ppl:   2.8297, duration: 44.1601s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 00:27:06,803 Epoch  27 Step:   246100 Batch Loss:     1.211522 Tokens per Sec:    17377, Lr: 0.000050\n",
      "2019-10-31 00:27:21,658 Epoch  27 Step:   246200 Batch Loss:     1.101580 Tokens per Sec:    16819, Lr: 0.000050\n",
      "2019-10-31 00:27:36,765 Epoch  27 Step:   246300 Batch Loss:     1.388117 Tokens per Sec:    17060, Lr: 0.000050\n",
      "2019-10-31 00:27:51,841 Epoch  27 Step:   246400 Batch Loss:     1.183427 Tokens per Sec:    17068, Lr: 0.000050\n",
      "2019-10-31 00:28:07,127 Epoch  27 Step:   246500 Batch Loss:     1.203829 Tokens per Sec:    17102, Lr: 0.000050\n",
      "2019-10-31 00:28:22,090 Epoch  27 Step:   246600 Batch Loss:     1.256560 Tokens per Sec:    17150, Lr: 0.000050\n",
      "2019-10-31 00:28:37,208 Epoch  27 Step:   246700 Batch Loss:     1.145541 Tokens per Sec:    17234, Lr: 0.000050\n",
      "2019-10-31 00:28:52,206 Epoch  27 Step:   246800 Batch Loss:     1.077309 Tokens per Sec:    17201, Lr: 0.000050\n",
      "2019-10-31 00:29:07,344 Epoch  27 Step:   246900 Batch Loss:     1.070156 Tokens per Sec:    17486, Lr: 0.000050\n",
      "2019-10-31 00:29:22,083 Epoch  27 Step:   247000 Batch Loss:     1.153566 Tokens per Sec:    17035, Lr: 0.000050\n",
      "2019-10-31 00:30:05,822 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 00:30:05,823 Saving new checkpoint.\n",
      "2019-10-31 00:30:05,963 Example #0\n",
      "2019-10-31 00:30:05,963 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:30:05,963 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:30:05,963 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:30:05,963 Example #1\n",
      "2019-10-31 00:30:05,963 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:30:05,963 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:30:05,963 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:30:05,963 Example #2\n",
      "2019-10-31 00:30:05,963 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:30:05,963 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:30:05,963 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:30:05,963 Example #3\n",
      "2019-10-31 00:30:05,964 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:30:05,964 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:30:05,964 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:30:05,964 Validation result at epoch  27, step   247000: bleu:  37.79, loss: 28585.2852, ppl:   2.8294, duration: 43.8801s\n",
      "2019-10-31 00:30:20,910 Epoch  27 Step:   247100 Batch Loss:     1.224441 Tokens per Sec:    17380, Lr: 0.000050\n",
      "2019-10-31 00:30:35,899 Epoch  27 Step:   247200 Batch Loss:     1.252130 Tokens per Sec:    17104, Lr: 0.000050\n",
      "2019-10-31 00:30:50,887 Epoch  27 Step:   247300 Batch Loss:     1.030315 Tokens per Sec:    17218, Lr: 0.000050\n",
      "2019-10-31 00:31:06,047 Epoch  27 Step:   247400 Batch Loss:     1.126136 Tokens per Sec:    17086, Lr: 0.000050\n",
      "2019-10-31 00:31:21,081 Epoch  27 Step:   247500 Batch Loss:     1.165495 Tokens per Sec:    17010, Lr: 0.000050\n",
      "2019-10-31 00:31:36,088 Epoch  27 Step:   247600 Batch Loss:     1.109614 Tokens per Sec:    17053, Lr: 0.000050\n",
      "2019-10-31 00:31:51,115 Epoch  27 Step:   247700 Batch Loss:     1.096090 Tokens per Sec:    17248, Lr: 0.000050\n",
      "2019-10-31 00:32:05,945 Epoch  27 Step:   247800 Batch Loss:     1.095983 Tokens per Sec:    17291, Lr: 0.000050\n",
      "2019-10-31 00:32:20,787 Epoch  27 Step:   247900 Batch Loss:     1.109339 Tokens per Sec:    17248, Lr: 0.000050\n",
      "2019-10-31 00:32:35,683 Epoch  27 Step:   248000 Batch Loss:     1.180546 Tokens per Sec:    17382, Lr: 0.000050\n",
      "2019-10-31 00:33:19,452 Example #0\n",
      "2019-10-31 00:33:19,452 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:33:19,452 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:33:19,452 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:33:19,452 Example #1\n",
      "2019-10-31 00:33:19,452 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:33:19,452 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:33:19,452 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:33:19,452 Example #2\n",
      "2019-10-31 00:33:19,452 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:33:19,452 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:33:19,452 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 00:33:19,452 Example #3\n",
      "2019-10-31 00:33:19,453 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:33:19,453 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:33:19,453 \tHypothesis: Vitu vya kukumbatia na busu .\n",
      "2019-10-31 00:33:19,453 Validation result at epoch  27, step   248000: bleu:  37.83, loss: 28586.1484, ppl:   2.8295, duration: 43.7690s\n",
      "2019-10-31 00:33:34,261 Epoch  27 Step:   248100 Batch Loss:     1.154222 Tokens per Sec:    17280, Lr: 0.000050\n",
      "2019-10-31 00:33:49,025 Epoch  27 Step:   248200 Batch Loss:     1.010104 Tokens per Sec:    17198, Lr: 0.000050\n",
      "2019-10-31 00:34:03,943 Epoch  27 Step:   248300 Batch Loss:     1.022265 Tokens per Sec:    17144, Lr: 0.000050\n",
      "2019-10-31 00:34:18,729 Epoch  27 Step:   248400 Batch Loss:     1.108331 Tokens per Sec:    17022, Lr: 0.000050\n",
      "2019-10-31 00:34:33,604 Epoch  27 Step:   248500 Batch Loss:     1.108624 Tokens per Sec:    17087, Lr: 0.000050\n",
      "2019-10-31 00:34:48,631 Epoch  27 Step:   248600 Batch Loss:     1.136411 Tokens per Sec:    17065, Lr: 0.000050\n",
      "2019-10-31 00:35:03,392 Epoch  27 Step:   248700 Batch Loss:     1.232816 Tokens per Sec:    17188, Lr: 0.000050\n",
      "2019-10-31 00:35:18,255 Epoch  27 Step:   248800 Batch Loss:     1.177297 Tokens per Sec:    17312, Lr: 0.000050\n",
      "2019-10-31 00:35:32,982 Epoch  27 Step:   248900 Batch Loss:     1.162507 Tokens per Sec:    17236, Lr: 0.000050\n",
      "2019-10-31 00:35:47,976 Epoch  27 Step:   249000 Batch Loss:     1.042751 Tokens per Sec:    17076, Lr: 0.000050\n",
      "2019-10-31 00:36:31,963 Example #0\n",
      "2019-10-31 00:36:31,963 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:36:31,963 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:36:31,963 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:36:31,963 Example #1\n",
      "2019-10-31 00:36:31,963 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:36:31,963 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:36:31,963 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:36:31,963 Example #2\n",
      "2019-10-31 00:36:31,963 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:36:31,963 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:36:31,963 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 00:36:31,964 Example #3\n",
      "2019-10-31 00:36:31,964 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:36:31,964 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:36:31,964 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:36:31,964 Validation result at epoch  27, step   249000: bleu:  37.70, loss: 28608.9316, ppl:   2.8319, duration: 43.9874s\n",
      "2019-10-31 00:36:47,029 Epoch  27 Step:   249100 Batch Loss:     1.028249 Tokens per Sec:    17185, Lr: 0.000050\n",
      "2019-10-31 00:37:01,901 Epoch  27 Step:   249200 Batch Loss:     1.144421 Tokens per Sec:    17002, Lr: 0.000050\n",
      "2019-10-31 00:37:16,624 Epoch  27 Step:   249300 Batch Loss:     1.196336 Tokens per Sec:    17075, Lr: 0.000050\n",
      "2019-10-31 00:37:31,659 Epoch  27 Step:   249400 Batch Loss:     1.149417 Tokens per Sec:    17353, Lr: 0.000050\n",
      "2019-10-31 00:37:46,675 Epoch  27 Step:   249500 Batch Loss:     1.276701 Tokens per Sec:    17196, Lr: 0.000050\n",
      "2019-10-31 00:38:01,533 Epoch  27 Step:   249600 Batch Loss:     1.036114 Tokens per Sec:    17263, Lr: 0.000050\n",
      "2019-10-31 00:38:16,561 Epoch  27 Step:   249700 Batch Loss:     1.110699 Tokens per Sec:    17083, Lr: 0.000050\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 00:38:31,460 Epoch  27 Step:   249800 Batch Loss:     1.072620 Tokens per Sec:    16940, Lr: 0.000050\n",
      "2019-10-31 00:38:46,499 Epoch  27 Step:   249900 Batch Loss:     1.109842 Tokens per Sec:    16780, Lr: 0.000050\n",
      "2019-10-31 00:39:01,481 Epoch  27 Step:   250000 Batch Loss:     1.133363 Tokens per Sec:    17235, Lr: 0.000050\n",
      "2019-10-31 00:39:45,274 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 00:39:45,274 Saving new checkpoint.\n",
      "2019-10-31 00:39:45,416 Example #0\n",
      "2019-10-31 00:39:45,417 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:39:45,417 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:39:45,417 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:39:45,417 Example #1\n",
      "2019-10-31 00:39:45,417 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:39:45,417 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:39:45,417 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:39:45,417 Example #2\n",
      "2019-10-31 00:39:45,417 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:39:45,417 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:39:45,417 \tHypothesis: Ni mambo gani yanayosababisha wasiwasi , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:39:45,417 Example #3\n",
      "2019-10-31 00:39:45,417 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:39:45,417 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:39:45,417 \tHypothesis: Mabusu na mashavu .\n",
      "2019-10-31 00:39:45,418 Validation result at epoch  27, step   250000: bleu:  37.62, loss: 28513.5234, ppl:   2.8220, duration: 43.9360s\n",
      "2019-10-31 00:40:00,303 Epoch  27 Step:   250100 Batch Loss:     1.180445 Tokens per Sec:    16970, Lr: 0.000050\n",
      "2019-10-31 00:40:15,242 Epoch  27 Step:   250200 Batch Loss:     0.978602 Tokens per Sec:    17362, Lr: 0.000050\n",
      "2019-10-31 00:40:30,154 Epoch  27 Step:   250300 Batch Loss:     1.029252 Tokens per Sec:    17112, Lr: 0.000050\n",
      "2019-10-31 00:40:45,021 Epoch  27 Step:   250400 Batch Loss:     1.135223 Tokens per Sec:    17268, Lr: 0.000050\n",
      "2019-10-31 00:40:59,798 Epoch  27 Step:   250500 Batch Loss:     1.148565 Tokens per Sec:    17182, Lr: 0.000050\n",
      "2019-10-31 00:41:14,684 Epoch  27 Step:   250600 Batch Loss:     1.139532 Tokens per Sec:    17473, Lr: 0.000050\n",
      "2019-10-31 00:41:29,439 Epoch  27 Step:   250700 Batch Loss:     1.312482 Tokens per Sec:    17590, Lr: 0.000050\n",
      "2019-10-31 00:41:44,394 Epoch  27 Step:   250800 Batch Loss:     1.295541 Tokens per Sec:    17235, Lr: 0.000050\n",
      "2019-10-31 00:41:59,500 Epoch  27 Step:   250900 Batch Loss:     1.105141 Tokens per Sec:    17408, Lr: 0.000050\n",
      "2019-10-31 00:42:14,624 Epoch  27 Step:   251000 Batch Loss:     1.152834 Tokens per Sec:    17143, Lr: 0.000050\n",
      "2019-10-31 00:42:58,864 Example #0\n",
      "2019-10-31 00:42:58,864 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:42:58,864 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:42:58,864 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:42:58,864 Example #1\n",
      "2019-10-31 00:42:58,864 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:42:58,864 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:42:58,864 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:42:58,865 Example #2\n",
      "2019-10-31 00:42:58,865 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:42:58,865 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:42:58,865 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:42:58,865 Example #3\n",
      "2019-10-31 00:42:58,865 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:42:58,865 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:42:58,865 \tHypothesis: Mabusu na mashavu .\n",
      "2019-10-31 00:42:58,865 Validation result at epoch  27, step   251000: bleu:  37.95, loss: 28528.0996, ppl:   2.8235, duration: 44.2410s\n",
      "2019-10-31 00:43:13,768 Epoch  27 Step:   251100 Batch Loss:     1.255761 Tokens per Sec:    17054, Lr: 0.000050\n",
      "2019-10-31 00:43:28,695 Epoch  27 Step:   251200 Batch Loss:     1.114572 Tokens per Sec:    17247, Lr: 0.000050\n",
      "2019-10-31 00:43:43,546 Epoch  27 Step:   251300 Batch Loss:     1.150222 Tokens per Sec:    16716, Lr: 0.000050\n",
      "2019-10-31 00:43:58,618 Epoch  27 Step:   251400 Batch Loss:     1.219812 Tokens per Sec:    17402, Lr: 0.000050\n",
      "2019-10-31 00:44:13,595 Epoch  27 Step:   251500 Batch Loss:     1.276964 Tokens per Sec:    17064, Lr: 0.000050\n",
      "2019-10-31 00:44:28,309 Epoch  27 Step:   251600 Batch Loss:     1.284030 Tokens per Sec:    17286, Lr: 0.000050\n",
      "2019-10-31 00:44:43,338 Epoch  27 Step:   251700 Batch Loss:     0.994881 Tokens per Sec:    17352, Lr: 0.000050\n",
      "2019-10-31 00:44:58,230 Epoch  27 Step:   251800 Batch Loss:     1.155859 Tokens per Sec:    17131, Lr: 0.000050\n",
      "2019-10-31 00:45:13,343 Epoch  27 Step:   251900 Batch Loss:     1.139223 Tokens per Sec:    16721, Lr: 0.000050\n",
      "2019-10-31 00:45:28,402 Epoch  27 Step:   252000 Batch Loss:     1.228520 Tokens per Sec:    17193, Lr: 0.000050\n",
      "2019-10-31 00:46:12,427 Example #0\n",
      "2019-10-31 00:46:12,428 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:46:12,428 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:46:12,428 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:46:12,428 Example #1\n",
      "2019-10-31 00:46:12,428 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:46:12,428 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:46:12,428 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:46:12,428 Example #2\n",
      "2019-10-31 00:46:12,428 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:46:12,428 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:46:12,428 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:46:12,428 Example #3\n",
      "2019-10-31 00:46:12,428 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:46:12,429 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:46:12,429 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 00:46:12,429 Validation result at epoch  27, step   252000: bleu:  37.63, loss: 28538.8477, ppl:   2.8246, duration: 44.0259s\n",
      "2019-10-31 00:46:27,316 Epoch  27 Step:   252100 Batch Loss:     1.146650 Tokens per Sec:    16922, Lr: 0.000050\n",
      "2019-10-31 00:46:42,232 Epoch  27 Step:   252200 Batch Loss:     1.047491 Tokens per Sec:    17458, Lr: 0.000050\n",
      "2019-10-31 00:46:57,160 Epoch  27 Step:   252300 Batch Loss:     1.229419 Tokens per Sec:    17268, Lr: 0.000050\n",
      "2019-10-31 00:47:11,832 Epoch  27 Step:   252400 Batch Loss:     0.975677 Tokens per Sec:    16980, Lr: 0.000050\n",
      "2019-10-31 00:47:26,647 Epoch  27 Step:   252500 Batch Loss:     1.134763 Tokens per Sec:    16921, Lr: 0.000050\n",
      "2019-10-31 00:47:41,502 Epoch  27 Step:   252600 Batch Loss:     1.082347 Tokens per Sec:    17432, Lr: 0.000050\n",
      "2019-10-31 00:47:56,312 Epoch  27 Step:   252700 Batch Loss:     1.305530 Tokens per Sec:    17150, Lr: 0.000050\n",
      "2019-10-31 00:48:11,196 Epoch  27 Step:   252800 Batch Loss:     1.182814 Tokens per Sec:    17124, Lr: 0.000050\n",
      "2019-10-31 00:48:26,008 Epoch  27 Step:   252900 Batch Loss:     1.030319 Tokens per Sec:    16912, Lr: 0.000050\n",
      "2019-10-31 00:48:40,936 Epoch  27 Step:   253000 Batch Loss:     0.994816 Tokens per Sec:    17364, Lr: 0.000050\n",
      "2019-10-31 00:49:24,767 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 00:49:24,767 Saving new checkpoint.\n",
      "2019-10-31 00:49:24,908 Example #0\n",
      "2019-10-31 00:49:24,908 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:49:24,908 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:49:24,908 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:49:24,908 Example #1\n",
      "2019-10-31 00:49:24,909 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:49:24,909 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:49:24,909 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:49:24,909 Example #2\n",
      "2019-10-31 00:49:24,909 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:49:24,909 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:49:24,909 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 00:49:24,909 Example #3\n",
      "2019-10-31 00:49:24,909 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:49:24,909 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:49:24,909 \tHypothesis: Mabusu na mabusu .\n",
      "2019-10-31 00:49:24,909 Validation result at epoch  27, step   253000: bleu:  38.05, loss: 28498.2070, ppl:   2.8205, duration: 43.9731s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 00:49:39,806 Epoch  27 Step:   253100 Batch Loss:     1.175210 Tokens per Sec:    17412, Lr: 0.000050\n",
      "2019-10-31 00:49:54,575 Epoch  27 Step:   253200 Batch Loss:     1.302589 Tokens per Sec:    17307, Lr: 0.000050\n",
      "2019-10-31 00:50:09,708 Epoch  27 Step:   253300 Batch Loss:     1.023007 Tokens per Sec:    17129, Lr: 0.000050\n",
      "2019-10-31 00:50:23,267 Epoch  27: total training loss 10848.33\n",
      "2019-10-31 00:50:23,267 EPOCH 28\n",
      "2019-10-31 00:50:25,860 Epoch  28 Step:   253400 Batch Loss:     1.140431 Tokens per Sec:     9679, Lr: 0.000050\n",
      "2019-10-31 00:50:40,776 Epoch  28 Step:   253500 Batch Loss:     1.138314 Tokens per Sec:    17102, Lr: 0.000050\n",
      "2019-10-31 00:50:55,785 Epoch  28 Step:   253600 Batch Loss:     1.096120 Tokens per Sec:    17101, Lr: 0.000050\n",
      "2019-10-31 00:51:10,872 Epoch  28 Step:   253700 Batch Loss:     1.191981 Tokens per Sec:    17296, Lr: 0.000050\n",
      "2019-10-31 00:51:25,741 Epoch  28 Step:   253800 Batch Loss:     1.131024 Tokens per Sec:    17282, Lr: 0.000050\n",
      "2019-10-31 00:51:40,483 Epoch  28 Step:   253900 Batch Loss:     1.234991 Tokens per Sec:    17253, Lr: 0.000050\n",
      "2019-10-31 00:51:55,446 Epoch  28 Step:   254000 Batch Loss:     1.048741 Tokens per Sec:    17353, Lr: 0.000050\n",
      "2019-10-31 00:52:39,714 Example #0\n",
      "2019-10-31 00:52:39,714 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:52:39,714 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:52:39,714 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:52:39,714 Example #1\n",
      "2019-10-31 00:52:39,714 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:52:39,714 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:52:39,714 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:52:39,714 Example #2\n",
      "2019-10-31 00:52:39,715 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:52:39,715 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:52:39,715 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 00:52:39,715 Example #3\n",
      "2019-10-31 00:52:39,715 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:52:39,715 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:52:39,715 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 00:52:39,715 Validation result at epoch  28, step   254000: bleu:  37.89, loss: 28569.5664, ppl:   2.8278, duration: 44.2682s\n",
      "2019-10-31 00:52:54,700 Epoch  28 Step:   254100 Batch Loss:     1.143687 Tokens per Sec:    17096, Lr: 0.000050\n",
      "2019-10-31 00:53:09,568 Epoch  28 Step:   254200 Batch Loss:     1.182241 Tokens per Sec:    16958, Lr: 0.000050\n",
      "2019-10-31 00:53:24,421 Epoch  28 Step:   254300 Batch Loss:     1.065972 Tokens per Sec:    17120, Lr: 0.000050\n",
      "2019-10-31 00:53:39,342 Epoch  28 Step:   254400 Batch Loss:     1.056661 Tokens per Sec:    17048, Lr: 0.000050\n",
      "2019-10-31 00:53:54,213 Epoch  28 Step:   254500 Batch Loss:     1.252573 Tokens per Sec:    17319, Lr: 0.000050\n",
      "2019-10-31 00:54:08,977 Epoch  28 Step:   254600 Batch Loss:     1.288250 Tokens per Sec:    17294, Lr: 0.000050\n",
      "2019-10-31 00:54:23,817 Epoch  28 Step:   254700 Batch Loss:     0.980397 Tokens per Sec:    17368, Lr: 0.000050\n",
      "2019-10-31 00:54:38,631 Epoch  28 Step:   254800 Batch Loss:     1.101215 Tokens per Sec:    17317, Lr: 0.000050\n",
      "2019-10-31 00:54:53,586 Epoch  28 Step:   254900 Batch Loss:     1.238678 Tokens per Sec:    17232, Lr: 0.000050\n",
      "2019-10-31 00:55:08,531 Epoch  28 Step:   255000 Batch Loss:     0.978966 Tokens per Sec:    17035, Lr: 0.000050\n",
      "2019-10-31 00:55:52,478 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 00:55:52,478 Saving new checkpoint.\n",
      "2019-10-31 00:55:52,668 Example #0\n",
      "2019-10-31 00:55:52,668 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:55:52,668 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:55:52,668 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:55:52,668 Example #1\n",
      "2019-10-31 00:55:52,668 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:55:52,668 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:55:52,669 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:55:52,669 Example #2\n",
      "2019-10-31 00:55:52,669 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:55:52,669 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:55:52,669 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukukuaje ?\n",
      "2019-10-31 00:55:52,669 Example #3\n",
      "2019-10-31 00:55:52,669 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:55:52,669 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:55:52,669 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 00:55:52,669 Validation result at epoch  28, step   255000: bleu:  37.94, loss: 28489.4844, ppl:   2.8196, duration: 44.1373s\n",
      "2019-10-31 00:56:07,744 Epoch  28 Step:   255100 Batch Loss:     1.163554 Tokens per Sec:    17130, Lr: 0.000050\n",
      "2019-10-31 00:56:22,605 Epoch  28 Step:   255200 Batch Loss:     0.909865 Tokens per Sec:    17355, Lr: 0.000050\n",
      "2019-10-31 00:56:37,511 Epoch  28 Step:   255300 Batch Loss:     1.077826 Tokens per Sec:    17203, Lr: 0.000050\n",
      "2019-10-31 00:56:52,489 Epoch  28 Step:   255400 Batch Loss:     1.376691 Tokens per Sec:    17354, Lr: 0.000050\n",
      "2019-10-31 00:57:07,498 Epoch  28 Step:   255500 Batch Loss:     1.096758 Tokens per Sec:    17080, Lr: 0.000050\n",
      "2019-10-31 00:57:22,598 Epoch  28 Step:   255600 Batch Loss:     1.096077 Tokens per Sec:    17259, Lr: 0.000050\n",
      "2019-10-31 00:57:37,775 Epoch  28 Step:   255700 Batch Loss:     1.049784 Tokens per Sec:    17246, Lr: 0.000050\n",
      "2019-10-31 00:57:52,887 Epoch  28 Step:   255800 Batch Loss:     1.282757 Tokens per Sec:    17340, Lr: 0.000050\n",
      "2019-10-31 00:58:07,758 Epoch  28 Step:   255900 Batch Loss:     1.211904 Tokens per Sec:    17447, Lr: 0.000050\n",
      "2019-10-31 00:58:22,693 Epoch  28 Step:   256000 Batch Loss:     1.385004 Tokens per Sec:    17207, Lr: 0.000050\n",
      "2019-10-31 00:59:06,925 Example #0\n",
      "2019-10-31 00:59:06,925 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 00:59:06,925 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 00:59:06,925 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 00:59:06,925 Example #1\n",
      "2019-10-31 00:59:06,925 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 00:59:06,925 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 00:59:06,925 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 00:59:06,925 Example #2\n",
      "2019-10-31 00:59:06,926 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 00:59:06,926 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 00:59:06,926 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 00:59:06,926 Example #3\n",
      "2019-10-31 00:59:06,926 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 00:59:06,926 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 00:59:06,926 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 00:59:06,926 Validation result at epoch  28, step   256000: bleu:  37.92, loss: 28504.7676, ppl:   2.8211, duration: 44.2322s\n",
      "2019-10-31 00:59:21,923 Epoch  28 Step:   256100 Batch Loss:     1.244940 Tokens per Sec:    17177, Lr: 0.000050\n",
      "2019-10-31 00:59:36,922 Epoch  28 Step:   256200 Batch Loss:     1.136847 Tokens per Sec:    17306, Lr: 0.000050\n",
      "2019-10-31 00:59:51,643 Epoch  28 Step:   256300 Batch Loss:     1.095925 Tokens per Sec:    16937, Lr: 0.000050\n",
      "2019-10-31 01:00:06,841 Epoch  28 Step:   256400 Batch Loss:     1.039073 Tokens per Sec:    16869, Lr: 0.000050\n",
      "2019-10-31 01:00:21,980 Epoch  28 Step:   256500 Batch Loss:     1.041773 Tokens per Sec:    17158, Lr: 0.000050\n",
      "2019-10-31 01:00:37,041 Epoch  28 Step:   256600 Batch Loss:     1.123659 Tokens per Sec:    17265, Lr: 0.000050\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 01:00:51,924 Epoch  28 Step:   256700 Batch Loss:     1.128314 Tokens per Sec:    17239, Lr: 0.000050\n",
      "2019-10-31 01:01:06,975 Epoch  28 Step:   256800 Batch Loss:     1.135376 Tokens per Sec:    17093, Lr: 0.000050\n",
      "2019-10-31 01:01:21,566 Epoch  28 Step:   256900 Batch Loss:     1.086909 Tokens per Sec:    17189, Lr: 0.000050\n",
      "2019-10-31 01:01:36,631 Epoch  28 Step:   257000 Batch Loss:     1.060560 Tokens per Sec:    17346, Lr: 0.000050\n",
      "2019-10-31 01:02:20,457 Example #0\n",
      "2019-10-31 01:02:20,458 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:02:20,458 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:02:20,458 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:02:20,458 Example #1\n",
      "2019-10-31 01:02:20,458 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:02:20,458 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:02:20,458 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:02:20,458 Example #2\n",
      "2019-10-31 01:02:20,458 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:02:20,458 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:02:20,458 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:02:20,458 Example #3\n",
      "2019-10-31 01:02:20,458 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:02:20,459 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:02:20,459 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 01:02:20,459 Validation result at epoch  28, step   257000: bleu:  37.82, loss: 28495.8711, ppl:   2.8202, duration: 43.8272s\n",
      "2019-10-31 01:02:35,437 Epoch  28 Step:   257100 Batch Loss:     1.303201 Tokens per Sec:    17300, Lr: 0.000050\n",
      "2019-10-31 01:02:50,119 Epoch  28 Step:   257200 Batch Loss:     1.069149 Tokens per Sec:    17128, Lr: 0.000050\n",
      "2019-10-31 01:03:05,267 Epoch  28 Step:   257300 Batch Loss:     1.179756 Tokens per Sec:    16649, Lr: 0.000050\n",
      "2019-10-31 01:03:20,176 Epoch  28 Step:   257400 Batch Loss:     1.160263 Tokens per Sec:    17042, Lr: 0.000050\n",
      "2019-10-31 01:03:35,280 Epoch  28 Step:   257500 Batch Loss:     0.996306 Tokens per Sec:    16993, Lr: 0.000050\n",
      "2019-10-31 01:03:50,372 Epoch  28 Step:   257600 Batch Loss:     1.055156 Tokens per Sec:    17040, Lr: 0.000050\n",
      "2019-10-31 01:04:05,101 Epoch  28 Step:   257700 Batch Loss:     1.084436 Tokens per Sec:    17328, Lr: 0.000050\n",
      "2019-10-31 01:04:20,236 Epoch  28 Step:   257800 Batch Loss:     1.065525 Tokens per Sec:    17339, Lr: 0.000050\n",
      "2019-10-31 01:04:35,338 Epoch  28 Step:   257900 Batch Loss:     0.982730 Tokens per Sec:    17115, Lr: 0.000050\n",
      "2019-10-31 01:04:50,282 Epoch  28 Step:   258000 Batch Loss:     1.171988 Tokens per Sec:    17421, Lr: 0.000050\n",
      "2019-10-31 01:05:34,048 Example #0\n",
      "2019-10-31 01:05:34,049 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:05:34,049 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:05:34,049 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:05:34,049 Example #1\n",
      "2019-10-31 01:05:34,049 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:05:34,049 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:05:34,049 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:05:34,049 Example #2\n",
      "2019-10-31 01:05:34,049 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:05:34,049 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:05:34,049 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:05:34,049 Example #3\n",
      "2019-10-31 01:05:34,050 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:05:34,050 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:05:34,050 \tHypothesis: Mabusu na mabusu .\n",
      "2019-10-31 01:05:34,050 Validation result at epoch  28, step   258000: bleu:  37.80, loss: 28509.7930, ppl:   2.8217, duration: 43.7669s\n",
      "2019-10-31 01:05:48,866 Epoch  28 Step:   258100 Batch Loss:     1.195750 Tokens per Sec:    17589, Lr: 0.000050\n",
      "2019-10-31 01:06:03,467 Epoch  28 Step:   258200 Batch Loss:     1.089333 Tokens per Sec:    17681, Lr: 0.000050\n",
      "2019-10-31 01:06:18,343 Epoch  28 Step:   258300 Batch Loss:     1.072356 Tokens per Sec:    17714, Lr: 0.000050\n",
      "2019-10-31 01:06:33,455 Epoch  28 Step:   258400 Batch Loss:     1.133633 Tokens per Sec:    16846, Lr: 0.000050\n",
      "2019-10-31 01:06:48,399 Epoch  28 Step:   258500 Batch Loss:     1.211970 Tokens per Sec:    17101, Lr: 0.000050\n",
      "2019-10-31 01:07:03,340 Epoch  28 Step:   258600 Batch Loss:     1.145305 Tokens per Sec:    17250, Lr: 0.000050\n",
      "2019-10-31 01:07:18,209 Epoch  28 Step:   258700 Batch Loss:     1.242937 Tokens per Sec:    16854, Lr: 0.000050\n",
      "2019-10-31 01:07:33,098 Epoch  28 Step:   258800 Batch Loss:     1.225018 Tokens per Sec:    17218, Lr: 0.000050\n",
      "2019-10-31 01:07:48,140 Epoch  28 Step:   258900 Batch Loss:     1.003730 Tokens per Sec:    17299, Lr: 0.000050\n",
      "2019-10-31 01:08:03,291 Epoch  28 Step:   259000 Batch Loss:     1.051669 Tokens per Sec:    17412, Lr: 0.000050\n",
      "2019-10-31 01:08:47,485 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 01:08:47,485 Saving new checkpoint.\n",
      "2019-10-31 01:08:47,640 Example #0\n",
      "2019-10-31 01:08:47,640 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:08:47,640 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:08:47,640 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:08:47,641 Example #1\n",
      "2019-10-31 01:08:47,641 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:08:47,641 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:08:47,641 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:08:47,641 Example #2\n",
      "2019-10-31 01:08:47,641 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:08:47,641 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:08:47,641 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:08:47,641 Example #3\n",
      "2019-10-31 01:08:47,641 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:08:47,641 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:08:47,641 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:08:47,641 Validation result at epoch  28, step   259000: bleu:  37.69, loss: 28443.0723, ppl:   2.8148, duration: 44.3500s\n",
      "2019-10-31 01:09:02,395 Epoch  28 Step:   259100 Batch Loss:     1.128406 Tokens per Sec:    17166, Lr: 0.000050\n",
      "2019-10-31 01:09:17,201 Epoch  28 Step:   259200 Batch Loss:     1.102135 Tokens per Sec:    17362, Lr: 0.000050\n",
      "2019-10-31 01:09:31,939 Epoch  28 Step:   259300 Batch Loss:     1.220477 Tokens per Sec:    17211, Lr: 0.000050\n",
      "2019-10-31 01:09:46,832 Epoch  28 Step:   259400 Batch Loss:     1.188061 Tokens per Sec:    17384, Lr: 0.000050\n",
      "2019-10-31 01:10:01,730 Epoch  28 Step:   259500 Batch Loss:     1.370724 Tokens per Sec:    17443, Lr: 0.000050\n",
      "2019-10-31 01:10:16,861 Epoch  28 Step:   259600 Batch Loss:     1.177995 Tokens per Sec:    17417, Lr: 0.000050\n",
      "2019-10-31 01:10:31,874 Epoch  28 Step:   259700 Batch Loss:     1.159092 Tokens per Sec:    17359, Lr: 0.000050\n",
      "2019-10-31 01:10:46,873 Epoch  28 Step:   259800 Batch Loss:     1.270686 Tokens per Sec:    17059, Lr: 0.000050\n",
      "2019-10-31 01:11:01,870 Epoch  28 Step:   259900 Batch Loss:     1.145332 Tokens per Sec:    16912, Lr: 0.000050\n",
      "2019-10-31 01:11:16,863 Epoch  28 Step:   260000 Batch Loss:     1.070473 Tokens per Sec:    17427, Lr: 0.000050\n",
      "2019-10-31 01:12:00,907 Example #0\n",
      "2019-10-31 01:12:00,907 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:12:00,907 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:12:00,907 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:12:00,907 Example #1\n",
      "2019-10-31 01:12:00,907 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:12:00,907 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:12:00,908 \tHypothesis: Biblia haidumii sifa ya kukubali sababu .\n",
      "2019-10-31 01:12:00,908 Example #2\n",
      "2019-10-31 01:12:00,908 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:12:00,908 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:12:00,908 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 01:12:00,908 Example #3\n",
      "2019-10-31 01:12:00,908 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:12:00,908 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:12:00,908 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 01:12:00,908 Validation result at epoch  28, step   260000: bleu:  37.89, loss: 28479.4062, ppl:   2.8185, duration: 44.0449s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 01:12:16,073 Epoch  28 Step:   260100 Batch Loss:     1.196066 Tokens per Sec:    17356, Lr: 0.000050\n",
      "2019-10-31 01:12:30,928 Epoch  28 Step:   260200 Batch Loss:     1.148694 Tokens per Sec:    17171, Lr: 0.000050\n",
      "2019-10-31 01:12:45,886 Epoch  28 Step:   260300 Batch Loss:     1.080635 Tokens per Sec:    17267, Lr: 0.000050\n",
      "2019-10-31 01:13:00,689 Epoch  28 Step:   260400 Batch Loss:     1.126804 Tokens per Sec:    17361, Lr: 0.000050\n",
      "2019-10-31 01:13:15,491 Epoch  28 Step:   260500 Batch Loss:     1.183277 Tokens per Sec:    16793, Lr: 0.000050\n",
      "2019-10-31 01:13:30,431 Epoch  28 Step:   260600 Batch Loss:     1.347538 Tokens per Sec:    16942, Lr: 0.000050\n",
      "2019-10-31 01:13:45,381 Epoch  28 Step:   260700 Batch Loss:     1.134909 Tokens per Sec:    17123, Lr: 0.000050\n",
      "2019-10-31 01:14:00,314 Epoch  28 Step:   260800 Batch Loss:     1.032122 Tokens per Sec:    16767, Lr: 0.000050\n",
      "2019-10-31 01:14:15,316 Epoch  28 Step:   260900 Batch Loss:     1.153789 Tokens per Sec:    16878, Lr: 0.000050\n",
      "2019-10-31 01:14:30,135 Epoch  28 Step:   261000 Batch Loss:     1.201040 Tokens per Sec:    16839, Lr: 0.000050\n",
      "2019-10-31 01:15:13,872 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 01:15:13,873 Saving new checkpoint.\n",
      "2019-10-31 01:15:14,012 Example #0\n",
      "2019-10-31 01:15:14,012 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:15:14,012 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:15:14,012 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:15:14,012 Example #1\n",
      "2019-10-31 01:15:14,012 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:15:14,012 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:15:14,012 \tHypothesis: Biblia haidumii sifa ya kukubali sababu .\n",
      "2019-10-31 01:15:14,012 Example #2\n",
      "2019-10-31 01:15:14,013 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:15:14,013 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:15:14,013 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-31 01:15:14,013 Example #3\n",
      "2019-10-31 01:15:14,013 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:15:14,013 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:15:14,013 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 01:15:14,013 Validation result at epoch  28, step   261000: bleu:  38.05, loss: 28417.9531, ppl:   2.8122, duration: 43.8771s\n",
      "2019-10-31 01:15:28,950 Epoch  28 Step:   261100 Batch Loss:     1.331246 Tokens per Sec:    17077, Lr: 0.000050\n",
      "2019-10-31 01:15:43,885 Epoch  28 Step:   261200 Batch Loss:     1.407833 Tokens per Sec:    17149, Lr: 0.000050\n",
      "2019-10-31 01:15:58,717 Epoch  28 Step:   261300 Batch Loss:     1.138933 Tokens per Sec:    17399, Lr: 0.000050\n",
      "2019-10-31 01:16:13,480 Epoch  28 Step:   261400 Batch Loss:     1.131627 Tokens per Sec:    17032, Lr: 0.000050\n",
      "2019-10-31 01:16:28,500 Epoch  28 Step:   261500 Batch Loss:     1.174077 Tokens per Sec:    16853, Lr: 0.000050\n",
      "2019-10-31 01:16:43,331 Epoch  28 Step:   261600 Batch Loss:     0.995078 Tokens per Sec:    17192, Lr: 0.000050\n",
      "2019-10-31 01:16:58,233 Epoch  28 Step:   261700 Batch Loss:     1.080200 Tokens per Sec:    17308, Lr: 0.000050\n",
      "2019-10-31 01:17:13,091 Epoch  28 Step:   261800 Batch Loss:     1.293798 Tokens per Sec:    17385, Lr: 0.000050\n",
      "2019-10-31 01:17:28,005 Epoch  28 Step:   261900 Batch Loss:     0.996824 Tokens per Sec:    17183, Lr: 0.000050\n",
      "2019-10-31 01:17:42,964 Epoch  28 Step:   262000 Batch Loss:     1.002541 Tokens per Sec:    17006, Lr: 0.000050\n",
      "2019-10-31 01:18:26,872 Example #0\n",
      "2019-10-31 01:18:26,872 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:18:26,872 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:18:26,872 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:18:26,872 Example #1\n",
      "2019-10-31 01:18:26,872 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:18:26,872 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:18:26,872 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:18:26,872 Example #2\n",
      "2019-10-31 01:18:26,872 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:18:26,873 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:18:26,873 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-31 01:18:26,873 Example #3\n",
      "2019-10-31 01:18:26,873 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:18:26,873 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:18:26,873 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 01:18:26,873 Validation result at epoch  28, step   262000: bleu:  37.90, loss: 28520.1543, ppl:   2.8227, duration: 43.9081s\n",
      "2019-10-31 01:18:41,778 Epoch  28 Step:   262100 Batch Loss:     1.156278 Tokens per Sec:    17282, Lr: 0.000050\n",
      "2019-10-31 01:18:56,489 Epoch  28 Step:   262200 Batch Loss:     1.030326 Tokens per Sec:    17226, Lr: 0.000050\n",
      "2019-10-31 01:19:11,417 Epoch  28 Step:   262300 Batch Loss:     1.035660 Tokens per Sec:    17345, Lr: 0.000050\n",
      "2019-10-31 01:19:26,481 Epoch  28 Step:   262400 Batch Loss:     1.151180 Tokens per Sec:    16930, Lr: 0.000050\n",
      "2019-10-31 01:19:41,424 Epoch  28 Step:   262500 Batch Loss:     1.183569 Tokens per Sec:    17381, Lr: 0.000050\n",
      "2019-10-31 01:19:56,133 Epoch  28 Step:   262600 Batch Loss:     0.959659 Tokens per Sec:    17112, Lr: 0.000050\n",
      "2019-10-31 01:20:11,134 Epoch  28 Step:   262700 Batch Loss:     1.130035 Tokens per Sec:    17216, Lr: 0.000050\n",
      "2019-10-31 01:20:23,605 Epoch  28: total training loss 10820.25\n",
      "2019-10-31 01:20:23,605 EPOCH 29\n",
      "2019-10-31 01:20:27,008 Epoch  29 Step:   262800 Batch Loss:     1.162025 Tokens per Sec:    11615, Lr: 0.000050\n",
      "2019-10-31 01:20:41,908 Epoch  29 Step:   262900 Batch Loss:     1.074819 Tokens per Sec:    17087, Lr: 0.000050\n",
      "2019-10-31 01:20:56,890 Epoch  29 Step:   263000 Batch Loss:     1.043417 Tokens per Sec:    17227, Lr: 0.000050\n",
      "2019-10-31 01:21:41,000 Example #0\n",
      "2019-10-31 01:21:41,000 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:21:41,000 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:21:41,000 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:21:41,000 Example #1\n",
      "2019-10-31 01:21:41,000 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:21:41,000 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:21:41,000 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:21:41,000 Example #2\n",
      "2019-10-31 01:21:41,000 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:21:41,001 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:21:41,001 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 01:21:41,001 Example #3\n",
      "2019-10-31 01:21:41,001 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:21:41,001 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:21:41,001 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 01:21:41,001 Validation result at epoch  29, step   263000: bleu:  38.02, loss: 28451.1445, ppl:   2.8156, duration: 44.1106s\n",
      "2019-10-31 01:21:56,065 Epoch  29 Step:   263100 Batch Loss:     1.176980 Tokens per Sec:    16981, Lr: 0.000050\n",
      "2019-10-31 01:22:10,952 Epoch  29 Step:   263200 Batch Loss:     1.091191 Tokens per Sec:    17201, Lr: 0.000050\n",
      "2019-10-31 01:22:25,662 Epoch  29 Step:   263300 Batch Loss:     1.306210 Tokens per Sec:    17506, Lr: 0.000050\n",
      "2019-10-31 01:22:40,388 Epoch  29 Step:   263400 Batch Loss:     1.154829 Tokens per Sec:    17170, Lr: 0.000050\n",
      "2019-10-31 01:22:55,151 Epoch  29 Step:   263500 Batch Loss:     1.094762 Tokens per Sec:    17305, Lr: 0.000050\n",
      "2019-10-31 01:23:10,023 Epoch  29 Step:   263600 Batch Loss:     1.298048 Tokens per Sec:    17075, Lr: 0.000050\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 01:23:24,873 Epoch  29 Step:   263700 Batch Loss:     1.118350 Tokens per Sec:    17185, Lr: 0.000050\n",
      "2019-10-31 01:23:39,925 Epoch  29 Step:   263800 Batch Loss:     1.077559 Tokens per Sec:    17647, Lr: 0.000050\n",
      "2019-10-31 01:23:54,799 Epoch  29 Step:   263900 Batch Loss:     1.285641 Tokens per Sec:    17354, Lr: 0.000050\n",
      "2019-10-31 01:24:09,812 Epoch  29 Step:   264000 Batch Loss:     0.961988 Tokens per Sec:    17449, Lr: 0.000050\n",
      "2019-10-31 01:24:54,030 Example #0\n",
      "2019-10-31 01:24:54,030 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:24:54,030 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:24:54,030 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:24:54,030 Example #1\n",
      "2019-10-31 01:24:54,030 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:24:54,030 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:24:54,030 \tHypothesis: Biblia haidumii sifa ya kukubali sababu .\n",
      "2019-10-31 01:24:54,030 Example #2\n",
      "2019-10-31 01:24:54,031 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:24:54,031 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:24:54,031 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:24:54,031 Example #3\n",
      "2019-10-31 01:24:54,031 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:24:54,031 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:24:54,031 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 01:24:54,031 Validation result at epoch  29, step   264000: bleu:  37.78, loss: 28441.5918, ppl:   2.8147, duration: 44.2181s\n",
      "2019-10-31 01:25:09,062 Epoch  29 Step:   264100 Batch Loss:     1.101451 Tokens per Sec:    17118, Lr: 0.000050\n",
      "2019-10-31 01:25:24,092 Epoch  29 Step:   264200 Batch Loss:     1.181681 Tokens per Sec:    16911, Lr: 0.000050\n",
      "2019-10-31 01:25:38,891 Epoch  29 Step:   264300 Batch Loss:     1.327332 Tokens per Sec:    17190, Lr: 0.000050\n",
      "2019-10-31 01:25:53,634 Epoch  29 Step:   264400 Batch Loss:     1.035049 Tokens per Sec:    17236, Lr: 0.000050\n",
      "2019-10-31 01:26:08,499 Epoch  29 Step:   264500 Batch Loss:     1.210810 Tokens per Sec:    17114, Lr: 0.000050\n",
      "2019-10-31 01:26:23,321 Epoch  29 Step:   264600 Batch Loss:     1.226975 Tokens per Sec:    17491, Lr: 0.000050\n",
      "2019-10-31 01:26:38,350 Epoch  29 Step:   264700 Batch Loss:     1.079522 Tokens per Sec:    17174, Lr: 0.000050\n",
      "2019-10-31 01:26:53,394 Epoch  29 Step:   264800 Batch Loss:     1.165457 Tokens per Sec:    17496, Lr: 0.000050\n",
      "2019-10-31 01:27:08,288 Epoch  29 Step:   264900 Batch Loss:     1.074618 Tokens per Sec:    17165, Lr: 0.000050\n",
      "2019-10-31 01:27:23,159 Epoch  29 Step:   265000 Batch Loss:     1.244239 Tokens per Sec:    17304, Lr: 0.000050\n",
      "2019-10-31 01:28:06,871 Example #0\n",
      "2019-10-31 01:28:06,871 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:28:06,871 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:28:06,871 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:28:06,871 Example #1\n",
      "2019-10-31 01:28:06,871 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:28:06,871 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:28:06,872 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:28:06,872 Example #2\n",
      "2019-10-31 01:28:06,872 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:28:06,872 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:28:06,872 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:28:06,872 Example #3\n",
      "2019-10-31 01:28:06,872 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:28:06,872 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:28:06,872 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-31 01:28:06,872 Validation result at epoch  29, step   265000: bleu:  37.95, loss: 28423.2852, ppl:   2.8128, duration: 43.7126s\n",
      "2019-10-31 01:28:21,619 Epoch  29 Step:   265100 Batch Loss:     1.055206 Tokens per Sec:    17136, Lr: 0.000050\n",
      "2019-10-31 01:28:36,500 Epoch  29 Step:   265200 Batch Loss:     1.032699 Tokens per Sec:    17641, Lr: 0.000050\n",
      "2019-10-31 01:28:51,192 Epoch  29 Step:   265300 Batch Loss:     1.225783 Tokens per Sec:    17311, Lr: 0.000050\n",
      "2019-10-31 01:29:05,895 Epoch  29 Step:   265400 Batch Loss:     1.049718 Tokens per Sec:    17387, Lr: 0.000050\n",
      "2019-10-31 01:29:20,874 Epoch  29 Step:   265500 Batch Loss:     1.054870 Tokens per Sec:    17447, Lr: 0.000050\n",
      "2019-10-31 01:29:35,768 Epoch  29 Step:   265600 Batch Loss:     1.096329 Tokens per Sec:    17401, Lr: 0.000050\n",
      "2019-10-31 01:29:50,636 Epoch  29 Step:   265700 Batch Loss:     1.162173 Tokens per Sec:    17755, Lr: 0.000050\n",
      "2019-10-31 01:30:05,461 Epoch  29 Step:   265800 Batch Loss:     1.304756 Tokens per Sec:    17185, Lr: 0.000050\n",
      "2019-10-31 01:30:20,758 Epoch  29 Step:   265900 Batch Loss:     1.226340 Tokens per Sec:    16886, Lr: 0.000050\n",
      "2019-10-31 01:30:35,892 Epoch  29 Step:   266000 Batch Loss:     1.268325 Tokens per Sec:    17046, Lr: 0.000050\n",
      "2019-10-31 01:31:20,219 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 01:31:20,219 Saving new checkpoint.\n",
      "2019-10-31 01:31:20,378 Example #0\n",
      "2019-10-31 01:31:20,379 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:31:20,379 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:31:20,379 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:31:20,379 Example #1\n",
      "2019-10-31 01:31:20,379 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:31:20,379 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:31:20,379 \tHypothesis: Biblia haidumii sifa ya kukubali sifa .\n",
      "2019-10-31 01:31:20,379 Example #2\n",
      "2019-10-31 01:31:20,379 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:31:20,379 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:31:20,379 \tHypothesis: Ni mambo gani yanayosababisha hangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:31:20,379 Example #3\n",
      "2019-10-31 01:31:20,379 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:31:20,379 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:31:20,379 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:31:20,380 Validation result at epoch  29, step   266000: bleu:  37.82, loss: 28401.6172, ppl:   2.8106, duration: 44.4874s\n",
      "2019-10-31 01:31:36,167 Epoch  29 Step:   266100 Batch Loss:     1.131889 Tokens per Sec:    16611, Lr: 0.000050\n",
      "2019-10-31 01:31:51,773 Epoch  29 Step:   266200 Batch Loss:     1.175464 Tokens per Sec:    16784, Lr: 0.000050\n",
      "2019-10-31 01:32:06,559 Epoch  29 Step:   266300 Batch Loss:     1.318093 Tokens per Sec:    17227, Lr: 0.000050\n",
      "2019-10-31 01:32:21,604 Epoch  29 Step:   266400 Batch Loss:     1.280722 Tokens per Sec:    17347, Lr: 0.000050\n",
      "2019-10-31 01:32:36,508 Epoch  29 Step:   266500 Batch Loss:     1.022227 Tokens per Sec:    17234, Lr: 0.000050\n",
      "2019-10-31 01:32:51,470 Epoch  29 Step:   266600 Batch Loss:     1.062152 Tokens per Sec:    17240, Lr: 0.000050\n",
      "2019-10-31 01:33:06,438 Epoch  29 Step:   266700 Batch Loss:     1.121531 Tokens per Sec:    17092, Lr: 0.000050\n",
      "2019-10-31 01:33:21,254 Epoch  29 Step:   266800 Batch Loss:     1.246778 Tokens per Sec:    17352, Lr: 0.000050\n",
      "2019-10-31 01:33:36,294 Epoch  29 Step:   266900 Batch Loss:     1.113163 Tokens per Sec:    17371, Lr: 0.000050\n",
      "2019-10-31 01:33:51,315 Epoch  29 Step:   267000 Batch Loss:     1.202031 Tokens per Sec:    16897, Lr: 0.000050\n",
      "2019-10-31 01:34:35,226 Example #0\n",
      "2019-10-31 01:34:35,226 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:34:35,226 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:34:35,226 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:34:35,226 Example #1\n",
      "2019-10-31 01:34:35,226 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:34:35,226 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:34:35,226 \tHypothesis: Biblia haidumii sifa ya kukubali sababu .\n",
      "2019-10-31 01:34:35,227 Example #2\n",
      "2019-10-31 01:34:35,227 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:34:35,227 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:34:35,227 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:34:35,227 Example #3\n",
      "2019-10-31 01:34:35,227 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:34:35,227 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:34:35,227 \tHypothesis: Miguu na mabusu .\n",
      "2019-10-31 01:34:35,227 Validation result at epoch  29, step   267000: bleu:  37.78, loss: 28432.1406, ppl:   2.8137, duration: 43.9121s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 01:34:50,191 Epoch  29 Step:   267100 Batch Loss:     1.184764 Tokens per Sec:    17038, Lr: 0.000050\n",
      "2019-10-31 01:35:05,207 Epoch  29 Step:   267200 Batch Loss:     1.308176 Tokens per Sec:    17386, Lr: 0.000050\n",
      "2019-10-31 01:35:20,242 Epoch  29 Step:   267300 Batch Loss:     1.151002 Tokens per Sec:    17322, Lr: 0.000050\n",
      "2019-10-31 01:35:35,139 Epoch  29 Step:   267400 Batch Loss:     1.158971 Tokens per Sec:    17103, Lr: 0.000050\n",
      "2019-10-31 01:35:50,313 Epoch  29 Step:   267500 Batch Loss:     1.223896 Tokens per Sec:    17345, Lr: 0.000050\n",
      "2019-10-31 01:36:05,310 Epoch  29 Step:   267600 Batch Loss:     1.084948 Tokens per Sec:    16946, Lr: 0.000050\n",
      "2019-10-31 01:36:20,408 Epoch  29 Step:   267700 Batch Loss:     1.135738 Tokens per Sec:    16869, Lr: 0.000050\n",
      "2019-10-31 01:36:35,622 Epoch  29 Step:   267800 Batch Loss:     1.212309 Tokens per Sec:    16869, Lr: 0.000050\n",
      "2019-10-31 01:36:50,761 Epoch  29 Step:   267900 Batch Loss:     1.201303 Tokens per Sec:    17081, Lr: 0.000050\n",
      "2019-10-31 01:37:05,741 Epoch  29 Step:   268000 Batch Loss:     0.954360 Tokens per Sec:    16917, Lr: 0.000050\n",
      "2019-10-31 01:37:50,145 Example #0\n",
      "2019-10-31 01:37:50,146 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:37:50,146 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:37:50,146 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:37:50,146 Example #1\n",
      "2019-10-31 01:37:50,146 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:37:50,146 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:37:50,146 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:37:50,146 Example #2\n",
      "2019-10-31 01:37:50,146 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:37:50,146 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:37:50,146 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 01:37:50,146 Example #3\n",
      "2019-10-31 01:37:50,147 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:37:50,147 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:37:50,147 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:37:50,147 Validation result at epoch  29, step   268000: bleu:  38.05, loss: 28435.6309, ppl:   2.8141, duration: 44.4052s\n",
      "2019-10-31 01:38:05,424 Epoch  29 Step:   268100 Batch Loss:     1.219454 Tokens per Sec:    16860, Lr: 0.000050\n",
      "2019-10-31 01:38:20,372 Epoch  29 Step:   268200 Batch Loss:     1.041358 Tokens per Sec:    17316, Lr: 0.000050\n",
      "2019-10-31 01:38:35,555 Epoch  29 Step:   268300 Batch Loss:     1.200191 Tokens per Sec:    16980, Lr: 0.000050\n",
      "2019-10-31 01:38:50,738 Epoch  29 Step:   268400 Batch Loss:     1.216315 Tokens per Sec:    16856, Lr: 0.000050\n",
      "2019-10-31 01:39:05,757 Epoch  29 Step:   268500 Batch Loss:     1.088071 Tokens per Sec:    16770, Lr: 0.000050\n",
      "2019-10-31 01:39:20,708 Epoch  29 Step:   268600 Batch Loss:     1.156922 Tokens per Sec:    16975, Lr: 0.000050\n",
      "2019-10-31 01:39:35,667 Epoch  29 Step:   268700 Batch Loss:     1.089585 Tokens per Sec:    17119, Lr: 0.000050\n",
      "2019-10-31 01:39:50,878 Epoch  29 Step:   268800 Batch Loss:     1.050893 Tokens per Sec:    17070, Lr: 0.000050\n",
      "2019-10-31 01:40:06,259 Epoch  29 Step:   268900 Batch Loss:     0.929153 Tokens per Sec:    16939, Lr: 0.000050\n",
      "2019-10-31 01:40:21,291 Epoch  29 Step:   269000 Batch Loss:     1.289582 Tokens per Sec:    17003, Lr: 0.000050\n",
      "2019-10-31 01:41:05,260 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 01:41:05,260 Saving new checkpoint.\n",
      "2019-10-31 01:41:05,402 Example #0\n",
      "2019-10-31 01:41:05,403 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:41:05,403 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:41:05,403 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:41:05,403 Example #1\n",
      "2019-10-31 01:41:05,403 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:41:05,403 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:41:05,403 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:41:05,403 Example #2\n",
      "2019-10-31 01:41:05,403 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:41:05,403 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:41:05,403 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:41:05,403 Example #3\n",
      "2019-10-31 01:41:05,403 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:41:05,403 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:41:05,404 \tHypothesis: Nyasi na busu .\n",
      "2019-10-31 01:41:05,404 Validation result at epoch  29, step   269000: bleu:  38.09, loss: 28379.7344, ppl:   2.8083, duration: 44.1118s\n",
      "2019-10-31 01:41:20,825 Epoch  29 Step:   269100 Batch Loss:     1.226597 Tokens per Sec:    16938, Lr: 0.000050\n",
      "2019-10-31 01:41:36,196 Epoch  29 Step:   269200 Batch Loss:     1.078322 Tokens per Sec:    16584, Lr: 0.000050\n",
      "2019-10-31 01:41:51,209 Epoch  29 Step:   269300 Batch Loss:     1.165718 Tokens per Sec:    17150, Lr: 0.000050\n",
      "2019-10-31 01:42:06,302 Epoch  29 Step:   269400 Batch Loss:     1.119147 Tokens per Sec:    17256, Lr: 0.000050\n",
      "2019-10-31 01:42:21,426 Epoch  29 Step:   269500 Batch Loss:     1.129644 Tokens per Sec:    17576, Lr: 0.000050\n",
      "2019-10-31 01:42:36,436 Epoch  29 Step:   269600 Batch Loss:     1.104603 Tokens per Sec:    16974, Lr: 0.000050\n",
      "2019-10-31 01:42:51,397 Epoch  29 Step:   269700 Batch Loss:     1.149624 Tokens per Sec:    16722, Lr: 0.000050\n",
      "2019-10-31 01:43:06,578 Epoch  29 Step:   269800 Batch Loss:     1.164907 Tokens per Sec:    16675, Lr: 0.000050\n",
      "2019-10-31 01:43:21,787 Epoch  29 Step:   269900 Batch Loss:     1.174900 Tokens per Sec:    16876, Lr: 0.000050\n",
      "2019-10-31 01:43:36,567 Epoch  29 Step:   270000 Batch Loss:     0.996181 Tokens per Sec:    16912, Lr: 0.000050\n",
      "2019-10-31 01:44:20,748 Example #0\n",
      "2019-10-31 01:44:20,748 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:44:20,748 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:44:20,748 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:44:20,748 Example #1\n",
      "2019-10-31 01:44:20,748 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:44:20,748 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:44:20,748 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:44:20,749 Example #2\n",
      "2019-10-31 01:44:20,749 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:44:20,749 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:44:20,749 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:44:20,749 Example #3\n",
      "2019-10-31 01:44:20,749 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:44:20,749 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:44:20,749 \tHypothesis: Nyasi za makumbatio na busu .\n",
      "2019-10-31 01:44:20,749 Validation result at epoch  29, step   270000: bleu:  37.73, loss: 28460.4902, ppl:   2.8166, duration: 44.1816s\n",
      "2019-10-31 01:44:35,665 Epoch  29 Step:   270100 Batch Loss:     1.093185 Tokens per Sec:    16898, Lr: 0.000050\n",
      "2019-10-31 01:44:50,901 Epoch  29 Step:   270200 Batch Loss:     1.162820 Tokens per Sec:    16964, Lr: 0.000050\n",
      "2019-10-31 01:45:06,028 Epoch  29 Step:   270300 Batch Loss:     1.042492 Tokens per Sec:    16991, Lr: 0.000050\n",
      "2019-10-31 01:45:21,202 Epoch  29 Step:   270400 Batch Loss:     1.125732 Tokens per Sec:    16890, Lr: 0.000050\n",
      "2019-10-31 01:45:36,335 Epoch  29 Step:   270500 Batch Loss:     1.221716 Tokens per Sec:    17132, Lr: 0.000050\n",
      "2019-10-31 01:45:51,402 Epoch  29 Step:   270600 Batch Loss:     1.223352 Tokens per Sec:    16985, Lr: 0.000050\n",
      "2019-10-31 01:46:06,505 Epoch  29 Step:   270700 Batch Loss:     1.053933 Tokens per Sec:    16602, Lr: 0.000050\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 01:46:21,791 Epoch  29 Step:   270800 Batch Loss:     1.213597 Tokens per Sec:    17053, Lr: 0.000050\n",
      "2019-10-31 01:46:36,816 Epoch  29 Step:   270900 Batch Loss:     1.157011 Tokens per Sec:    17241, Lr: 0.000050\n",
      "2019-10-31 01:46:51,923 Epoch  29 Step:   271000 Batch Loss:     1.211065 Tokens per Sec:    17165, Lr: 0.000050\n",
      "2019-10-31 01:47:36,446 Example #0\n",
      "2019-10-31 01:47:36,447 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:47:36,447 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:47:36,447 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:47:36,447 Example #1\n",
      "2019-10-31 01:47:36,447 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:47:36,447 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:47:36,447 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:47:36,447 Example #2\n",
      "2019-10-31 01:47:36,447 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:47:36,447 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:47:36,447 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 01:47:36,447 Example #3\n",
      "2019-10-31 01:47:36,447 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:47:36,448 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:47:36,448 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:47:36,448 Validation result at epoch  29, step   271000: bleu:  37.81, loss: 28435.3281, ppl:   2.8140, duration: 44.5245s\n",
      "2019-10-31 01:47:51,464 Epoch  29 Step:   271100 Batch Loss:     1.095353 Tokens per Sec:    17045, Lr: 0.000050\n",
      "2019-10-31 01:48:06,658 Epoch  29 Step:   271200 Batch Loss:     1.289088 Tokens per Sec:    16562, Lr: 0.000050\n",
      "2019-10-31 01:48:21,618 Epoch  29 Step:   271300 Batch Loss:     1.219930 Tokens per Sec:    16821, Lr: 0.000050\n",
      "2019-10-31 01:48:36,746 Epoch  29 Step:   271400 Batch Loss:     1.194858 Tokens per Sec:    17080, Lr: 0.000050\n",
      "2019-10-31 01:48:51,927 Epoch  29 Step:   271500 Batch Loss:     1.021694 Tokens per Sec:    17074, Lr: 0.000050\n",
      "2019-10-31 01:49:07,120 Epoch  29 Step:   271600 Batch Loss:     1.219390 Tokens per Sec:    17301, Lr: 0.000050\n",
      "2019-10-31 01:49:22,142 Epoch  29 Step:   271700 Batch Loss:     1.062563 Tokens per Sec:    16951, Lr: 0.000050\n",
      "2019-10-31 01:49:37,540 Epoch  29 Step:   271800 Batch Loss:     1.224035 Tokens per Sec:    16343, Lr: 0.000050\n",
      "2019-10-31 01:49:52,938 Epoch  29 Step:   271900 Batch Loss:     1.106788 Tokens per Sec:    16749, Lr: 0.000050\n",
      "2019-10-31 01:50:08,380 Epoch  29 Step:   272000 Batch Loss:     1.193828 Tokens per Sec:    17028, Lr: 0.000050\n",
      "2019-10-31 01:50:52,638 Example #0\n",
      "2019-10-31 01:50:52,639 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:50:52,639 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:50:52,639 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:50:52,639 Example #1\n",
      "2019-10-31 01:50:52,639 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:50:52,639 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:50:52,639 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:50:52,639 Example #2\n",
      "2019-10-31 01:50:52,639 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:50:52,639 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:50:52,639 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 01:50:52,639 Example #3\n",
      "2019-10-31 01:50:52,640 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:50:52,640 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:50:52,640 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:50:52,640 Validation result at epoch  29, step   272000: bleu:  37.97, loss: 28441.6074, ppl:   2.8147, duration: 44.2588s\n",
      "2019-10-31 01:51:07,870 Epoch  29 Step:   272100 Batch Loss:     1.285985 Tokens per Sec:    17040, Lr: 0.000050\n",
      "2019-10-31 01:51:17,205 Epoch  29: total training loss 10781.41\n",
      "2019-10-31 01:51:17,205 EPOCH 30\n",
      "2019-10-31 01:51:24,029 Epoch  30 Step:   272200 Batch Loss:     1.179965 Tokens per Sec:    14119, Lr: 0.000050\n",
      "2019-10-31 01:51:38,955 Epoch  30 Step:   272300 Batch Loss:     1.142713 Tokens per Sec:    16985, Lr: 0.000050\n",
      "2019-10-31 01:51:53,920 Epoch  30 Step:   272400 Batch Loss:     1.091125 Tokens per Sec:    17105, Lr: 0.000050\n",
      "2019-10-31 01:52:09,126 Epoch  30 Step:   272500 Batch Loss:     1.180722 Tokens per Sec:    17268, Lr: 0.000050\n",
      "2019-10-31 01:52:24,141 Epoch  30 Step:   272600 Batch Loss:     1.176442 Tokens per Sec:    17122, Lr: 0.000050\n",
      "2019-10-31 01:52:39,222 Epoch  30 Step:   272700 Batch Loss:     1.081218 Tokens per Sec:    17091, Lr: 0.000050\n",
      "2019-10-31 01:52:54,161 Epoch  30 Step:   272800 Batch Loss:     1.139007 Tokens per Sec:    17065, Lr: 0.000050\n",
      "2019-10-31 01:53:09,253 Epoch  30 Step:   272900 Batch Loss:     1.000345 Tokens per Sec:    17382, Lr: 0.000050\n",
      "2019-10-31 01:53:24,313 Epoch  30 Step:   273000 Batch Loss:     1.107297 Tokens per Sec:    16995, Lr: 0.000050\n",
      "2019-10-31 01:54:08,243 Example #0\n",
      "2019-10-31 01:54:08,243 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:54:08,243 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:54:08,243 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:54:08,243 Example #1\n",
      "2019-10-31 01:54:08,243 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:54:08,243 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:54:08,243 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:54:08,243 Example #2\n",
      "2019-10-31 01:54:08,244 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:54:08,244 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:54:08,244 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukuathiri nini ?\n",
      "2019-10-31 01:54:08,244 Example #3\n",
      "2019-10-31 01:54:08,244 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:54:08,244 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:54:08,244 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:54:08,244 Validation result at epoch  30, step   273000: bleu:  38.14, loss: 28441.7637, ppl:   2.8147, duration: 43.9300s\n",
      "2019-10-31 01:54:23,272 Epoch  30 Step:   273100 Batch Loss:     1.221350 Tokens per Sec:    17124, Lr: 0.000050\n",
      "2019-10-31 01:54:38,085 Epoch  30 Step:   273200 Batch Loss:     0.976743 Tokens per Sec:    17354, Lr: 0.000050\n",
      "2019-10-31 01:54:52,941 Epoch  30 Step:   273300 Batch Loss:     1.307727 Tokens per Sec:    17399, Lr: 0.000050\n",
      "2019-10-31 01:55:07,877 Epoch  30 Step:   273400 Batch Loss:     1.008094 Tokens per Sec:    17270, Lr: 0.000050\n",
      "2019-10-31 01:55:22,820 Epoch  30 Step:   273500 Batch Loss:     1.164270 Tokens per Sec:    17295, Lr: 0.000050\n",
      "2019-10-31 01:55:37,583 Epoch  30 Step:   273600 Batch Loss:     1.140083 Tokens per Sec:    16867, Lr: 0.000050\n",
      "2019-10-31 01:55:52,601 Epoch  30 Step:   273700 Batch Loss:     1.255958 Tokens per Sec:    17172, Lr: 0.000050\n",
      "2019-10-31 01:56:07,526 Epoch  30 Step:   273800 Batch Loss:     1.119751 Tokens per Sec:    17182, Lr: 0.000050\n",
      "2019-10-31 01:56:22,236 Epoch  30 Step:   273900 Batch Loss:     1.103341 Tokens per Sec:    17282, Lr: 0.000050\n",
      "2019-10-31 01:56:37,175 Epoch  30 Step:   274000 Batch Loss:     1.064939 Tokens per Sec:    17053, Lr: 0.000050\n",
      "2019-10-31 01:57:21,030 Example #0\n",
      "2019-10-31 01:57:21,031 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 01:57:21,031 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 01:57:21,031 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 01:57:21,031 Example #1\n",
      "2019-10-31 01:57:21,031 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 01:57:21,031 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 01:57:21,031 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 01:57:21,031 Example #2\n",
      "2019-10-31 01:57:21,031 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 01:57:21,031 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 01:57:21,031 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 01:57:21,031 Example #3\n",
      "2019-10-31 01:57:21,032 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 01:57:21,032 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 01:57:21,032 \tHypothesis: Mabusu ya makumbatio na busu .\n",
      "2019-10-31 01:57:21,032 Validation result at epoch  30, step   274000: bleu:  38.23, loss: 28441.3535, ppl:   2.8146, duration: 43.8562s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 01:57:35,851 Epoch  30 Step:   274100 Batch Loss:     1.123149 Tokens per Sec:    17368, Lr: 0.000050\n",
      "2019-10-31 01:57:50,773 Epoch  30 Step:   274200 Batch Loss:     1.181982 Tokens per Sec:    17041, Lr: 0.000050\n",
      "2019-10-31 01:58:05,702 Epoch  30 Step:   274300 Batch Loss:     1.082892 Tokens per Sec:    17289, Lr: 0.000050\n",
      "2019-10-31 01:58:20,823 Epoch  30 Step:   274400 Batch Loss:     1.534480 Tokens per Sec:    16789, Lr: 0.000050\n",
      "2019-10-31 01:58:35,894 Epoch  30 Step:   274500 Batch Loss:     1.086880 Tokens per Sec:    17583, Lr: 0.000050\n",
      "2019-10-31 01:58:50,909 Epoch  30 Step:   274600 Batch Loss:     1.237480 Tokens per Sec:    17248, Lr: 0.000050\n",
      "2019-10-31 01:59:05,919 Epoch  30 Step:   274700 Batch Loss:     1.206151 Tokens per Sec:    17098, Lr: 0.000050\n",
      "2019-10-31 01:59:21,118 Epoch  30 Step:   274800 Batch Loss:     1.249125 Tokens per Sec:    17052, Lr: 0.000050\n",
      "2019-10-31 01:59:36,065 Epoch  30 Step:   274900 Batch Loss:     1.210787 Tokens per Sec:    17158, Lr: 0.000050\n",
      "2019-10-31 01:59:50,986 Epoch  30 Step:   275000 Batch Loss:     1.182308 Tokens per Sec:    17097, Lr: 0.000050\n",
      "2019-10-31 02:00:34,810 Example #0\n",
      "2019-10-31 02:00:34,810 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:00:34,810 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:00:34,810 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 02:00:34,810 Example #1\n",
      "2019-10-31 02:00:34,810 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:00:34,810 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:00:34,810 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:00:34,810 Example #2\n",
      "2019-10-31 02:00:34,810 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:00:34,811 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:00:34,811 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 02:00:34,811 Example #3\n",
      "2019-10-31 02:00:34,811 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:00:34,811 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:00:34,811 \tHypothesis: Miguu ya makumbatio na busu .\n",
      "2019-10-31 02:00:34,811 Validation result at epoch  30, step   275000: bleu:  37.98, loss: 28426.8047, ppl:   2.8131, duration: 43.8248s\n",
      "2019-10-31 02:00:49,701 Epoch  30 Step:   275100 Batch Loss:     1.165632 Tokens per Sec:    17119, Lr: 0.000035\n",
      "2019-10-31 02:01:04,834 Epoch  30 Step:   275200 Batch Loss:     1.222692 Tokens per Sec:    16674, Lr: 0.000035\n",
      "2019-10-31 02:01:19,840 Epoch  30 Step:   275300 Batch Loss:     1.194661 Tokens per Sec:    17278, Lr: 0.000035\n",
      "2019-10-31 02:01:34,742 Epoch  30 Step:   275400 Batch Loss:     1.242624 Tokens per Sec:    17366, Lr: 0.000035\n",
      "2019-10-31 02:01:49,640 Epoch  30 Step:   275500 Batch Loss:     1.127060 Tokens per Sec:    17485, Lr: 0.000035\n",
      "2019-10-31 02:02:04,587 Epoch  30 Step:   275600 Batch Loss:     1.094622 Tokens per Sec:    17580, Lr: 0.000035\n",
      "2019-10-31 02:02:19,577 Epoch  30 Step:   275700 Batch Loss:     1.060405 Tokens per Sec:    17290, Lr: 0.000035\n",
      "2019-10-31 02:02:34,744 Epoch  30 Step:   275800 Batch Loss:     1.109622 Tokens per Sec:    16838, Lr: 0.000035\n",
      "2019-10-31 02:02:49,770 Epoch  30 Step:   275900 Batch Loss:     1.203140 Tokens per Sec:    16885, Lr: 0.000035\n",
      "2019-10-31 02:03:04,597 Epoch  30 Step:   276000 Batch Loss:     1.205542 Tokens per Sec:    17262, Lr: 0.000035\n",
      "2019-10-31 02:03:48,395 Example #0\n",
      "2019-10-31 02:03:48,395 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:03:48,395 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:03:48,395 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 02:03:48,396 Example #1\n",
      "2019-10-31 02:03:48,396 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:03:48,396 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:03:48,396 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:03:48,396 Example #2\n",
      "2019-10-31 02:03:48,396 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:03:48,396 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:03:48,396 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 02:03:48,396 Example #3\n",
      "2019-10-31 02:03:48,396 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:03:48,396 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:03:48,396 \tHypothesis: Mabusu na makumbatio .\n",
      "2019-10-31 02:03:48,396 Validation result at epoch  30, step   276000: bleu:  37.79, loss: 28387.6367, ppl:   2.8091, duration: 43.7985s\n",
      "2019-10-31 02:04:03,420 Epoch  30 Step:   276100 Batch Loss:     0.952910 Tokens per Sec:    17254, Lr: 0.000035\n",
      "2019-10-31 02:04:18,118 Epoch  30 Step:   276200 Batch Loss:     1.194887 Tokens per Sec:    16933, Lr: 0.000035\n",
      "2019-10-31 02:04:33,181 Epoch  30 Step:   276300 Batch Loss:     1.032229 Tokens per Sec:    17047, Lr: 0.000035\n",
      "2019-10-31 02:04:48,204 Epoch  30 Step:   276400 Batch Loss:     1.171475 Tokens per Sec:    17552, Lr: 0.000035\n",
      "2019-10-31 02:05:03,417 Epoch  30 Step:   276500 Batch Loss:     1.321026 Tokens per Sec:    17379, Lr: 0.000035\n",
      "2019-10-31 02:05:18,304 Epoch  30 Step:   276600 Batch Loss:     1.062738 Tokens per Sec:    16893, Lr: 0.000035\n",
      "2019-10-31 02:05:33,243 Epoch  30 Step:   276700 Batch Loss:     1.188354 Tokens per Sec:    16932, Lr: 0.000035\n",
      "2019-10-31 02:05:48,260 Epoch  30 Step:   276800 Batch Loss:     1.136893 Tokens per Sec:    16949, Lr: 0.000035\n",
      "2019-10-31 02:06:03,427 Epoch  30 Step:   276900 Batch Loss:     1.093938 Tokens per Sec:    17028, Lr: 0.000035\n",
      "2019-10-31 02:06:18,493 Epoch  30 Step:   277000 Batch Loss:     1.158824 Tokens per Sec:    17042, Lr: 0.000035\n",
      "2019-10-31 02:07:02,584 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 02:07:02,584 Saving new checkpoint.\n",
      "2019-10-31 02:07:02,733 Example #0\n",
      "2019-10-31 02:07:02,733 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:07:02,733 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:07:02,733 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:07:02,733 Example #1\n",
      "2019-10-31 02:07:02,733 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:07:02,733 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:07:02,733 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:07:02,733 Example #2\n",
      "2019-10-31 02:07:02,733 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:07:02,733 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:07:02,734 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-31 02:07:02,734 Example #3\n",
      "2019-10-31 02:07:02,734 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:07:02,734 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:07:02,734 \tHypothesis: Mabusu na mabusu .\n",
      "2019-10-31 02:07:02,734 Validation result at epoch  30, step   277000: bleu:  38.11, loss: 28370.8652, ppl:   2.8074, duration: 44.2408s\n",
      "2019-10-31 02:07:17,711 Epoch  30 Step:   277100 Batch Loss:     1.030265 Tokens per Sec:    16771, Lr: 0.000035\n",
      "2019-10-31 02:07:32,628 Epoch  30 Step:   277200 Batch Loss:     1.051744 Tokens per Sec:    17330, Lr: 0.000035\n",
      "2019-10-31 02:07:47,631 Epoch  30 Step:   277300 Batch Loss:     1.150584 Tokens per Sec:    17142, Lr: 0.000035\n",
      "2019-10-31 02:08:02,841 Epoch  30 Step:   277400 Batch Loss:     1.113310 Tokens per Sec:    17002, Lr: 0.000035\n",
      "2019-10-31 02:08:17,913 Epoch  30 Step:   277500 Batch Loss:     1.135925 Tokens per Sec:    17404, Lr: 0.000035\n",
      "2019-10-31 02:08:32,731 Epoch  30 Step:   277600 Batch Loss:     1.035716 Tokens per Sec:    17492, Lr: 0.000035\n",
      "2019-10-31 02:08:47,566 Epoch  30 Step:   277700 Batch Loss:     0.973419 Tokens per Sec:    17405, Lr: 0.000035\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 02:09:02,272 Epoch  30 Step:   277800 Batch Loss:     1.105389 Tokens per Sec:    17456, Lr: 0.000035\n",
      "2019-10-31 02:09:17,075 Epoch  30 Step:   277900 Batch Loss:     1.212099 Tokens per Sec:    17489, Lr: 0.000035\n",
      "2019-10-31 02:09:32,138 Epoch  30 Step:   278000 Batch Loss:     1.091322 Tokens per Sec:    16919, Lr: 0.000035\n",
      "2019-10-31 02:10:16,195 Example #0\n",
      "2019-10-31 02:10:16,195 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:10:16,195 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:10:16,195 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 02:10:16,195 Example #1\n",
      "2019-10-31 02:10:16,195 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:10:16,195 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:10:16,195 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:10:16,195 Example #2\n",
      "2019-10-31 02:10:16,196 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:10:16,196 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:10:16,196 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kutokeza nini ?\n",
      "2019-10-31 02:10:16,196 Example #3\n",
      "2019-10-31 02:10:16,196 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:10:16,196 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:10:16,196 \tHypothesis: Makumbatio na busu .\n",
      "2019-10-31 02:10:16,196 Validation result at epoch  30, step   278000: bleu:  38.11, loss: 28399.1602, ppl:   2.8103, duration: 44.0574s\n",
      "2019-10-31 02:10:31,076 Epoch  30 Step:   278100 Batch Loss:     1.111642 Tokens per Sec:    17141, Lr: 0.000035\n",
      "2019-10-31 02:10:45,689 Epoch  30 Step:   278200 Batch Loss:     0.988922 Tokens per Sec:    17197, Lr: 0.000035\n",
      "2019-10-31 02:11:00,466 Epoch  30 Step:   278300 Batch Loss:     1.224692 Tokens per Sec:    17156, Lr: 0.000035\n",
      "2019-10-31 02:11:15,395 Epoch  30 Step:   278400 Batch Loss:     1.199802 Tokens per Sec:    17376, Lr: 0.000035\n",
      "2019-10-31 02:11:30,380 Epoch  30 Step:   278500 Batch Loss:     1.107169 Tokens per Sec:    17165, Lr: 0.000035\n",
      "2019-10-31 02:11:45,295 Epoch  30 Step:   278600 Batch Loss:     1.123515 Tokens per Sec:    17216, Lr: 0.000035\n",
      "2019-10-31 02:12:00,116 Epoch  30 Step:   278700 Batch Loss:     1.016174 Tokens per Sec:    17044, Lr: 0.000035\n",
      "2019-10-31 02:12:14,943 Epoch  30 Step:   278800 Batch Loss:     1.032269 Tokens per Sec:    17173, Lr: 0.000035\n",
      "2019-10-31 02:12:29,720 Epoch  30 Step:   278900 Batch Loss:     1.146346 Tokens per Sec:    16950, Lr: 0.000035\n",
      "2019-10-31 02:12:44,669 Epoch  30 Step:   279000 Batch Loss:     1.219164 Tokens per Sec:    17023, Lr: 0.000035\n",
      "2019-10-31 02:13:29,004 Hooray! New best validation result [ppl]!\n",
      "2019-10-31 02:13:29,004 Saving new checkpoint.\n",
      "2019-10-31 02:13:29,157 Example #0\n",
      "2019-10-31 02:13:29,157 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:13:29,157 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:13:29,157 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:13:29,157 Example #1\n",
      "2019-10-31 02:13:29,157 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:13:29,157 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:13:29,158 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:13:29,158 Example #2\n",
      "2019-10-31 02:13:29,158 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:13:29,158 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:13:29,158 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 02:13:29,158 Example #3\n",
      "2019-10-31 02:13:29,158 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:13:29,158 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:13:29,158 \tHypothesis: Mabusu na mabusu .\n",
      "2019-10-31 02:13:29,158 Validation result at epoch  30, step   279000: bleu:  37.99, loss: 28314.6426, ppl:   2.8017, duration: 44.4888s\n",
      "2019-10-31 02:13:44,401 Epoch  30 Step:   279100 Batch Loss:     1.158803 Tokens per Sec:    17134, Lr: 0.000035\n",
      "2019-10-31 02:13:59,208 Epoch  30 Step:   279200 Batch Loss:     1.066023 Tokens per Sec:    16968, Lr: 0.000035\n",
      "2019-10-31 02:14:14,261 Epoch  30 Step:   279300 Batch Loss:     1.160998 Tokens per Sec:    17030, Lr: 0.000035\n",
      "2019-10-31 02:14:29,174 Epoch  30 Step:   279400 Batch Loss:     1.133144 Tokens per Sec:    16988, Lr: 0.000035\n",
      "2019-10-31 02:14:44,068 Epoch  30 Step:   279500 Batch Loss:     1.142421 Tokens per Sec:    17273, Lr: 0.000035\n",
      "2019-10-31 02:14:58,889 Epoch  30 Step:   279600 Batch Loss:     1.165101 Tokens per Sec:    17352, Lr: 0.000035\n",
      "2019-10-31 02:15:13,628 Epoch  30 Step:   279700 Batch Loss:     1.113027 Tokens per Sec:    17147, Lr: 0.000035\n",
      "2019-10-31 02:15:28,356 Epoch  30 Step:   279800 Batch Loss:     1.214663 Tokens per Sec:    17574, Lr: 0.000035\n",
      "2019-10-31 02:15:43,454 Epoch  30 Step:   279900 Batch Loss:     1.040818 Tokens per Sec:    16920, Lr: 0.000035\n",
      "2019-10-31 02:15:58,415 Epoch  30 Step:   280000 Batch Loss:     1.284698 Tokens per Sec:    17168, Lr: 0.000035\n",
      "2019-10-31 02:16:42,190 Example #0\n",
      "2019-10-31 02:16:42,191 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:16:42,191 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:16:42,191 \tHypothesis: Alikuwa na mimba alipokuwa na umri wa miaka 17 .\n",
      "2019-10-31 02:16:42,191 Example #1\n",
      "2019-10-31 02:16:42,191 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:16:42,191 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:16:42,191 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:16:42,191 Example #2\n",
      "2019-10-31 02:16:42,191 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:16:42,191 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:16:42,191 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kusababisha nini ?\n",
      "2019-10-31 02:16:42,191 Example #3\n",
      "2019-10-31 02:16:42,191 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:16:42,191 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:16:42,191 \tHypothesis: Mabusu na mabusu .\n",
      "2019-10-31 02:16:42,192 Validation result at epoch  30, step   280000: bleu:  37.77, loss: 28372.3047, ppl:   2.8076, duration: 43.7756s\n",
      "2019-10-31 02:16:57,106 Epoch  30 Step:   280100 Batch Loss:     1.118863 Tokens per Sec:    17289, Lr: 0.000035\n",
      "2019-10-31 02:17:11,968 Epoch  30 Step:   280200 Batch Loss:     1.137116 Tokens per Sec:    17163, Lr: 0.000035\n",
      "2019-10-31 02:17:26,902 Epoch  30 Step:   280300 Batch Loss:     1.185914 Tokens per Sec:    17113, Lr: 0.000035\n",
      "2019-10-31 02:17:41,810 Epoch  30 Step:   280400 Batch Loss:     1.223860 Tokens per Sec:    17393, Lr: 0.000035\n",
      "2019-10-31 02:17:56,587 Epoch  30 Step:   280500 Batch Loss:     1.189232 Tokens per Sec:    17188, Lr: 0.000035\n",
      "2019-10-31 02:18:11,734 Epoch  30 Step:   280600 Batch Loss:     1.044229 Tokens per Sec:    17525, Lr: 0.000035\n",
      "2019-10-31 02:18:26,543 Epoch  30 Step:   280700 Batch Loss:     1.119903 Tokens per Sec:    17271, Lr: 0.000035\n",
      "2019-10-31 02:18:41,440 Epoch  30 Step:   280800 Batch Loss:     1.071672 Tokens per Sec:    17644, Lr: 0.000035\n",
      "2019-10-31 02:18:56,148 Epoch  30 Step:   280900 Batch Loss:     1.140345 Tokens per Sec:    17456, Lr: 0.000035\n",
      "2019-10-31 02:19:10,739 Epoch  30 Step:   281000 Batch Loss:     1.102866 Tokens per Sec:    17275, Lr: 0.000035\n",
      "2019-10-31 02:19:54,713 Example #0\n",
      "2019-10-31 02:19:54,713 \tSource:     She became pregnant at the age of 17 .\n",
      "2019-10-31 02:19:54,713 \tReference:  Alipata mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:19:54,713 \tHypothesis: Alikuwa na mimba akiwa na umri wa miaka 17 .\n",
      "2019-10-31 02:19:54,713 Example #1\n",
      "2019-10-31 02:19:54,714 \tSource:     The Bible does not promote credulity .\n",
      "2019-10-31 02:19:54,714 \tReference:  Biblia haipendekezi wepesi wa kuamini bila ushuhuda .\n",
      "2019-10-31 02:19:54,714 \tHypothesis: Biblia haisemi kwamba mtu ana sifa ya kukubali sababu .\n",
      "2019-10-31 02:19:54,714 Example #2\n",
      "2019-10-31 02:19:54,714 \tSource:     What are some things that cause anxiety , and what can anxiety result in ?\n",
      "2019-10-31 02:19:54,714 \tReference:  Ni baadhi ya mambo gani yanayosababisha hangaiko , na hangaiko laweza kutokeza nini ?\n",
      "2019-10-31 02:19:54,714 \tHypothesis: Ni mambo gani yanayosababisha mahangaiko , na mahangaiko yanaweza kukufanyaje ?\n",
      "2019-10-31 02:19:54,714 Example #3\n",
      "2019-10-31 02:19:54,714 \tSource:     Lots of hugs and kisses .\n",
      "2019-10-31 02:19:54,714 \tReference:  Kukumbatia kwingi na busu nyingi .\n",
      "2019-10-31 02:19:54,714 \tHypothesis: Mabusu na mabusu .\n",
      "2019-10-31 02:19:54,714 Validation result at epoch  30, step   281000: bleu:  37.97, loss: 28345.1191, ppl:   2.8048, duration: 43.9751s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-10-31 02:20:09,446 Epoch  30 Step:   281100 Batch Loss:     1.140924 Tokens per Sec:    17191, Lr: 0.000035\n",
      "2019-10-31 02:20:24,354 Epoch  30 Step:   281200 Batch Loss:     1.061550 Tokens per Sec:    17530, Lr: 0.000035\n",
      "2019-10-31 02:20:39,233 Epoch  30 Step:   281300 Batch Loss:     1.142339 Tokens per Sec:    17272, Lr: 0.000035\n",
      "2019-10-31 02:20:54,372 Epoch  30 Step:   281400 Batch Loss:     1.203827 Tokens per Sec:    16785, Lr: 0.000035\n",
      "2019-10-31 02:21:09,555 Epoch  30 Step:   281500 Batch Loss:     1.136458 Tokens per Sec:    17312, Lr: 0.000035\n",
      "2019-10-31 02:21:17,507 Epoch  30: total training loss 10767.81\n",
      "2019-10-31 02:21:17,508 Training ended after  30 epochs.\n",
      "2019-10-31 02:21:17,508 Best validation result at step   279000:   2.80 ppl.\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/ks/anaconda3/lib/python3.6/runpy.py\", line 193, in _run_module_as_main\n",
      "    \"__main__\", mod_spec)\n",
      "  File \"/home/ks/anaconda3/lib/python3.6/runpy.py\", line 85, in _run_code\n",
      "    exec(code, run_globals)\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/__main__.py\", line 41, in <module>\n",
      "    main()\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/__main__.py\", line 29, in main\n",
      "    train(cfg_file=args.config_path)\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/training.py\", line 573, in train\n",
      "    test(cfg_file, ckpt=ckpt, output_path=output_path, logger=trainer.logger)\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/prediction.py\", line 241, in test\n",
      "    beam_alpha=beam_alpha)\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/prediction.py\", line 98, in validate_on_data\n",
      "    max_output_length=max_output_length)\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/model.py\", line 182, in run_batch\n",
      "    decoder=self.decoder)\n",
      "  File \"/home/ks/.local/lib/python3.6/site-packages/joeynmt/search.py\", line 320, in beam_search\n",
      "    if is_finished.any():\n",
      "RuntimeError: CUDA error: an illegal memory access was encountered\n"
     ]
    }
   ],
   "source": [
    "# Train the model\n",
    "# You can press Ctrl-C to stop. And then run the next cell to save your checkpoints! \n",
    "!cd joeynmt; python3 -m joeynmt train /home/ks/code/tools/joeynmt/configs/transformer_filtered_$src$tgt.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "MBoDS09JM807"
   },
   "outputs": [],
   "source": [
    "# Copy the created models from the notebook storage to google drive for persistant storage \n",
    "!cp -r joeynmt/models/${src}${tgt}_transformer/* \"$gdrive_path/models/${src}${tgt}_transformer/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n94wlrCjVc17"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Steps: 1000\tLoss: 120892.71094\tPPL: 81.34162\tbleu: 0.87183\tLR: 0.00030000\t*\r\n",
      "Steps: 2000\tLoss: 99126.69531\tPPL: 36.84446\tbleu: 2.80529\tLR: 0.00030000\t*\r\n",
      "Steps: 3000\tLoss: 85728.57031\tPPL: 22.62866\tbleu: 5.11622\tLR: 0.00030000\t*\r\n",
      "Steps: 4000\tLoss: 77592.10938\tPPL: 16.83018\tbleu: 8.42625\tLR: 0.00030000\t*\r\n",
      "Steps: 5000\tLoss: 71383.70312\tPPL: 13.42720\tbleu: 10.88299\tLR: 0.00030000\t*\r\n",
      "Steps: 6000\tLoss: 67318.64062\tPPL: 11.58111\tbleu: 12.83706\tLR: 0.00030000\t*\r\n",
      "Steps: 7000\tLoss: 63303.48828\tPPL: 10.00700\tbleu: 15.14659\tLR: 0.00030000\t*\r\n",
      "Steps: 8000\tLoss: 60214.65625\tPPL: 8.94324\tbleu: 16.16055\tLR: 0.00030000\t*\r\n",
      "Steps: 9000\tLoss: 57998.01562\tPPL: 8.25027\tbleu: 17.94183\tLR: 0.00030000\t*\r\n",
      "Steps: 10000\tLoss: 55727.73438\tPPL: 7.59616\tbleu: 18.76534\tLR: 0.00030000\t*\r\n",
      "Steps: 11000\tLoss: 54107.03125\tPPL: 7.16117\tbleu: 20.55164\tLR: 0.00030000\t*\r\n",
      "Steps: 12000\tLoss: 52610.32812\tPPL: 6.78162\tbleu: 21.35097\tLR: 0.00030000\t*\r\n",
      "Steps: 13000\tLoss: 50830.55469\tPPL: 6.35638\tbleu: 22.80579\tLR: 0.00030000\t*\r\n",
      "Steps: 14000\tLoss: 49828.48828\tPPL: 6.12881\tbleu: 23.40427\tLR: 0.00030000\t*\r\n",
      "Steps: 15000\tLoss: 48824.37500\tPPL: 5.90893\tbleu: 23.88411\tLR: 0.00030000\t*\r\n",
      "Steps: 16000\tLoss: 47932.80078\tPPL: 5.72032\tbleu: 24.76646\tLR: 0.00030000\t*\r\n",
      "Steps: 17000\tLoss: 47172.92578\tPPL: 5.56434\tbleu: 25.35275\tLR: 0.00030000\t*\r\n",
      "Steps: 18000\tLoss: 46360.53516\tPPL: 5.40227\tbleu: 25.92302\tLR: 0.00030000\t*\r\n",
      "Steps: 19000\tLoss: 46063.87109\tPPL: 5.34427\tbleu: 25.96658\tLR: 0.00030000\t*\r\n",
      "Steps: 20000\tLoss: 45050.66797\tPPL: 5.15084\tbleu: 26.61159\tLR: 0.00030000\t*\r\n",
      "Steps: 21000\tLoss: 44776.55078\tPPL: 5.09972\tbleu: 26.45332\tLR: 0.00030000\t*\r\n",
      "Steps: 22000\tLoss: 44034.16016\tPPL: 4.96381\tbleu: 27.41357\tLR: 0.00030000\t*\r\n",
      "Steps: 23000\tLoss: 43545.26953\tPPL: 4.87630\tbleu: 27.42019\tLR: 0.00030000\t*\r\n",
      "Steps: 24000\tLoss: 43294.89844\tPPL: 4.83208\tbleu: 27.59287\tLR: 0.00030000\t*\r\n",
      "Steps: 25000\tLoss: 42622.92578\tPPL: 4.71537\tbleu: 28.40310\tLR: 0.00030000\t*\r\n",
      "Steps: 26000\tLoss: 42317.86328\tPPL: 4.66332\tbleu: 28.17688\tLR: 0.00030000\t*\r\n",
      "Steps: 27000\tLoss: 42408.57031\tPPL: 4.67873\tbleu: 28.80856\tLR: 0.00030000\t\r\n",
      "Steps: 28000\tLoss: 41475.07812\tPPL: 4.52249\tbleu: 29.24517\tLR: 0.00030000\t*\r\n",
      "Steps: 29000\tLoss: 41200.60156\tPPL: 4.47755\tbleu: 28.83580\tLR: 0.00030000\t*\r\n",
      "Steps: 30000\tLoss: 40869.16406\tPPL: 4.42388\tbleu: 29.89765\tLR: 0.00030000\t*\r\n",
      "Steps: 31000\tLoss: 40537.29297\tPPL: 4.37078\tbleu: 29.47853\tLR: 0.00030000\t*\r\n",
      "Steps: 32000\tLoss: 40215.58203\tPPL: 4.31992\tbleu: 29.72621\tLR: 0.00030000\t*\r\n",
      "Steps: 33000\tLoss: 40052.18750\tPPL: 4.29431\tbleu: 30.27762\tLR: 0.00030000\t*\r\n",
      "Steps: 34000\tLoss: 39650.90234\tPPL: 4.23207\tbleu: 30.38181\tLR: 0.00030000\t*\r\n",
      "Steps: 35000\tLoss: 39420.60938\tPPL: 4.19675\tbleu: 30.11570\tLR: 0.00030000\t*\r\n",
      "Steps: 36000\tLoss: 39353.98438\tPPL: 4.18659\tbleu: 30.77619\tLR: 0.00030000\t*\r\n",
      "Steps: 37000\tLoss: 39078.60547\tPPL: 4.14485\tbleu: 30.60997\tLR: 0.00030000\t*\r\n",
      "Steps: 38000\tLoss: 38805.41016\tPPL: 4.10386\tbleu: 31.08538\tLR: 0.00030000\t*\r\n",
      "Steps: 39000\tLoss: 38516.14453\tPPL: 4.06089\tbleu: 31.06407\tLR: 0.00030000\t*\r\n",
      "Steps: 40000\tLoss: 38459.91406\tPPL: 4.05259\tbleu: 31.37077\tLR: 0.00030000\t*\r\n",
      "Steps: 41000\tLoss: 38292.45703\tPPL: 4.02797\tbleu: 31.16915\tLR: 0.00030000\t*\r\n",
      "Steps: 42000\tLoss: 37927.41016\tPPL: 3.97483\tbleu: 31.57028\tLR: 0.00030000\t*\r\n",
      "Steps: 43000\tLoss: 38000.16797\tPPL: 3.98536\tbleu: 31.99779\tLR: 0.00030000\t\r\n",
      "Steps: 44000\tLoss: 37800.33203\tPPL: 3.95649\tbleu: 31.55655\tLR: 0.00030000\t*\r\n",
      "Steps: 45000\tLoss: 37423.08594\tPPL: 3.90256\tbleu: 32.23505\tLR: 0.00030000\t*\r\n",
      "Steps: 46000\tLoss: 37306.10156\tPPL: 3.88598\tbleu: 32.13818\tLR: 0.00030000\t*\r\n",
      "Steps: 47000\tLoss: 37149.23047\tPPL: 3.86386\tbleu: 31.42404\tLR: 0.00030000\t*\r\n",
      "Steps: 48000\tLoss: 36985.19922\tPPL: 3.84087\tbleu: 32.34572\tLR: 0.00030000\t*\r\n",
      "Steps: 49000\tLoss: 37501.17188\tPPL: 3.91366\tbleu: 32.25137\tLR: 0.00030000\t\r\n",
      "Steps: 50000\tLoss: 36772.99609\tPPL: 3.81133\tbleu: 32.28899\tLR: 0.00030000\t*\r\n",
      "Steps: 51000\tLoss: 36618.37891\tPPL: 3.78995\tbleu: 32.56781\tLR: 0.00030000\t*\r\n",
      "Steps: 52000\tLoss: 36361.38672\tPPL: 3.75468\tbleu: 32.54116\tLR: 0.00030000\t*\r\n",
      "Steps: 53000\tLoss: 36417.44531\tPPL: 3.76234\tbleu: 32.98250\tLR: 0.00030000\t\r\n",
      "Steps: 54000\tLoss: 36260.96484\tPPL: 3.74098\tbleu: 32.72846\tLR: 0.00030000\t*\r\n",
      "Steps: 55000\tLoss: 36357.16016\tPPL: 3.75410\tbleu: 32.65082\tLR: 0.00030000\t\r\n",
      "Steps: 56000\tLoss: 35905.95703\tPPL: 3.69297\tbleu: 33.16483\tLR: 0.00030000\t*\r\n",
      "Steps: 57000\tLoss: 35867.71484\tPPL: 3.68784\tbleu: 33.19776\tLR: 0.00030000\t*\r\n",
      "Steps: 58000\tLoss: 35726.80078\tPPL: 3.66898\tbleu: 32.85108\tLR: 0.00030000\t*\r\n",
      "Steps: 59000\tLoss: 35735.72656\tPPL: 3.67017\tbleu: 33.47774\tLR: 0.00030000\t\r\n",
      "Steps: 60000\tLoss: 35672.89062\tPPL: 3.66179\tbleu: 33.39130\tLR: 0.00030000\t*\r\n",
      "Steps: 61000\tLoss: 35411.03125\tPPL: 3.62706\tbleu: 33.59951\tLR: 0.00030000\t*\r\n",
      "Steps: 62000\tLoss: 35429.22656\tPPL: 3.62947\tbleu: 33.24404\tLR: 0.00030000\t\r\n",
      "Steps: 63000\tLoss: 35401.96875\tPPL: 3.62587\tbleu: 33.50544\tLR: 0.00030000\t*\r\n",
      "Steps: 64000\tLoss: 35357.35938\tPPL: 3.61999\tbleu: 33.53313\tLR: 0.00030000\t*\r\n",
      "Steps: 65000\tLoss: 35351.80078\tPPL: 3.61926\tbleu: 33.31014\tLR: 0.00030000\t*\r\n",
      "Steps: 66000\tLoss: 34965.41016\tPPL: 3.56873\tbleu: 33.85054\tLR: 0.00030000\t*\r\n",
      "Steps: 67000\tLoss: 35010.45703\tPPL: 3.57458\tbleu: 33.87821\tLR: 0.00030000\t\r\n",
      "Steps: 68000\tLoss: 34708.29688\tPPL: 3.53550\tbleu: 33.97063\tLR: 0.00030000\t*\r\n",
      "Steps: 69000\tLoss: 34829.85156\tPPL: 3.55117\tbleu: 33.97692\tLR: 0.00030000\t\r\n",
      "Steps: 70000\tLoss: 34715.03906\tPPL: 3.53637\tbleu: 34.01601\tLR: 0.00030000\t\r\n",
      "Steps: 71000\tLoss: 34794.76953\tPPL: 3.54664\tbleu: 33.72710\tLR: 0.00030000\t\r\n",
      "Steps: 72000\tLoss: 34570.58203\tPPL: 3.51783\tbleu: 33.76552\tLR: 0.00030000\t*\r\n",
      "Steps: 73000\tLoss: 34559.21484\tPPL: 3.51637\tbleu: 34.03367\tLR: 0.00030000\t*\r\n",
      "Steps: 74000\tLoss: 34489.00781\tPPL: 3.50740\tbleu: 34.48546\tLR: 0.00030000\t*\r\n",
      "Steps: 75000\tLoss: 34189.32812\tPPL: 3.46937\tbleu: 34.03985\tLR: 0.00030000\t*\r\n",
      "Steps: 76000\tLoss: 34224.55859\tPPL: 3.47382\tbleu: 34.34199\tLR: 0.00030000\t\r\n",
      "Steps: 77000\tLoss: 34033.79297\tPPL: 3.44979\tbleu: 34.25632\tLR: 0.00030000\t*\r\n",
      "Steps: 78000\tLoss: 34203.16016\tPPL: 3.47111\tbleu: 33.84891\tLR: 0.00030000\t\r\n",
      "Steps: 79000\tLoss: 34139.79297\tPPL: 3.46312\tbleu: 33.68260\tLR: 0.00030000\t\r\n",
      "Steps: 80000\tLoss: 33816.40625\tPPL: 3.42261\tbleu: 34.26447\tLR: 0.00030000\t*\r\n",
      "Steps: 81000\tLoss: 33810.94922\tPPL: 3.42193\tbleu: 34.39860\tLR: 0.00030000\t*\r\n",
      "Steps: 82000\tLoss: 33773.25781\tPPL: 3.41724\tbleu: 34.04448\tLR: 0.00030000\t*\r\n",
      "Steps: 83000\tLoss: 33684.87891\tPPL: 3.40627\tbleu: 34.24579\tLR: 0.00030000\t*\r\n",
      "Steps: 84000\tLoss: 33735.37500\tPPL: 3.41253\tbleu: 34.56783\tLR: 0.00030000\t\r\n",
      "Steps: 85000\tLoss: 33511.31641\tPPL: 3.38483\tbleu: 34.65834\tLR: 0.00030000\t*\r\n",
      "Steps: 86000\tLoss: 33704.41406\tPPL: 3.40869\tbleu: 34.34353\tLR: 0.00030000\t\r\n",
      "Steps: 87000\tLoss: 33461.07422\tPPL: 3.37864\tbleu: 34.54206\tLR: 0.00030000\t*\r\n",
      "Steps: 88000\tLoss: 33560.43750\tPPL: 3.39088\tbleu: 34.83735\tLR: 0.00030000\t\r\n",
      "Steps: 89000\tLoss: 33420.62891\tPPL: 3.37368\tbleu: 34.73079\tLR: 0.00030000\t*\r\n",
      "Steps: 90000\tLoss: 33172.35547\tPPL: 3.34334\tbleu: 34.02525\tLR: 0.00030000\t*\r\n",
      "Steps: 91000\tLoss: 33278.90625\tPPL: 3.35632\tbleu: 34.31515\tLR: 0.00030000\t\r\n",
      "Steps: 92000\tLoss: 33262.96094\tPPL: 3.35438\tbleu: 35.01830\tLR: 0.00030000\t\r\n",
      "Steps: 93000\tLoss: 33005.48438\tPPL: 3.32310\tbleu: 34.94418\tLR: 0.00030000\t*\r\n",
      "Steps: 94000\tLoss: 33049.39844\tPPL: 3.32841\tbleu: 34.91126\tLR: 0.00030000\t\r\n",
      "Steps: 95000\tLoss: 33007.13281\tPPL: 3.32330\tbleu: 35.46520\tLR: 0.00030000\t\r\n",
      "Steps: 96000\tLoss: 33020.64453\tPPL: 3.32493\tbleu: 34.92529\tLR: 0.00030000\t\r\n",
      "Steps: 97000\tLoss: 32866.23438\tPPL: 3.30631\tbleu: 34.74421\tLR: 0.00030000\t*\r\n",
      "Steps: 98000\tLoss: 32813.84375\tPPL: 3.30001\tbleu: 35.08813\tLR: 0.00030000\t*\r\n",
      "Steps: 99000\tLoss: 32815.85156\tPPL: 3.30025\tbleu: 35.17029\tLR: 0.00030000\t\r\n",
      "Steps: 100000\tLoss: 32683.27734\tPPL: 3.28437\tbleu: 34.93472\tLR: 0.00030000\t*\r\n",
      "Steps: 101000\tLoss: 32718.93164\tPPL: 3.28863\tbleu: 35.49196\tLR: 0.00030000\t\r\n",
      "Steps: 102000\tLoss: 32614.15625\tPPL: 3.27612\tbleu: 35.17246\tLR: 0.00030000\t*\r\n",
      "Steps: 103000\tLoss: 32521.37109\tPPL: 3.26508\tbleu: 34.71640\tLR: 0.00030000\t*\r\n",
      "Steps: 104000\tLoss: 32619.69531\tPPL: 3.27678\tbleu: 35.24257\tLR: 0.00030000\t\r\n",
      "Steps: 105000\tLoss: 32434.09570\tPPL: 3.25473\tbleu: 35.00183\tLR: 0.00030000\t*\r\n",
      "Steps: 106000\tLoss: 32509.03516\tPPL: 3.26361\tbleu: 35.48746\tLR: 0.00030000\t\r\n",
      "Steps: 107000\tLoss: 32454.41016\tPPL: 3.25713\tbleu: 35.39349\tLR: 0.00030000\t\r\n",
      "Steps: 108000\tLoss: 32438.54102\tPPL: 3.25525\tbleu: 35.68454\tLR: 0.00030000\t\r\n",
      "Steps: 109000\tLoss: 32308.58008\tPPL: 3.23990\tbleu: 35.18858\tLR: 0.00030000\t*\r\n",
      "Steps: 110000\tLoss: 32375.71289\tPPL: 3.24782\tbleu: 35.63733\tLR: 0.00030000\t\r\n",
      "Steps: 111000\tLoss: 32240.75977\tPPL: 3.23191\tbleu: 35.54290\tLR: 0.00030000\t*\r\n",
      "Steps: 112000\tLoss: 32313.73633\tPPL: 3.24050\tbleu: 35.58986\tLR: 0.00030000\t\r\n",
      "Steps: 113000\tLoss: 32175.10547\tPPL: 3.22420\tbleu: 35.47357\tLR: 0.00030000\t*\r\n",
      "Steps: 114000\tLoss: 32020.22461\tPPL: 3.20608\tbleu: 35.23472\tLR: 0.00030000\t*\r\n",
      "Steps: 115000\tLoss: 32154.16406\tPPL: 3.22174\tbleu: 35.49017\tLR: 0.00030000\t\r\n",
      "Steps: 116000\tLoss: 32157.87500\tPPL: 3.22218\tbleu: 35.52521\tLR: 0.00030000\t\r\n",
      "Steps: 117000\tLoss: 32123.21094\tPPL: 3.21812\tbleu: 35.67953\tLR: 0.00030000\t\r\n",
      "Steps: 118000\tLoss: 31925.08203\tPPL: 3.19500\tbleu: 35.35512\tLR: 0.00030000\t*\r\n",
      "Steps: 119000\tLoss: 32032.62109\tPPL: 3.20753\tbleu: 35.59057\tLR: 0.00030000\t\r\n",
      "Steps: 120000\tLoss: 31874.06641\tPPL: 3.18908\tbleu: 35.46932\tLR: 0.00030000\t*\r\n",
      "Steps: 121000\tLoss: 31844.57031\tPPL: 3.18566\tbleu: 35.69936\tLR: 0.00030000\t*\r\n",
      "Steps: 122000\tLoss: 31838.44336\tPPL: 3.18495\tbleu: 35.55048\tLR: 0.00030000\t*\r\n",
      "Steps: 123000\tLoss: 31738.65234\tPPL: 3.17340\tbleu: 35.49872\tLR: 0.00030000\t*\r\n",
      "Steps: 124000\tLoss: 31784.07227\tPPL: 3.17865\tbleu: 35.33343\tLR: 0.00030000\t\r\n",
      "Steps: 125000\tLoss: 31742.44727\tPPL: 3.17384\tbleu: 35.68538\tLR: 0.00030000\t\r\n",
      "Steps: 126000\tLoss: 31644.57031\tPPL: 3.16256\tbleu: 35.81932\tLR: 0.00030000\t*\r\n",
      "Steps: 127000\tLoss: 31720.33203\tPPL: 3.17129\tbleu: 36.15370\tLR: 0.00030000\t\r\n",
      "Steps: 128000\tLoss: 31720.56836\tPPL: 3.17132\tbleu: 35.91907\tLR: 0.00030000\t\r\n",
      "Steps: 129000\tLoss: 31583.33984\tPPL: 3.15552\tbleu: 35.64690\tLR: 0.00030000\t*\r\n",
      "Steps: 130000\tLoss: 31557.44141\tPPL: 3.15255\tbleu: 35.91141\tLR: 0.00030000\t*\r\n",
      "Steps: 131000\tLoss: 31527.26172\tPPL: 3.14909\tbleu: 35.75349\tLR: 0.00030000\t*\r\n",
      "Steps: 132000\tLoss: 31488.20898\tPPL: 3.14462\tbleu: 36.35795\tLR: 0.00030000\t*\r\n",
      "Steps: 133000\tLoss: 31434.69922\tPPL: 3.13850\tbleu: 36.09124\tLR: 0.00030000\t*\r\n",
      "Steps: 134000\tLoss: 31535.37695\tPPL: 3.15002\tbleu: 35.53583\tLR: 0.00030000\t\r\n",
      "Steps: 135000\tLoss: 31373.82422\tPPL: 3.13156\tbleu: 36.36092\tLR: 0.00030000\t*\r\n",
      "Steps: 136000\tLoss: 31369.91797\tPPL: 3.13111\tbleu: 35.97234\tLR: 0.00030000\t*\r\n",
      "Steps: 137000\tLoss: 31290.93750\tPPL: 3.12213\tbleu: 35.82189\tLR: 0.00030000\t*\r\n",
      "Steps: 138000\tLoss: 31235.03125\tPPL: 3.11578\tbleu: 35.94173\tLR: 0.00030000\t*\r\n",
      "Steps: 139000\tLoss: 31268.12305\tPPL: 3.11954\tbleu: 35.81351\tLR: 0.00030000\t\r\n",
      "Steps: 140000\tLoss: 31296.95703\tPPL: 3.12281\tbleu: 36.14845\tLR: 0.00030000\t\r\n",
      "Steps: 141000\tLoss: 31286.57812\tPPL: 3.12163\tbleu: 36.42376\tLR: 0.00030000\t\r\n",
      "Steps: 142000\tLoss: 31173.16992\tPPL: 3.10878\tbleu: 35.94043\tLR: 0.00030000\t*\r\n",
      "Steps: 143000\tLoss: 31141.29102\tPPL: 3.10517\tbleu: 36.41295\tLR: 0.00030000\t*\r\n",
      "Steps: 144000\tLoss: 31070.70703\tPPL: 3.09721\tbleu: 35.68332\tLR: 0.00030000\t*\r\n",
      "Steps: 145000\tLoss: 31191.16406\tPPL: 3.11081\tbleu: 36.18333\tLR: 0.00030000\t\r\n",
      "Steps: 146000\tLoss: 31040.19922\tPPL: 3.09377\tbleu: 36.32898\tLR: 0.00030000\t*\r\n",
      "Steps: 147000\tLoss: 31003.89258\tPPL: 3.08969\tbleu: 36.47442\tLR: 0.00030000\t*\r\n",
      "Steps: 148000\tLoss: 31210.75391\tPPL: 3.11303\tbleu: 36.53440\tLR: 0.00030000\t\r\n",
      "Steps: 149000\tLoss: 30965.78906\tPPL: 3.08541\tbleu: 36.24380\tLR: 0.00030000\t*\r\n",
      "Steps: 150000\tLoss: 30939.54492\tPPL: 3.08246\tbleu: 36.32715\tLR: 0.00030000\t*\r\n",
      "Steps: 151000\tLoss: 30992.36133\tPPL: 3.08839\tbleu: 36.40443\tLR: 0.00030000\t\r\n",
      "Steps: 152000\tLoss: 30908.83789\tPPL: 3.07902\tbleu: 36.48690\tLR: 0.00030000\t*\r\n",
      "Steps: 153000\tLoss: 30863.58008\tPPL: 3.07396\tbleu: 36.16061\tLR: 0.00030000\t*\r\n",
      "Steps: 154000\tLoss: 30814.66797\tPPL: 3.06849\tbleu: 36.57391\tLR: 0.00030000\t*\r\n",
      "Steps: 155000\tLoss: 30763.00000\tPPL: 3.06273\tbleu: 36.29865\tLR: 0.00030000\t*\r\n",
      "Steps: 156000\tLoss: 30876.60352\tPPL: 3.07541\tbleu: 36.12984\tLR: 0.00030000\t\r\n",
      "Steps: 157000\tLoss: 30818.56445\tPPL: 3.06893\tbleu: 36.14602\tLR: 0.00030000\t\r\n",
      "Steps: 158000\tLoss: 30851.71289\tPPL: 3.07263\tbleu: 36.55903\tLR: 0.00030000\t\r\n",
      "Steps: 159000\tLoss: 30773.61719\tPPL: 3.06391\tbleu: 36.51544\tLR: 0.00030000\t\r\n",
      "Steps: 160000\tLoss: 30843.44141\tPPL: 3.07170\tbleu: 36.37023\tLR: 0.00030000\t\r\n",
      "Steps: 161000\tLoss: 30862.95703\tPPL: 3.07389\tbleu: 36.45834\tLR: 0.00021000\t\r\n",
      "Steps: 162000\tLoss: 30537.57422\tPPL: 3.03771\tbleu: 36.44355\tLR: 0.00021000\t*\r\n",
      "Steps: 163000\tLoss: 30440.13477\tPPL: 3.02696\tbleu: 36.44646\tLR: 0.00021000\t*\r\n",
      "Steps: 164000\tLoss: 30418.24609\tPPL: 3.02455\tbleu: 36.34867\tLR: 0.00021000\t*\r\n",
      "Steps: 165000\tLoss: 30333.68555\tPPL: 3.01526\tbleu: 36.57710\tLR: 0.00021000\t*\r\n",
      "Steps: 166000\tLoss: 30200.56641\tPPL: 3.00069\tbleu: 36.58072\tLR: 0.00021000\t*\r\n",
      "Steps: 167000\tLoss: 30290.33984\tPPL: 3.01051\tbleu: 36.59814\tLR: 0.00021000\t\r\n",
      "Steps: 168000\tLoss: 30225.70312\tPPL: 3.00343\tbleu: 36.64594\tLR: 0.00021000\t\r\n",
      "Steps: 169000\tLoss: 30177.41992\tPPL: 2.99816\tbleu: 36.61167\tLR: 0.00021000\t*\r\n",
      "Steps: 170000\tLoss: 30123.81250\tPPL: 2.99232\tbleu: 36.48993\tLR: 0.00021000\t*\r\n",
      "Steps: 171000\tLoss: 30197.59180\tPPL: 3.00036\tbleu: 36.56658\tLR: 0.00021000\t\r\n",
      "Steps: 172000\tLoss: 30229.29297\tPPL: 3.00383\tbleu: 36.94808\tLR: 0.00021000\t\r\n",
      "Steps: 173000\tLoss: 30027.27734\tPPL: 2.98183\tbleu: 36.64431\tLR: 0.00021000\t*\r\n",
      "Steps: 174000\tLoss: 30011.84180\tPPL: 2.98015\tbleu: 37.01477\tLR: 0.00021000\t*\r\n",
      "Steps: 175000\tLoss: 29989.25977\tPPL: 2.97771\tbleu: 37.09254\tLR: 0.00021000\t*\r\n",
      "Steps: 176000\tLoss: 29995.85742\tPPL: 2.97842\tbleu: 36.82352\tLR: 0.00021000\t\r\n",
      "Steps: 177000\tLoss: 30005.16992\tPPL: 2.97943\tbleu: 36.76634\tLR: 0.00021000\t\r\n",
      "Steps: 178000\tLoss: 29901.91602\tPPL: 2.96826\tbleu: 37.05637\tLR: 0.00021000\t*\r\n",
      "Steps: 179000\tLoss: 29897.16211\tPPL: 2.96774\tbleu: 37.05351\tLR: 0.00021000\t*\r\n",
      "Steps: 180000\tLoss: 29835.01758\tPPL: 2.96104\tbleu: 37.03724\tLR: 0.00021000\t*\r\n",
      "Steps: 181000\tLoss: 29920.95117\tPPL: 2.97031\tbleu: 36.88921\tLR: 0.00021000\t\r\n",
      "Steps: 182000\tLoss: 29908.97461\tPPL: 2.96902\tbleu: 36.78764\tLR: 0.00021000\t\r\n",
      "Steps: 183000\tLoss: 29935.26758\tPPL: 2.97186\tbleu: 36.93712\tLR: 0.00021000\t\r\n",
      "Steps: 184000\tLoss: 29881.81445\tPPL: 2.96609\tbleu: 36.62247\tLR: 0.00021000\t\r\n",
      "Steps: 185000\tLoss: 29937.57617\tPPL: 2.97211\tbleu: 36.87817\tLR: 0.00021000\t\r\n",
      "Steps: 186000\tLoss: 29682.03711\tPPL: 2.94461\tbleu: 36.53399\tLR: 0.00021000\t*\r\n",
      "Steps: 187000\tLoss: 29816.94531\tPPL: 2.95910\tbleu: 36.93393\tLR: 0.00021000\t\r\n",
      "Steps: 188000\tLoss: 29805.35156\tPPL: 2.95785\tbleu: 37.33005\tLR: 0.00021000\t\r\n",
      "Steps: 189000\tLoss: 29883.98047\tPPL: 2.96632\tbleu: 36.83310\tLR: 0.00021000\t\r\n",
      "Steps: 190000\tLoss: 29882.80078\tPPL: 2.96619\tbleu: 37.01218\tLR: 0.00021000\t\r\n",
      "Steps: 191000\tLoss: 29804.84961\tPPL: 2.95779\tbleu: 36.49870\tLR: 0.00021000\t\r\n",
      "Steps: 192000\tLoss: 29788.22070\tPPL: 2.95600\tbleu: 36.91227\tLR: 0.00014700\t\r\n",
      "Steps: 193000\tLoss: 29647.83203\tPPL: 2.94094\tbleu: 37.22473\tLR: 0.00014700\t*\r\n",
      "Steps: 194000\tLoss: 29573.94141\tPPL: 2.93305\tbleu: 36.90816\tLR: 0.00014700\t*\r\n",
      "Steps: 195000\tLoss: 29530.69531\tPPL: 2.92844\tbleu: 37.14978\tLR: 0.00014700\t*\r\n",
      "Steps: 196000\tLoss: 29436.17969\tPPL: 2.91838\tbleu: 37.14303\tLR: 0.00014700\t*\r\n",
      "Steps: 197000\tLoss: 29434.23828\tPPL: 2.91818\tbleu: 37.09761\tLR: 0.00014700\t*\r\n",
      "Steps: 198000\tLoss: 29360.58984\tPPL: 2.91037\tbleu: 37.16443\tLR: 0.00014700\t*\r\n",
      "Steps: 199000\tLoss: 29496.45703\tPPL: 2.92479\tbleu: 37.13453\tLR: 0.00014700\t\r\n",
      "Steps: 200000\tLoss: 29402.97656\tPPL: 2.91486\tbleu: 37.21013\tLR: 0.00014700\t\r\n",
      "Steps: 201000\tLoss: 29291.97461\tPPL: 2.90311\tbleu: 37.39108\tLR: 0.00014700\t*\r\n",
      "Steps: 202000\tLoss: 29369.01172\tPPL: 2.91126\tbleu: 37.13780\tLR: 0.00014700\t\r\n",
      "Steps: 203000\tLoss: 29327.39453\tPPL: 2.90685\tbleu: 37.45803\tLR: 0.00014700\t\r\n",
      "Steps: 204000\tLoss: 29321.82422\tPPL: 2.90627\tbleu: 37.49813\tLR: 0.00014700\t\r\n",
      "Steps: 205000\tLoss: 29341.52539\tPPL: 2.90835\tbleu: 37.45145\tLR: 0.00014700\t\r\n",
      "Steps: 206000\tLoss: 29334.04102\tPPL: 2.90756\tbleu: 37.43468\tLR: 0.00014700\t\r\n",
      "Steps: 207000\tLoss: 29393.31641\tPPL: 2.91383\tbleu: 37.26231\tLR: 0.00010290\t\r\n",
      "Steps: 208000\tLoss: 29155.85742\tPPL: 2.88877\tbleu: 37.26409\tLR: 0.00010290\t*\r\n",
      "Steps: 209000\tLoss: 29194.03516\tPPL: 2.89278\tbleu: 37.36260\tLR: 0.00010290\t\r\n",
      "Steps: 210000\tLoss: 29146.25977\tPPL: 2.88776\tbleu: 37.39524\tLR: 0.00010290\t*\r\n",
      "Steps: 211000\tLoss: 29122.31055\tPPL: 2.88524\tbleu: 37.01233\tLR: 0.00010290\t*\r\n",
      "Steps: 212000\tLoss: 29096.93359\tPPL: 2.88258\tbleu: 37.23002\tLR: 0.00010290\t*\r\n",
      "Steps: 213000\tLoss: 29072.66406\tPPL: 2.88004\tbleu: 36.99175\tLR: 0.00010290\t*\r\n",
      "Steps: 214000\tLoss: 28987.45703\tPPL: 2.87112\tbleu: 37.21892\tLR: 0.00010290\t*\r\n",
      "Steps: 215000\tLoss: 29014.08789\tPPL: 2.87391\tbleu: 37.18682\tLR: 0.00010290\t\r\n",
      "Steps: 216000\tLoss: 29029.67969\tPPL: 2.87554\tbleu: 37.54762\tLR: 0.00010290\t\r\n",
      "Steps: 217000\tLoss: 28986.49805\tPPL: 2.87102\tbleu: 37.04205\tLR: 0.00010290\t*\r\n",
      "Steps: 218000\tLoss: 28982.62695\tPPL: 2.87062\tbleu: 37.66553\tLR: 0.00010290\t*\r\n",
      "Steps: 219000\tLoss: 28973.81836\tPPL: 2.86970\tbleu: 37.35386\tLR: 0.00010290\t*\r\n",
      "Steps: 220000\tLoss: 28919.06055\tPPL: 2.86399\tbleu: 37.48145\tLR: 0.00010290\t*\r\n",
      "Steps: 221000\tLoss: 28956.25000\tPPL: 2.86786\tbleu: 37.42864\tLR: 0.00010290\t\r\n",
      "Steps: 222000\tLoss: 28910.69531\tPPL: 2.86311\tbleu: 37.58392\tLR: 0.00010290\t*\r\n",
      "Steps: 223000\tLoss: 28949.12891\tPPL: 2.86712\tbleu: 37.17959\tLR: 0.00010290\t\r\n",
      "Steps: 224000\tLoss: 28782.72266\tPPL: 2.84981\tbleu: 37.59428\tLR: 0.00010290\t*\r\n",
      "Steps: 225000\tLoss: 28923.23047\tPPL: 2.86442\tbleu: 37.48888\tLR: 0.00010290\t\r\n",
      "Steps: 226000\tLoss: 28956.22461\tPPL: 2.86786\tbleu: 37.74981\tLR: 0.00010290\t\r\n",
      "Steps: 227000\tLoss: 28897.18945\tPPL: 2.86171\tbleu: 37.43573\tLR: 0.00010290\t\r\n",
      "Steps: 228000\tLoss: 28834.16602\tPPL: 2.85515\tbleu: 37.49122\tLR: 0.00010290\t\r\n",
      "Steps: 229000\tLoss: 28807.19531\tPPL: 2.85235\tbleu: 37.71690\tLR: 0.00010290\t\r\n",
      "Steps: 230000\tLoss: 28813.53516\tPPL: 2.85301\tbleu: 37.66064\tLR: 0.00007203\t\r\n",
      "Steps: 231000\tLoss: 28809.15234\tPPL: 2.85256\tbleu: 37.64625\tLR: 0.00007203\t\r\n",
      "Steps: 232000\tLoss: 28742.60742\tPPL: 2.84566\tbleu: 37.52442\tLR: 0.00007203\t*\r\n",
      "Steps: 233000\tLoss: 28759.66992\tPPL: 2.84742\tbleu: 37.70894\tLR: 0.00007203\t\r\n",
      "Steps: 234000\tLoss: 28694.12891\tPPL: 2.84064\tbleu: 37.54531\tLR: 0.00007203\t*\r\n",
      "Steps: 235000\tLoss: 28671.17773\tPPL: 2.83827\tbleu: 37.58787\tLR: 0.00007203\t*\r\n",
      "Steps: 236000\tLoss: 28682.88477\tPPL: 2.83948\tbleu: 37.94599\tLR: 0.00007203\t\r\n",
      "Steps: 237000\tLoss: 28648.08789\tPPL: 2.83589\tbleu: 37.52206\tLR: 0.00007203\t*\r\n",
      "Steps: 238000\tLoss: 28714.46875\tPPL: 2.84275\tbleu: 37.87414\tLR: 0.00007203\t\r\n",
      "Steps: 239000\tLoss: 28606.39258\tPPL: 2.83159\tbleu: 37.99517\tLR: 0.00007203\t*\r\n",
      "Steps: 240000\tLoss: 28655.81836\tPPL: 2.83669\tbleu: 37.76859\tLR: 0.00007203\t\r\n",
      "Steps: 241000\tLoss: 28657.20312\tPPL: 2.83683\tbleu: 37.75576\tLR: 0.00007203\t\r\n",
      "Steps: 242000\tLoss: 28689.37695\tPPL: 2.84015\tbleu: 37.86473\tLR: 0.00007203\t\r\n",
      "Steps: 243000\tLoss: 28619.05469\tPPL: 2.83289\tbleu: 37.62647\tLR: 0.00007203\t\r\n",
      "Steps: 244000\tLoss: 28617.82031\tPPL: 2.83277\tbleu: 37.56021\tLR: 0.00007203\t\r\n",
      "Steps: 245000\tLoss: 28618.39453\tPPL: 2.83283\tbleu: 37.99256\tLR: 0.00005042\t\r\n",
      "Steps: 246000\tLoss: 28587.62891\tPPL: 2.82966\tbleu: 37.68274\tLR: 0.00005042\t*\r\n",
      "Steps: 247000\tLoss: 28585.28516\tPPL: 2.82942\tbleu: 37.79265\tLR: 0.00005042\t*\r\n",
      "Steps: 248000\tLoss: 28586.14844\tPPL: 2.82950\tbleu: 37.82660\tLR: 0.00005042\t\r\n",
      "Steps: 249000\tLoss: 28608.93164\tPPL: 2.83185\tbleu: 37.69959\tLR: 0.00005042\t\r\n",
      "Steps: 250000\tLoss: 28513.52344\tPPL: 2.82204\tbleu: 37.62150\tLR: 0.00005042\t*\r\n",
      "Steps: 251000\tLoss: 28528.09961\tPPL: 2.82353\tbleu: 37.94771\tLR: 0.00005042\t\r\n",
      "Steps: 252000\tLoss: 28538.84766\tPPL: 2.82464\tbleu: 37.63490\tLR: 0.00005042\t\r\n",
      "Steps: 253000\tLoss: 28498.20703\tPPL: 2.82046\tbleu: 38.04884\tLR: 0.00005042\t*\r\n",
      "Steps: 254000\tLoss: 28569.56641\tPPL: 2.82780\tbleu: 37.89235\tLR: 0.00005042\t\r\n",
      "Steps: 255000\tLoss: 28489.48438\tPPL: 2.81957\tbleu: 37.93596\tLR: 0.00005042\t*\r\n",
      "Steps: 256000\tLoss: 28504.76758\tPPL: 2.82114\tbleu: 37.91740\tLR: 0.00005042\t\r\n",
      "Steps: 257000\tLoss: 28495.87109\tPPL: 2.82023\tbleu: 37.81626\tLR: 0.00005042\t\r\n",
      "Steps: 258000\tLoss: 28509.79297\tPPL: 2.82165\tbleu: 37.79714\tLR: 0.00005042\t\r\n",
      "Steps: 259000\tLoss: 28443.07227\tPPL: 2.81481\tbleu: 37.69474\tLR: 0.00005042\t*\r\n",
      "Steps: 260000\tLoss: 28479.40625\tPPL: 2.81854\tbleu: 37.88885\tLR: 0.00005042\t\r\n",
      "Steps: 261000\tLoss: 28417.95312\tPPL: 2.81224\tbleu: 38.05370\tLR: 0.00005042\t*\r\n",
      "Steps: 262000\tLoss: 28520.15430\tPPL: 2.82272\tbleu: 37.89591\tLR: 0.00005042\t\r\n",
      "Steps: 263000\tLoss: 28451.14453\tPPL: 2.81564\tbleu: 38.02270\tLR: 0.00005042\t\r\n",
      "Steps: 264000\tLoss: 28441.59180\tPPL: 2.81466\tbleu: 37.77965\tLR: 0.00005042\t\r\n",
      "Steps: 265000\tLoss: 28423.28516\tPPL: 2.81279\tbleu: 37.95014\tLR: 0.00005042\t\r\n",
      "Steps: 266000\tLoss: 28401.61719\tPPL: 2.81057\tbleu: 37.82350\tLR: 0.00005042\t*\r\n",
      "Steps: 267000\tLoss: 28432.14062\tPPL: 2.81369\tbleu: 37.77687\tLR: 0.00005042\t\r\n",
      "Steps: 268000\tLoss: 28435.63086\tPPL: 2.81405\tbleu: 38.04545\tLR: 0.00005042\t\r\n",
      "Steps: 269000\tLoss: 28379.73438\tPPL: 2.80833\tbleu: 38.08942\tLR: 0.00005042\t*\r\n",
      "Steps: 270000\tLoss: 28460.49023\tPPL: 2.81660\tbleu: 37.72939\tLR: 0.00005042\t\r\n",
      "Steps: 271000\tLoss: 28435.32812\tPPL: 2.81402\tbleu: 37.81233\tLR: 0.00005042\t\r\n",
      "Steps: 272000\tLoss: 28441.60742\tPPL: 2.81466\tbleu: 37.97019\tLR: 0.00005042\t\r\n",
      "Steps: 273000\tLoss: 28441.76367\tPPL: 2.81468\tbleu: 38.13910\tLR: 0.00005042\t\r\n",
      "Steps: 274000\tLoss: 28441.35352\tPPL: 2.81464\tbleu: 38.22926\tLR: 0.00005042\t\r\n",
      "Steps: 275000\tLoss: 28426.80469\tPPL: 2.81315\tbleu: 37.98489\tLR: 0.00003529\t\r\n",
      "Steps: 276000\tLoss: 28387.63672\tPPL: 2.80914\tbleu: 37.79125\tLR: 0.00003529\t\r\n",
      "Steps: 277000\tLoss: 28370.86523\tPPL: 2.80743\tbleu: 38.10532\tLR: 0.00003529\t*\r\n",
      "Steps: 278000\tLoss: 28399.16016\tPPL: 2.81032\tbleu: 38.11465\tLR: 0.00003529\t\r\n",
      "Steps: 279000\tLoss: 28314.64258\tPPL: 2.80169\tbleu: 37.98826\tLR: 0.00003529\t*\r\n",
      "Steps: 280000\tLoss: 28372.30469\tPPL: 2.80757\tbleu: 37.76615\tLR: 0.00003529\t\r\n",
      "Steps: 281000\tLoss: 28345.11914\tPPL: 2.80480\tbleu: 37.96877\tLR: 0.00003529\t\r\n"
     ]
    }
   ],
   "source": [
    "# Output our validation accuracy\n",
    "! cat \"/home/ks/code/tools/joeynmt/models/${src}${tgt}_filtered_transformer/validations.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "66WhRE9lIhoD"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/home/ks/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "Killed\n"
     ]
    }
   ],
   "source": [
    "# Test our model\n",
    "! cd joeynmt; python3 -m joeynmt test \"/home/ks/code/tools/joeynmt/models/${src}${tgt}_filtered_transformer/config.yaml\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "starter_notebook.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}