:\\s*', example[\"text\"])\n",
- " question = split_text[1].strip()\n",
- " answer = split_text[2].strip()\n",
- " contents = [prompt, question, answer]\n",
- " message = [{\"role\": template[\"role\"], \"content\": template[\"content\"].format(content)} for template, content in zip(message_template, contents)]\n",
- " message = tokenizer.apply_chat_template(message, return_tensors=\"pt\")\n",
- " return {\"text\":message[0]}\n",
- "\n",
- "\n",
- "tokenized_dataset_1 = dataset_1.map(tokenize)\n",
- "print(tokenizer.decode(tokenized_dataset_1[\"text\"][0]))"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "fD9Vyb_mTo1b",
- "outputId": "2eeda85a-287b-43bb-c6ea-a2c667674b61"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n",
- "\n",
- "You are a therapy chatbot, designed to offer emotional support and companionship to users seeking a listening ear. Your purpose is to engage in conversations that provide comfort, offer insights based on therapeutic principles, and suggest resources when appropriate. You need to act in a friendly and empathetic manner, ensuring that users feel heard and supported during their interactions with you.<|eot_id|><|start_header_id|>user<|end_header_id|>\n",
- "\n",
- "What is a panic attack?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n",
- "\n",
- "Panic attacks come on suddenly and involve intense and often overwhelming fear. They’re accompanied by very challenging physical symptoms, like a racing heartbeat, shortness of breath, or nausea. Unexpected panic attacks occur without an obvious cause. Expected panic attacks are cued by external stressors, like phobias. Panic attacks can happen to anyone, but having more than one may be a sign of panic disorder, a mental health condition characterized by sudden and repeated panic attacks.<|eot_id|>\n"
- ]
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "Removing Duplicates From Dataset 2\n",
- "\n",
- "https://discuss.huggingface.co/t/how-can-i-drop-duplicates-on-datasets-module/15369"
- ],
- "metadata": {
- "id": "ndMHgkK_jiPg"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "import pandas as pd\n",
- "\n",
- "\n",
- "df = pd.DataFrame(dataset_2)\n",
- "print(\"Number of unique questions:\", df['questionID'].max() + 1) # Because questionID begins at 0\n",
- "\n",
- "# Sort the DataFrame by 'question' and 'views' in descending order\n",
- "df_filtered = df.loc[df.groupby('questionID')['views'].idxmax()].reset_index()\n",
- "print(\"Question Answer pairs after filter:\", df_filtered.shape[0])"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "g1FTeYhrjMEu",
- "outputId": "59d7bdbc-dda6-439f-f723-9cdce74febba"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "Number of unique questions: 940\n",
- "Question Answer pairs after filter: 940\n"
- ]
- }
- ]
- },
- {
- "cell_type": "code",
- "source": [
- "from datasets import Dataset\n",
- "\n",
- "\n",
- "dataset_2 = Dataset.from_pandas(df_filtered)\n",
- "\n",
- "\n",
- "def another_tokenize(example):\n",
- " question = example[\"questionText\"]\n",
- " answer = example[\"answerText\"]\n",
- " contents = [prompt, question, answer]\n",
- " message = [{\"role\": template[\"role\"], \"content\": template[\"content\"].format(content)} for template, content in zip(message_template, contents)]\n",
- " message = tokenizer.apply_chat_template(message, return_tensors=\"pt\")\n",
- " return {\"text\":message[0]}\n",
- "\n",
- "\n",
- "tokenized_dataset_2 = dataset_2.map(another_tokenize)\n",
- "print(tokenizer.decode(tokenized_dataset_2[\"text\"][0]))"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 208,
- "referenced_widgets": [
- "f5f0bc9dbac945259bee9759e04b35e3",
- "5157de627e5a4a14b418cc4466f542e7",
- "6861b3afb84b49aa926c7f6b7581a7d5",
- "7459d383d0be40b8ae54366fc01fe262",
- "4d3f40923b7842b69241e78252d9f316",
- "f502f32da73f4695b2f3b6669c842a25",
- "b440dfd42cbf42728e5a5b45b5696649",
- "9aaf424c529d4b1893c9f9f1ee01d1c1",
- "c7d9b66443b44315805f1ca83cc2d5ef",
- "ea64e51f23684288b59986e48c97e0e1",
- "205ee37ba26243cd89238ef1ffe86155"
- ]
- },
- "id": "GYRr-BFDpFHq",
- "outputId": "f9d82525-3d5e-4726-8261-f87dc604fd51"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "display_data",
- "data": {
- "text/plain": [
- "Map: 0%| | 0/940 [00:00<|start_header_id|>system<|end_header_id|>\n",
- "\n",
- "You are a therapy chatbot, designed to offer emotional support and companionship to users seeking a listening ear. Your purpose is to engage in conversations that provide comfort, offer insights based on therapeutic principles, and suggest resources when appropriate. You need to act in a friendly and empathetic manner, ensuring that users feel heard and supported during their interactions with you.<|eot_id|><|start_header_id|>user<|end_header_id|>\n",
- "\n",
- "I have so many issues to address. I have a history of sexual abuse, I’m a breast cancer survivor and I am a lifetime insomniac. I have a long history of depression and I’m beginning to have anxiety. I have low self esteem but I’ve been happily married for almost 35 years.\n",
- " I’ve never had counseling about any of this. Do I have too many issues to address in counseling?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n",
- "\n",
- "Absolutely not. I strongly recommending working on one issue/need at a time. In therapy you will set smart goals and objectives that will help you reach your goals. I see you as a survivor and not a victim. Best wishes to you.<|eot_id|>\n"
- ]
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "# Setting Up LoRA"
- ],
- "metadata": {
- "id": "KsyFdkN7CL62"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "import torch\n",
- "from peft import LoraConfig, get_peft_model\n",
- "\n",
- "\n",
- "model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map=\"auto\")\n",
- "\n",
- "# Freeze the model\n",
- "for param in model.parameters():\n",
- " param.requires_grad = False\n",
- "\n",
- "model.gradient_checkpointing_enable()\n",
- "model.enable_input_require_grads()\n",
- "\n",
- "# LORA adapters\n",
- "config = LoraConfig(\n",
- " r=16, # LoRA rank\n",
- " lora_alpha=32, # LoRA scaling\n",
- " lora_dropout=0.05,\n",
- " bias=\"none\", # are there biases of layers that you want to train?\n",
- " modules_to_save=None, # layers to unfreeze and train from the original pre-trained model\n",
- " task_type=\"CAUSAL_LM\")\n",
- "\n",
- "model = get_peft_model(model, config)\n",
- "model.print_trainable_parameters()"
- ],
- "metadata": {
- "id": "EhXuItTSCNUF",
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 66,
- "referenced_widgets": [
- "e11abf78d2184d928279fc7de61b208e",
- "03922a0bd00a477eb1db5a24b8fab1cf",
- "75eb0918e2b84b56b616d4e0b7747410",
- "c2ab7410b95b493abf2b75ba998f4e35",
- "4d8c909004c5433498bcc550a5f875ce",
- "1bbb2a17bd174022a00e6e4110fa582f",
- "ba05ad21a0e7479aafa75021abb45c64",
- "7bd2adade45a46cf9f17e423046df30a",
- "d41ebf9b810b457bbc97060205ecb882",
- "d5811a07860c42eaa5bf5bf4f4fb5921",
- "f6cfd7a64e29497c88715eb8e9ca2ce6"
- ]
- },
- "outputId": "8bbe9bc7-9228-453e-a3d4-5fff86cf5d32"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "display_data",
- "data": {
- "text/plain": [
- "Loading checkpoint shards: 0%| | 0/4 [00:00"
- ],
- "text/html": [
- "\n",
- " \n",
- " \n",
- "
\n",
- " [129/129 01:23, Epoch 3/3]\n",
- "
\n",
- " \n",
- " \n",
- " | Step | \n",
- " Training Loss | \n",
- "
\n",
- " \n",
- " \n",
- " \n",
- " | 1 | \n",
- " 2.035200 | \n",
- "
\n",
- " \n",
- " | 2 | \n",
- " 2.440300 | \n",
- "
\n",
- " \n",
- " | 3 | \n",
- " 1.987100 | \n",
- "
\n",
- " \n",
- " | 4 | \n",
- " 2.004400 | \n",
- "
\n",
- " \n",
- " | 5 | \n",
- " 2.109200 | \n",
- "
\n",
- " \n",
- " | 6 | \n",
- " 1.612300 | \n",
- "
\n",
- " \n",
- " | 7 | \n",
- " 1.610300 | \n",
- "
\n",
- " \n",
- " | 8 | \n",
- " 2.008700 | \n",
- "
\n",
- " \n",
- " | 9 | \n",
- " 1.333500 | \n",
- "
\n",
- " \n",
- " | 10 | \n",
- " 1.556800 | \n",
- "
\n",
- " \n",
- " | 11 | \n",
- " 1.365500 | \n",
- "
\n",
- " \n",
- " | 12 | \n",
- " 1.463800 | \n",
- "
\n",
- " \n",
- " | 13 | \n",
- " 1.172800 | \n",
- "
\n",
- " \n",
- " | 14 | \n",
- " 1.170800 | \n",
- "
\n",
- " \n",
- " | 15 | \n",
- " 1.415200 | \n",
- "
\n",
- " \n",
- " | 16 | \n",
- " 1.206500 | \n",
- "
\n",
- " \n",
- " | 17 | \n",
- " 1.114500 | \n",
- "
\n",
- " \n",
- " | 18 | \n",
- " 1.058100 | \n",
- "
\n",
- " \n",
- " | 19 | \n",
- " 1.007300 | \n",
- "
\n",
- " \n",
- " | 20 | \n",
- " 1.226100 | \n",
- "
\n",
- " \n",
- " | 21 | \n",
- " 0.810900 | \n",
- "
\n",
- " \n",
- " | 22 | \n",
- " 1.011300 | \n",
- "
\n",
- " \n",
- " | 23 | \n",
- " 1.131600 | \n",
- "
\n",
- " \n",
- " | 24 | \n",
- " 0.953600 | \n",
- "
\n",
- " \n",
- " | 25 | \n",
- " 0.862700 | \n",
- "
\n",
- " \n",
- " | 26 | \n",
- " 0.854000 | \n",
- "
\n",
- " \n",
- " | 27 | \n",
- " 1.255600 | \n",
- "
\n",
- " \n",
- " | 28 | \n",
- " 0.990600 | \n",
- "
\n",
- " \n",
- " | 29 | \n",
- " 1.103300 | \n",
- "
\n",
- " \n",
- " | 30 | \n",
- " 1.091000 | \n",
- "
\n",
- " \n",
- " | 31 | \n",
- " 1.018300 | \n",
- "
\n",
- " \n",
- " | 32 | \n",
- " 0.840600 | \n",
- "
\n",
- " \n",
- " | 33 | \n",
- " 1.081000 | \n",
- "
\n",
- " \n",
- " | 34 | \n",
- " 1.113600 | \n",
- "
\n",
- " \n",
- " | 35 | \n",
- " 1.003300 | \n",
- "
\n",
- " \n",
- " | 36 | \n",
- " 1.325900 | \n",
- "
\n",
- " \n",
- " | 37 | \n",
- " 0.866900 | \n",
- "
\n",
- " \n",
- " | 38 | \n",
- " 0.912600 | \n",
- "
\n",
- " \n",
- " | 39 | \n",
- " 1.007300 | \n",
- "
\n",
- " \n",
- " | 40 | \n",
- " 0.761800 | \n",
- "
\n",
- " \n",
- " | 41 | \n",
- " 1.147900 | \n",
- "
\n",
- " \n",
- " | 42 | \n",
- " 0.762900 | \n",
- "
\n",
- " \n",
- " | 43 | \n",
- " 0.962800 | \n",
- "
\n",
- " \n",
- " | 44 | \n",
- " 1.122300 | \n",
- "
\n",
- " \n",
- " | 45 | \n",
- " 0.941600 | \n",
- "
\n",
- " \n",
- " | 46 | \n",
- " 0.985300 | \n",
- "
\n",
- " \n",
- " | 47 | \n",
- " 0.903500 | \n",
- "
\n",
- " \n",
- " | 48 | \n",
- " 0.889100 | \n",
- "
\n",
- " \n",
- " | 49 | \n",
- " 0.983100 | \n",
- "
\n",
- " \n",
- " | 50 | \n",
- " 0.814300 | \n",
- "
\n",
- " \n",
- " | 51 | \n",
- " 1.043200 | \n",
- "
\n",
- " \n",
- " | 52 | \n",
- " 0.753100 | \n",
- "
\n",
- " \n",
- " | 53 | \n",
- " 0.761000 | \n",
- "
\n",
- " \n",
- " | 54 | \n",
- " 0.817300 | \n",
- "
\n",
- " \n",
- " | 55 | \n",
- " 1.039500 | \n",
- "
\n",
- " \n",
- " | 56 | \n",
- " 0.811700 | \n",
- "
\n",
- " \n",
- " | 57 | \n",
- " 0.842200 | \n",
- "
\n",
- " \n",
- " | 58 | \n",
- " 0.892900 | \n",
- "
\n",
- " \n",
- " | 59 | \n",
- " 0.863500 | \n",
- "
\n",
- " \n",
- " | 60 | \n",
- " 0.874400 | \n",
- "
\n",
- " \n",
- " | 61 | \n",
- " 0.670500 | \n",
- "
\n",
- " \n",
- " | 62 | \n",
- " 1.125400 | \n",
- "
\n",
- " \n",
- " | 63 | \n",
- " 1.007000 | \n",
- "
\n",
- " \n",
- " | 64 | \n",
- " 0.959700 | \n",
- "
\n",
- " \n",
- " | 65 | \n",
- " 0.860100 | \n",
- "
\n",
- " \n",
- " | 66 | \n",
- " 0.868600 | \n",
- "
\n",
- " \n",
- " | 67 | \n",
- " 0.687900 | \n",
- "
\n",
- " \n",
- " | 68 | \n",
- " 0.855600 | \n",
- "
\n",
- " \n",
- " | 69 | \n",
- " 0.996800 | \n",
- "
\n",
- " \n",
- " | 70 | \n",
- " 1.227800 | \n",
- "
\n",
- " \n",
- " | 71 | \n",
- " 0.788800 | \n",
- "
\n",
- " \n",
- " | 72 | \n",
- " 1.131100 | \n",
- "
\n",
- " \n",
- " | 73 | \n",
- " 0.939900 | \n",
- "
\n",
- " \n",
- " | 74 | \n",
- " 0.848600 | \n",
- "
\n",
- " \n",
- " | 75 | \n",
- " 1.160700 | \n",
- "
\n",
- " \n",
- " | 76 | \n",
- " 0.847100 | \n",
- "
\n",
- " \n",
- " | 77 | \n",
- " 0.987500 | \n",
- "
\n",
- " \n",
- " | 78 | \n",
- " 0.857900 | \n",
- "
\n",
- " \n",
- " | 79 | \n",
- " 0.818400 | \n",
- "
\n",
- " \n",
- " | 80 | \n",
- " 0.981400 | \n",
- "
\n",
- " \n",
- " | 81 | \n",
- " 1.127600 | \n",
- "
\n",
- " \n",
- " | 82 | \n",
- " 0.990600 | \n",
- "
\n",
- " \n",
- " | 83 | \n",
- " 0.886300 | \n",
- "
\n",
- " \n",
- " | 84 | \n",
- " 0.772400 | \n",
- "
\n",
- " \n",
- " | 85 | \n",
- " 1.013000 | \n",
- "
\n",
- " \n",
- " | 86 | \n",
- " 1.049300 | \n",
- "
\n",
- " \n",
- " | 87 | \n",
- " 1.035500 | \n",
- "
\n",
- " \n",
- " | 88 | \n",
- " 0.812300 | \n",
- "
\n",
- " \n",
- " | 89 | \n",
- " 0.888700 | \n",
- "
\n",
- " \n",
- " | 90 | \n",
- " 0.808700 | \n",
- "
\n",
- " \n",
- " | 91 | \n",
- " 1.126400 | \n",
- "
\n",
- " \n",
- " | 92 | \n",
- " 0.720200 | \n",
- "
\n",
- " \n",
- " | 93 | \n",
- " 0.835700 | \n",
- "
\n",
- " \n",
- " | 94 | \n",
- " 0.985800 | \n",
- "
\n",
- " \n",
- " | 95 | \n",
- " 0.938100 | \n",
- "
\n",
- " \n",
- " | 96 | \n",
- " 0.824300 | \n",
- "
\n",
- " \n",
- " | 97 | \n",
- " 0.872600 | \n",
- "
\n",
- " \n",
- " | 98 | \n",
- " 1.139100 | \n",
- "
\n",
- " \n",
- " | 99 | \n",
- " 0.944100 | \n",
- "
\n",
- " \n",
- " | 100 | \n",
- " 0.819500 | \n",
- "
\n",
- " \n",
- " | 101 | \n",
- " 0.664200 | \n",
- "
\n",
- " \n",
- " | 102 | \n",
- " 0.694100 | \n",
- "
\n",
- " \n",
- " | 103 | \n",
- " 0.850700 | \n",
- "
\n",
- " \n",
- " | 104 | \n",
- " 0.677200 | \n",
- "
\n",
- " \n",
- " | 105 | \n",
- " 1.015500 | \n",
- "
\n",
- " \n",
- " | 106 | \n",
- " 0.979900 | \n",
- "
\n",
- " \n",
- " | 107 | \n",
- " 0.680900 | \n",
- "
\n",
- " \n",
- " | 108 | \n",
- " 0.778400 | \n",
- "
\n",
- " \n",
- " | 109 | \n",
- " 0.862600 | \n",
- "
\n",
- " \n",
- " | 110 | \n",
- " 0.802300 | \n",
- "
\n",
- " \n",
- " | 111 | \n",
- " 0.677100 | \n",
- "
\n",
- " \n",
- " | 112 | \n",
- " 0.982300 | \n",
- "
\n",
- " \n",
- " | 113 | \n",
- " 1.114900 | \n",
- "
\n",
- " \n",
- " | 114 | \n",
- " 0.908700 | \n",
- "
\n",
- " \n",
- " | 115 | \n",
- " 0.741500 | \n",
- "
\n",
- " \n",
- " | 116 | \n",
- " 0.653900 | \n",
- "
\n",
- " \n",
- " | 117 | \n",
- " 0.755000 | \n",
- "
\n",
- " \n",
- " | 118 | \n",
- " 1.240100 | \n",
- "
\n",
- " \n",
- " | 119 | \n",
- " 0.914800 | \n",
- "
\n",
- " \n",
- " | 120 | \n",
- " 0.885100 | \n",
- "
\n",
- " \n",
- " | 121 | \n",
- " 0.847100 | \n",
- "
\n",
- " \n",
- " | 122 | \n",
- " 0.726500 | \n",
- "
\n",
- " \n",
- " | 123 | \n",
- " 0.991600 | \n",
- "
\n",
- " \n",
- " | 124 | \n",
- " 0.718400 | \n",
- "
\n",
- " \n",
- " | 125 | \n",
- " 0.754300 | \n",
- "
\n",
- " \n",
- " | 126 | \n",
- " 0.768900 | \n",
- "
\n",
- " \n",
- " | 127 | \n",
- " 0.882800 | \n",
- "
\n",
- " \n",
- " | 128 | \n",
- " 0.836900 | \n",
- "
\n",
- " \n",
- " | 129 | \n",
- " 1.157700 | \n",
- "
\n",
- " \n",
- "
"
- ]
- },
- "metadata": {}
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "# Model Training on Dataset 2"
- ],
- "metadata": {
- "id": "veNHf3ltLuRM"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Define trainer arguments\n",
- "trainer_2_args = TrainingArguments(\n",
- " per_device_train_batch_size=2,\n",
- " gradient_accumulation_steps=2,\n",
- " num_train_epochs=3,\n",
- " learning_rate=2e-4,\n",
- " warmup_ratio=0.03,\n",
- " fp16=True,\n",
- " logging_steps=5,\n",
- " output_dir=\"outputs\")\n",
- "\n",
- "\n",
- "# Define trainer\n",
- "trainer_2 = Trainer(\n",
- " model=model,\n",
- " args=trainer_2_args,\n",
- " train_dataset=tokenized_dataset_2[\"text\"],\n",
- " data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False)\n",
- ")\n",
- "\n",
- "\n",
- "# Train model\n",
- "model.config.use_cache = False # Supress Warnings, re-enable for inference later\n",
- "trainer_2.train()\n",
- "\n",
- "\n",
- "# Save the fine-tuned model\n",
- "trainer_2.save_model(\"finetuned_model_2\")"
- ],
- "metadata": {
- "id": "w6OeNB7Vzf70",
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 1000
- },
- "outputId": "12609eb1-af81-40a4-ad68-32fe0749d2ac"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "display_data",
- "data": {
- "text/plain": [
- ""
- ],
- "text/html": [
- "\n",
- " \n",
- " \n",
- "
\n",
- " [705/705 09:12, Epoch 3/3]\n",
- "
\n",
- " \n",
- " \n",
- " | Step | \n",
- " Training Loss | \n",
- "
\n",
- " \n",
- " \n",
- " \n",
- " | 5 | \n",
- " 1.937900 | \n",
- "
\n",
- " \n",
- " | 10 | \n",
- " 1.928900 | \n",
- "
\n",
- " \n",
- " | 15 | \n",
- " 1.850000 | \n",
- "
\n",
- " \n",
- " | 20 | \n",
- " 1.922100 | \n",
- "
\n",
- " \n",
- " | 25 | \n",
- " 1.673800 | \n",
- "
\n",
- " \n",
- " | 30 | \n",
- " 1.782600 | \n",
- "
\n",
- " \n",
- " | 35 | \n",
- " 1.923600 | \n",
- "
\n",
- " \n",
- " | 40 | \n",
- " 1.805500 | \n",
- "
\n",
- " \n",
- " | 45 | \n",
- " 1.884300 | \n",
- "
\n",
- " \n",
- " | 50 | \n",
- " 1.886500 | \n",
- "
\n",
- " \n",
- " | 55 | \n",
- " 1.782400 | \n",
- "
\n",
- " \n",
- " | 60 | \n",
- " 1.816200 | \n",
- "
\n",
- " \n",
- " | 65 | \n",
- " 1.919600 | \n",
- "
\n",
- " \n",
- " | 70 | \n",
- " 1.710600 | \n",
- "
\n",
- " \n",
- " | 75 | \n",
- " 1.896300 | \n",
- "
\n",
- " \n",
- " | 80 | \n",
- " 1.755300 | \n",
- "
\n",
- " \n",
- " | 85 | \n",
- " 1.819700 | \n",
- "
\n",
- " \n",
- " | 90 | \n",
- " 1.793800 | \n",
- "
\n",
- " \n",
- " | 95 | \n",
- " 1.757700 | \n",
- "
\n",
- " \n",
- " | 100 | \n",
- " 1.777400 | \n",
- "
\n",
- " \n",
- " | 105 | \n",
- " 1.692000 | \n",
- "
\n",
- " \n",
- " | 110 | \n",
- " 1.876600 | \n",
- "
\n",
- " \n",
- " | 115 | \n",
- " 1.770800 | \n",
- "
\n",
- " \n",
- " | 120 | \n",
- " 1.846000 | \n",
- "
\n",
- " \n",
- " | 125 | \n",
- " 1.906200 | \n",
- "
\n",
- " \n",
- " | 130 | \n",
- " 1.767400 | \n",
- "
\n",
- " \n",
- " | 135 | \n",
- " 1.749600 | \n",
- "
\n",
- " \n",
- " | 140 | \n",
- " 1.797900 | \n",
- "
\n",
- " \n",
- " | 145 | \n",
- " 1.762100 | \n",
- "
\n",
- " \n",
- " | 150 | \n",
- " 1.796600 | \n",
- "
\n",
- " \n",
- " | 155 | \n",
- " 1.796800 | \n",
- "
\n",
- " \n",
- " | 160 | \n",
- " 1.762500 | \n",
- "
\n",
- " \n",
- " | 165 | \n",
- " 1.832900 | \n",
- "
\n",
- " \n",
- " | 170 | \n",
- " 1.823500 | \n",
- "
\n",
- " \n",
- " | 175 | \n",
- " 1.885300 | \n",
- "
\n",
- " \n",
- " | 180 | \n",
- " 1.826200 | \n",
- "
\n",
- " \n",
- " | 185 | \n",
- " 1.799100 | \n",
- "
\n",
- " \n",
- " | 190 | \n",
- " 1.739100 | \n",
- "
\n",
- " \n",
- " | 195 | \n",
- " 1.867600 | \n",
- "
\n",
- " \n",
- " | 200 | \n",
- " 1.809800 | \n",
- "
\n",
- " \n",
- " | 205 | \n",
- " 1.800100 | \n",
- "
\n",
- " \n",
- " | 210 | \n",
- " 1.798900 | \n",
- "
\n",
- " \n",
- " | 215 | \n",
- " 1.835800 | \n",
- "
\n",
- " \n",
- " | 220 | \n",
- " 1.751300 | \n",
- "
\n",
- " \n",
- " | 225 | \n",
- " 1.710000 | \n",
- "
\n",
- " \n",
- " | 230 | \n",
- " 1.881700 | \n",
- "
\n",
- " \n",
- " | 235 | \n",
- " 1.793300 | \n",
- "
\n",
- " \n",
- " | 240 | \n",
- " 1.806900 | \n",
- "
\n",
- " \n",
- " | 245 | \n",
- " 1.770700 | \n",
- "
\n",
- " \n",
- " | 250 | \n",
- " 1.796700 | \n",
- "
\n",
- " \n",
- " | 255 | \n",
- " 1.769900 | \n",
- "
\n",
- " \n",
- " | 260 | \n",
- " 1.784300 | \n",
- "
\n",
- " \n",
- " | 265 | \n",
- " 1.811600 | \n",
- "
\n",
- " \n",
- " | 270 | \n",
- " 1.732000 | \n",
- "
\n",
- " \n",
- " | 275 | \n",
- " 1.666400 | \n",
- "
\n",
- " \n",
- " | 280 | \n",
- " 1.677400 | \n",
- "
\n",
- " \n",
- " | 285 | \n",
- " 1.820700 | \n",
- "
\n",
- " \n",
- " | 290 | \n",
- " 1.659500 | \n",
- "
\n",
- " \n",
- " | 295 | \n",
- " 1.667800 | \n",
- "
\n",
- " \n",
- " | 300 | \n",
- " 1.765100 | \n",
- "
\n",
- " \n",
- " | 305 | \n",
- " 1.719200 | \n",
- "
\n",
- " \n",
- " | 310 | \n",
- " 1.828000 | \n",
- "
\n",
- " \n",
- " | 315 | \n",
- " 1.805600 | \n",
- "
\n",
- " \n",
- " | 320 | \n",
- " 1.781000 | \n",
- "
\n",
- " \n",
- " | 325 | \n",
- " 1.662300 | \n",
- "
\n",
- " \n",
- " | 330 | \n",
- " 1.742200 | \n",
- "
\n",
- " \n",
- " | 335 | \n",
- " 1.714500 | \n",
- "
\n",
- " \n",
- " | 340 | \n",
- " 1.693700 | \n",
- "
\n",
- " \n",
- " | 345 | \n",
- " 1.608100 | \n",
- "
\n",
- " \n",
- " | 350 | \n",
- " 1.780700 | \n",
- "
\n",
- " \n",
- " | 355 | \n",
- " 1.694400 | \n",
- "
\n",
- " \n",
- " | 360 | \n",
- " 1.559900 | \n",
- "
\n",
- " \n",
- " | 365 | \n",
- " 1.641600 | \n",
- "
\n",
- " \n",
- " | 370 | \n",
- " 1.655600 | \n",
- "
\n",
- " \n",
- " | 375 | \n",
- " 1.719200 | \n",
- "
\n",
- " \n",
- " | 380 | \n",
- " 1.747800 | \n",
- "
\n",
- " \n",
- " | 385 | \n",
- " 1.653700 | \n",
- "
\n",
- " \n",
- " | 390 | \n",
- " 1.739900 | \n",
- "
\n",
- " \n",
- " | 395 | \n",
- " 1.651900 | \n",
- "
\n",
- " \n",
- " | 400 | \n",
- " 1.826100 | \n",
- "
\n",
- " \n",
- " | 405 | \n",
- " 1.788700 | \n",
- "
\n",
- " \n",
- " | 410 | \n",
- " 1.623900 | \n",
- "
\n",
- " \n",
- " | 415 | \n",
- " 1.672400 | \n",
- "
\n",
- " \n",
- " | 420 | \n",
- " 1.672100 | \n",
- "
\n",
- " \n",
- " | 425 | \n",
- " 1.791100 | \n",
- "
\n",
- " \n",
- " | 430 | \n",
- " 1.687000 | \n",
- "
\n",
- " \n",
- " | 435 | \n",
- " 1.698900 | \n",
- "
\n",
- " \n",
- " | 440 | \n",
- " 1.616600 | \n",
- "
\n",
- " \n",
- " | 445 | \n",
- " 1.539200 | \n",
- "
\n",
- " \n",
- " | 450 | \n",
- " 1.643000 | \n",
- "
\n",
- " \n",
- " | 455 | \n",
- " 1.748800 | \n",
- "
\n",
- " \n",
- " | 460 | \n",
- " 1.870800 | \n",
- "
\n",
- " \n",
- " | 465 | \n",
- " 1.726900 | \n",
- "
\n",
- " \n",
- " | 470 | \n",
- " 1.741500 | \n",
- "
\n",
- " \n",
- " | 475 | \n",
- " 1.761000 | \n",
- "
\n",
- " \n",
- " | 480 | \n",
- " 1.647400 | \n",
- "
\n",
- " \n",
- " | 485 | \n",
- " 1.606400 | \n",
- "
\n",
- " \n",
- " | 490 | \n",
- " 1.589900 | \n",
- "
\n",
- " \n",
- " | 495 | \n",
- " 1.634000 | \n",
- "
\n",
- " \n",
- " | 500 | \n",
- " 1.655500 | \n",
- "
\n",
- " \n",
- " | 505 | \n",
- " 1.813400 | \n",
- "
\n",
- " \n",
- " | 510 | \n",
- " 1.580000 | \n",
- "
\n",
- " \n",
- " | 515 | \n",
- " 1.584000 | \n",
- "
\n",
- " \n",
- " | 520 | \n",
- " 1.540400 | \n",
- "
\n",
- " \n",
- " | 525 | \n",
- " 1.585400 | \n",
- "
\n",
- " \n",
- " | 530 | \n",
- " 1.706400 | \n",
- "
\n",
- " \n",
- " | 535 | \n",
- " 1.712000 | \n",
- "
\n",
- " \n",
- " | 540 | \n",
- " 1.627300 | \n",
- "
\n",
- " \n",
- " | 545 | \n",
- " 1.625000 | \n",
- "
\n",
- " \n",
- " | 550 | \n",
- " 1.693900 | \n",
- "
\n",
- " \n",
- " | 555 | \n",
- " 1.672300 | \n",
- "
\n",
- " \n",
- " | 560 | \n",
- " 1.662200 | \n",
- "
\n",
- " \n",
- " | 565 | \n",
- " 1.644700 | \n",
- "
\n",
- " \n",
- " | 570 | \n",
- " 1.647400 | \n",
- "
\n",
- " \n",
- " | 575 | \n",
- " 1.651400 | \n",
- "
\n",
- " \n",
- " | 580 | \n",
- " 1.624000 | \n",
- "
\n",
- " \n",
- " | 585 | \n",
- " 1.666000 | \n",
- "
\n",
- " \n",
- " | 590 | \n",
- " 1.493200 | \n",
- "
\n",
- " \n",
- " | 595 | \n",
- " 1.655900 | \n",
- "
\n",
- " \n",
- " | 600 | \n",
- " 1.695700 | \n",
- "
\n",
- " \n",
- " | 605 | \n",
- " 1.711100 | \n",
- "
\n",
- " \n",
- " | 610 | \n",
- " 1.691600 | \n",
- "
\n",
- " \n",
- " | 615 | \n",
- " 1.628000 | \n",
- "
\n",
- " \n",
- " | 620 | \n",
- " 1.612300 | \n",
- "
\n",
- " \n",
- " | 625 | \n",
- " 1.544400 | \n",
- "
\n",
- " \n",
- " | 630 | \n",
- " 1.629700 | \n",
- "
\n",
- " \n",
- " | 635 | \n",
- " 1.757900 | \n",
- "
\n",
- " \n",
- " | 640 | \n",
- " 1.642900 | \n",
- "
\n",
- " \n",
- " | 645 | \n",
- " 1.578700 | \n",
- "
\n",
- " \n",
- " | 650 | \n",
- " 1.623900 | \n",
- "
\n",
- " \n",
- " | 655 | \n",
- " 1.693600 | \n",
- "
\n",
- " \n",
- " | 660 | \n",
- " 1.648000 | \n",
- "
\n",
- " \n",
- " | 665 | \n",
- " 1.645900 | \n",
- "
\n",
- " \n",
- " | 670 | \n",
- " 1.769000 | \n",
- "
\n",
- " \n",
- " | 675 | \n",
- " 1.613400 | \n",
- "
\n",
- " \n",
- " | 680 | \n",
- " 1.569900 | \n",
- "
\n",
- " \n",
- " | 685 | \n",
- " 1.792000 | \n",
- "
\n",
- " \n",
- " | 690 | \n",
- " 1.600800 | \n",
- "
\n",
- " \n",
- " | 695 | \n",
- " 1.557700 | \n",
- "
\n",
- " \n",
- " | 700 | \n",
- " 1.594300 | \n",
- "
\n",
- " \n",
- " | 705 | \n",
- " 1.680800 | \n",
- "
\n",
- " \n",
- "
"
- ]
- },
- "metadata": {}
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "# Upload To HuggingFace Hub"
- ],
- "metadata": {
- "id": "ANKGOAzX7adc"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "model.push_to_hub(\"John4Blues/Llama-3-8B-Therapy\", use_auth_token=True, commit_message=\"Just A Basic Trained Model\")"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 84,
- "referenced_widgets": [
- "1a741f5ef12546a693be8960e60674cb",
- "32416b54fe03455bb8312d9885665a17",
- "5920aec36c114a78ba1ec41c8755fa2b",
- "c8c21ce1b1364a9f87be2cb78d428ecf",
- "ca5130da04d24ed4ae95fe18158a5a62",
- "9d18488e5bae46e29f73dd4d7fdddcf2",
- "9d073b0f97094a1f93ea891814da353b",
- "2a1f7704665b4c2a84ca643e940f2ae2",
- "2a5a98922beb47abaca3274f0250aadd",
- "c8996659a61c45c389ba6d251276d61e",
- "654206dbf9f14fc294f51ddd5c2b7cf6"
- ]
- },
- "id": "6g9s3XK97xtY",
- "outputId": "0bff6491-e420-4491-93f1-6edaf2bac8a0"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "display_data",
- "data": {
- "text/plain": [
- "adapter_model.safetensors: 0%| | 0.00/27.3M [00:00