README.md

metadata

datasets:
  - heliosbrahma/mental_health_chatbot_dataset
  - mpingale/mental-health-chat-dataset
library_name: peft
pipeline_tag: text-generation
tags:
  - SFT
  - PEFT
  - Mental Health
  - Psychotherapy
  - Fine-tuning
  - Text Generation
  - Chatbot

Model Card for Model ID

A LORA fine tuned version of Llama 3 8B instruct that is meant to serve you as a outlet to your negative thoughts

• Developed by: John4Blues (Alt account for 9Skies)
• Finetuned from model: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct
• Demo: https://huggingface.co/spaces/John4Blues/Therapy_Llama_3_8B

Risks and Limitations

• By no means is this suppose to replace a therapy counselor, please seek professional help if you believe you need it.

• The responses from the model may not be factually accurate, please double check with other sources when you believe you need to.

How to Get Started with the Model

Use the code below to get started with the model.

The LORA/PEFT has already been merged with the uploaded model.

from transformers import AutoModelForCausalLM
model_id = "John4Blues/Llama-3-8B-Therapy"
model = AutoModelForCausalLM.from_pretrained(model_id)

Training Details

Training Data

1. Amod/mental_health_counseling_conversations
2. mpingale/mental-health-chat-dataset (processed)

Training Procedure

google colab notebook

Training Hyperparameters

• batch_size: 2
• gradient_accumulation_steps: 2
• epochs: 3
• learning_rate: 2e-4
• warmup_ratio: 0.03
• dtype: fp16