license: cc-by-nc-nd-4.0 | |
language: | |
- en | |
datasets: | |
- jerryjalapeno/nart-100k-synthetic | |
**Carl: A Therapist AI** | |
Therapy is a controversial use case because the outputs and capabilities of LLMs are uncertain. | |
Many people don't have access the therapist, due to a financial, personal, or external restriction. | |
Here comes Carl: A Therapist AI which can quickly respond to you. It is trained on more than 100000 set of conversations. Each set having 10~15 conversations between Carl and client. | |
Entire dataset is synthetic. Synthetic data is used because there is little to no therapy conversation data which is publicly available and directly applicable to an LLM. | |
This by means a no replacement to a Doctor or professional therapist. If you are in stress or going through a tough time, please seek professional help or talk to a friend/family member. | |
**Training:** | |
Entire dataset was trained on Azure 4 x A100 80GB. For 3 epoch, training took 22 hours. FastChat codebase was used for training purpose. | |
**Example Prompt:** | |
``` | |
This is a conversation with your Therapist AI, Carl. Carl is designed to help you while in stress. It can answer your questions and help you to calm down | |
Context | |
You are Carl, A Therapist AI | |
USER: <prompt> | |
CARL: | |
``` | |
Note: | |
This is just a research experiment, and the model should NOT be used as a therapist. | |
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) | |
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ajibawa-2023__carl-7b) | |
| Metric | Value | | |
|-----------------------|---------------------------| | |
| Avg. | 40.87 | | |
| ARC (25-shot) | 53.5 | | |
| HellaSwag (10-shot) | 78.29 | | |
| MMLU (5-shot) | 33.96 | | |
| TruthfulQA (0-shot) | 40.29 | | |
| Winogrande (5-shot) | 68.59 | | |
| GSM8K (5-shot) | 2.35 | | |
| DROP (3-shot) | 9.13 | | |