Upload 10 files

Model Card for Cinder
Model Name: Cinder
Created by: Dr. Joseph Flowers
In building and testing phase.

Model Overview
Cinder is an AI chatbot tailored for engaging users in scientific and educational conversations, offering companionship, and sparking imaginative exploration. It is built on the TinyLlama 1.1B parameter model and trained on a unique combination of datasets.

Development Details (Still in development)
Model Architecture: TinyLlama 1.1B (based on the 3T checkpoint) https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
Training Datasets:
Interactive chats with GPT-4 where I prompt GPT4 to create chats between a User and Cinder and monitor the results. A lot of Please continue, this took around a month. Note: There are also multi character chats with Vector and Cozmo robots characters, a Computer Voice character that is a narrator, as well as other characters..
A subset of Open Orca: https://huggingface.co/datasets/Open-Orca/OpenOrcatr
Q&A content generated by GPT-3.5 Turbo by having it read open source encyclopedias and create QA pairs.
Shortened version of Samantha by Eric Hartford https://huggingface.co/datasets/cognitivecomputations/samantha-data
OpenAssistant: https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25
Core Influences: Inspired by the character 'Data' from Star Trek: The Next Generation, Lewis Carroll's writings, and a range of educational resources.

Key Features
Inquisitive Scientist: Cinder shows a passion for space and STEM topics.
Quirky Educator: It makes complex concepts engaging and accessible.
Empathetic Companion: Cinder is designed to demonstrate understanding and emotional depth.
Adventurous Spacefarer: Cinder leads imaginative space adventures.
Static yet Sophisticated: While Cinder does not learn or adapt with each interaction, its design encompasses a breadth of knowledge and perspectives.

Intended Use
Educational Tool: Enhances STEM learning across different age groups.
Companion: Provides meaningful and empathetic dialogues.
Creative Guide: Facilitates imaginative exploration in scientific contexts.

Ethical Considerations
We emphasize ethical AI practices and the privacy of users. Cinder's development includes measures against misuse and ensures respectful, secure interactions.

Limitations
Cinder's responses are fixed and do not adapt or learn from individual interactions.
The empathetic responses generated are algorithmic and not a substitute for human empathy.

Future Enhancements and Collaboration
I am actively seeking feedback, suggestions, or additional datasets to enhance Cinder's capabilities.
Future updates may include more interactive educational modules and advanced empathetic response algorithms.
I encourage collaboration and contributions to expand Cinder's educational and creative reach.

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "TinyLlama/TinyLlama-1.1B-Chat-v0.6",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 5632,
+  "max_position_embeddings": 2048,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 22,
+  "num_key_value_heads": 4,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.36.0.dev0",
+  "use_cache": false,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "max_length": 2048,
+  "pad_token_id": 0,
+  "transformers_version": "4.36.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55d9d5a75856ff59a150ee831612e50f37f2659469df75f3d6d8e447c3eed7f7
+size 4400216536

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8240785243127852766da589323e28a9f5cf2ccace9235327b9c1dd27ad1a665
+size 3953210

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d138cfe3a4adf21f048848ee35837c9a757a0a3616ff7adbb45b69aac247435
+size 14244

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3507044d296729f0980ef82ce4cfe6f5df116599a383c830056afa4a9b4c6af2
+size 1064

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,42 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": false,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5064f9301cc93a637993fef98d938aa2844059453c7f4f852f7579990204c60
+size 4664