Upload folder using huggingface_hub

README.md

@@ -1,3 +1,114 @@
----
-license: mit
----
+---
+tags:
+- pytorch
+- sentiment-analysis
+- yoruba
+- cnn
+- afriberta
+---
+
+# Yoruba Sentiment Analysis with CNN and Afriberta
+
+This repository contains a PyTorch model for sentiment analysis of Yoruba text. The model utilizes a Convolutional Neural Network (CNN) architecture on top of a pre-trained Afriberta model, specifically "Davlan/naija-twitter-sentiment-afriberta-large".
+
+## Model Description
+
+The model consists of the following components:
+
+- **Afriberta Base:** The pre-trained Afriberta model serves as a powerful feature extractor for Yoruba text.
+- **CNN Layers:** Multiple 1D convolutional layers with varying kernel sizes capture local patterns and n-gram features from the Afriberta embeddings.
+- **Max Pooling:** Max pooling layers extract the most salient features from the convolutional outputs.
+- **Dropout:** Dropout regularization helps prevent overfitting.
+- **Fully Connected Layer:** A final fully connected layer maps the concatenated pooled features to sentiment classes.
+
+## Intended Uses & Limitations
+
+This model is designed for sentiment analysis of Yoruba text and can be applied to various use cases, such as:
+
+- **Social Media Monitoring:** Analyze sentiment expressed in Yoruba tweets or social media posts.
+- **Customer Feedback Analysis:** Understand customer sentiment towards products or services in Yoruba.
+- **Opinion Mining:** Extract opinions and sentiments from Yoruba text data.
+
+**Limitations:**
+
+- The model's performance may be limited by the size and quality of the training data.
+- It may not generalize well to domains significantly different from the training data.
+- As with any language model, there's a risk of bias and potential for misuse.
+
+## Training and Evaluation Data
+
+The model was trained on a dataset of Yoruba tweets annotated with sentiment labels. The dataset was split into training, validation, and test sets.
+
+## Training Procedure
+
+The model was trained using the following steps:
+
+1. **Data Preprocessing:** Text data was tokenized using the Afriberta tokenizer.
+2. **Model Initialization:** The SentimentCNNModel was initialized with the pre-trained Afriberta model and CNN layers.
+3. **Optimization:** The model was trained using the Adam optimizer and cross-entropy loss.
+4. **Early Stopping:** Training was stopped early based on validation loss to prevent overfitting.
+
+## Evaluation Results
+
+The model achieved the following performance on the test set:
+
+- **Test Loss:** [0.6707]
+- **F1-Score:** [0.8095]
+
+## How to Use
+
+1. **Install Dependencies:** Ensure you have PyTorch and Transformers installed:
+
+   ```bash
+   pip install torch transformers
+   ```
+
+2. **Load the Model:** You can load the model using the Hugging Face `transformers` library:
+
+   ```python
+   from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+   model_name = "Testys/cnn_sent_yor"
+   tokenizer = AutoTokenizer.from_pretrained(model_name)
+   model = AutoModelForSequenceClassification.from_pretrained(model_name)
+   ```
+
+3. **Make Predictions:** Use the tokenizer to prepare your input text and the model to get predictions:
+
+   ```python
+   def predict(text):
+       inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)
+       outputs = model(**inputs)
+       return torch.softmax(outputs.logits, dim=1).items()
+
+   sample_text = "Your Yoruba text here"
+   prediction = predict(sample_text)
+   print("Sentiment:", prediction)
+   ```
+
+## Citing the Model
+
+If you use this model in your research, please cite it using the following format:
+
+```bibtex
+@misc{your_model_name,
+  author = {Your Name},
+  title = {Yoruba Sentiment Analysis with CNN and Afriberta},
+  year = {2024},
+  publisher = {Hugging Face's Model Hub},
+  journal = {Hugging Face's Model Hub},
+  howpublished = {\\url{https://huggingface.co/your_model_name}}
+}
+```
+
+## License
+
+This model is open-sourced under the MIT license. The license allows commercial use, modification, distribution, and private use. 
+
+## Contact Information
+
+For any queries regarding the model, feel free to reach out via GitHub or direct email:
+- **GitHub:** [https://github.com/dev-tyta]
+- **Email:** [[email protected]]
+
+```

config.json

@@ -0,0 +1 @@
+{"model_type": "CNNForSentimentAnalysis", "num_classes": 2, "max_length": 128, "pretrained_model_name": "Davlan/naija-twitter-sentiment-afriberta-large"}

modeling_cnn_sent.py

@@ -0,0 +1,46 @@
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers import AutoModel
+
+class SentimentCNNModel(nn.Module):
+    def __init__(self, transformer_model_name, num_classes, cnn_out_channels=100, cnn_kernel_sizes=[3, 5, 7]):
+        super(SentimentCNNModel, self).__init__()
+        # Load pre-trained transformer model
+        self.transformer = AutoModel.from_pretrained(transformer_model_name)
+        
+        # CNN layers with multiple kernel sizes
+        self.convs = nn.ModuleList([
+            nn.Conv1d(in_channels=self.transformer.config.hidden_size,
+                      out_channels=cnn_out_channels,
+                      kernel_size=k)
+            for k in cnn_kernel_sizes
+        ])
+        
+        # Dropout layer
+        self.dropout = nn.Dropout(0.5)
+        
+        # Fully connected layer
+        self.fc = nn.Linear(len(cnn_kernel_sizes) * cnn_out_channels, num_classes)
+    
+    def forward(self, input_ids, attention_mask):
+        # Get hidden states from the transformer model
+        transformer_outputs = self.transformer(input_ids=input_ids, attention_mask=attention_mask)
+        hidden_states = transformer_outputs.last_hidden_state  # Shape: (batch_size, seq_len, hidden_size)
+        
+        # Transpose for CNN input: (batch_size, hidden_size, seq_len)
+        hidden_states = hidden_states.transpose(1, 2)
+        
+        # Apply convolution and pooling
+        conv_outputs = [torch.relu(conv(hidden_states)) for conv in self.convs]
+        pooled_outputs = [torch.max(output, dim=2)[0] for output in conv_outputs]
+        
+        # Concatenate pooled outputs and apply dropout
+        cat_output = torch.cat(pooled_outputs, dim=1)
+        cat_output = self.dropout(cat_output)
+        
+        # Final classification
+        logits = self.fc(cat_output)
+        
+        return logits

sent_pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f9eb93cf4592a046678a953616e4281e7bba91703e19a563cb7074ed7c69d33
+size 507210373

sentencepiece.bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6419b3044bff45e94e0553cbb81425fd06046e9294b33555e23fdc69377dba6f
+size 1554839

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,56 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "70005": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "XLMRobertaTokenizer",
+  "unk_token": "<unk>"
+}