Model Name: DistilBERT for Sentiment Analysis

Model Description

Overview

This model is a fine-tuned version of distilbert-base-uncased on a social media dataset for the purpose of sentiment analysis. It can classify text into non-negative and negative sentiments.

Intended Use

This model is intended for sentiment analysis tasks, particularly for analyzing social media texts.

Model Architecture

This model is based on the DistilBertForSequenceClassification architecture, a distilled version of BERT that maintains comparable performance on downstream tasks while being more computationally efficient.

Training

Training Data

The model was trained on a dataset consisting of social media posts, surveys and interviews, labeled for sentiment (non-negative and negative). The dataset includes texts from a variety of sources and demographics.

Training Procedure

The model was trained using the following parameters:

Optimizer: AdamW
Learning Rate: 5e-5
Batch Size: 32
Epochs: 30

Training was conducted on Kaggle, utilizing two GPUs for accelerated training.