Update README.md
I think you forget to add the output mark ">>"
Hello,thank you very much for posting this model, it has been validated and works great, but I have a question.
The available outputs are the predicted scores for the three emotions, how do you determine which one the overall emotion is?
For example: 1. select the one emotion with the highest predicted score as the overall emotion;
2. find a threshold. Positive: highest positive score and above the threshold, Negative: highest negative score and above the threshold, and the rest are neutral;
The first method is flawed because data that would otherwise be neutral may be categorized as positive or negative, e.g., 0.35, 0.34, 0.31;
the second approach requires a threshold to be defined; how is this threshold determined?For example, 0.5, 0.7
So what criteria does the overall sentiment output depend on?
Can it help me? Thank you very much.
how do you determine which one the overall emotion is?
Hi,
Good question.
To decide the final label, we look at how we train the model, especially the type of dataset we use—whether it's a multiclass problem or a multilabel problem. This choice affects the type of loss function we use, like cross-entropy loss for multiclass problems and binary cross entropy loss (BCE) for multilabel problems. Of course, there are other variants of loss functions to tackle class imbalance issues, but that's another topic. For now, let's focus on how to pick the final label(s).
A multiclass problem means only one label can be correct. For example, a text can be labeled as positive, neutral, or negative—not more than one.
OTOH, a multilabel problem is defined as one where an example may have zero or more positive labels. For example, in the movie category problem, from categories like [funny, romantic, thrilling, scary, historical, sad], a movie can be tagged as both 'funny' and 'romantic' at the same time.
For multiclass problems, we use softmax in the last layer to make sure all the predicted scores add up to 1. We then choose the label with the highest score. For instance, if the scores are 0.35, 0.34, and 0.31, we pick the highest (0.35 in this case). If you think the scores are too close to each other, this usually means the model needs more varied and high-quality data, or it needs to be trained longer to learn to distinguish better.
It often boils down to a data issue: we just need to provide more high quality and more diverse data, and train the model for a longer period. This helps the model learn to distinguish better, so that in future instances like the one you mentioned, it will learn to push the probability of the first label (e.g., the one with a score of 0.35) as close to 1 as possible, while keeping the others as low as possible. Back to the example you mentioned, we could say that the model hasn't learned well enough to classify such examples accurately.
Also, you asked if we could add a rule where a label must not only be the highest but also above a certain threshold to be chosen. Yes, we could, but that might be too strict, leading to no prediction in many cases. This could be useful in situations where making a wrong prediction could be worse than making no prediction at all.
For multilabel problems, we usually use 0.5 as a threshold. If a label's score is above 0.5, we consider it positive. Depending on the situation, we could set a higher threshold, like 0.8, to be more certain.