Update README.md

README.md

@@ -1,5 +1,5 @@
 ---
-model_creators:
+model_creators: 
 - Leonardo Zilio, Hadeel Saadany, Prashant Sharma, Diptesh Kanojia, Constantin Orasan 
 license: mit
 tags:
@@ -51,7 +51,19 @@ It achieves the following results on the evaluation set:
 
 ## Model description
 
-More information needed
+RoBERTa is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means
+it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of
+publicly available data) with an automatic process to generate inputs and labels from those texts. 
+
+More precisely, it was pretrained with the Masked language modeling (MLM) objective. Taking a sentence, the model
+randomly masks 15% of the words in the input then run the entire masked sentence through the model and has to predict
+the masked words. This is different from traditional recurrent neural networks (RNNs) that usually see the words one
+after the other, or from autoregressive models like GPT which internally mask the future tokens. It allows the model to
+learn a bidirectional representation of the sentence.
+
+This way, the model learns an inner representation of the English language that can then be used to extract features
+useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard
+classifier using the features produced by the BERT model as inputs.
 
 ## Intended uses & limitations