umarbutler commited on
Commit
fa3ebea
1 Parent(s): 33e3c53

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -56,12 +56,12 @@ The training dataset was subsequently fed to [GPT2](https://huggingface.co/gpt2)
56
  | Weight decay | 0.01 |
57
  | Warmup ratio | 0.06 |
58
 
59
- After training for 3 epochs, or 465,441 steps, over a period of ~25 hours on two GeForce RTX 4090s, the model achieved a loss of 0.61.
60
 
61
  ## Limitations 🚧
62
  Although the model has not been tested for bias, one would expect it to exhibit much of the same, if not all, the biases of [GPT2](https://huggingface.co/gpt2).
63
 
64
- One might also expect the model to exhibit a bias towards the type of language employed in legislation and regulations (its source materials) as well as towards Commonwealth law (the largest source of legislation in [Open Australian Legal Corpus](https://huggingface.co/datasets/umarbutler/open-australian-legal-corpus) at the time of the model's creation).
65
 
66
  Finally, it is worth noting that the model may lack knowledge of Victorian, Northern Territory and Australian Capital Territory law as licensing restrictions had prevented their inclusion in the training data.
67
 
 
56
  | Weight decay | 0.01 |
57
  | Warmup ratio | 0.06 |
58
 
59
+ After training for 3 epochs, or 465,441 steps, over a period of ~25 hours on two GeForce RTX 4090s, the model achieved a training loss of 0.61.
60
 
61
  ## Limitations 🚧
62
  Although the model has not been tested for bias, one would expect it to exhibit much of the same, if not all, the biases of [GPT2](https://huggingface.co/gpt2).
63
 
64
+ One might also expect the model to exhibit a bias towards the type of language employed in legislation and regulations (its source materials) as well as towards Commonwealth law (the largest source of legislation in the [Open Australian Legal Corpus](https://huggingface.co/datasets/umarbutler/open-australian-legal-corpus) at the time of the model's creation).
65
 
66
  Finally, it is worth noting that the model may lack knowledge of Victorian, Northern Territory and Australian Capital Territory law as licensing restrictions had prevented their inclusion in the training data.
67