How do I actually *use* this?
#23
by
nightsd01
- opened
Sorry for the n00b question. I'm trying to evaluate this model with my own system + user prompts to see how well it does. The evaluation script provided doesn't seem to be working for me, it just gets stuck on this step:
2024-06-18:20:09:02,402 INFO [evaluator.py:357] Running loglikelihood requests
Running loglikelihood requests: 0%| | 0/81168 [00:00<?, ?it/s]
I get the feeling that this "evaluation" is just going to produce performance statistics and not actually allow me to interact with the model?