Issues loading model with ooabooga textgenwebui
I'm not able to get the model to load while using ooabooga textgenwebui. The Transformers returns the error
ValueError: rope_scaling must be a dictionary with two fields, type and factor, got {'factor': 8.0, 'low_freq_factor': 1.0, 'high_freq_factor': 4.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}
If I use llama.cpp as the loader I get
IndexError: list index out of range
I did open up the conda environment by using cmd_windows.bat file in the textgenwebui folder and then ran
pip install -r requirements.txt --upgrade
to try and update everything, and explicitly ran
pip install --upgrade transformers
pip install --upgrade trl
I've also enabled "trust-remote-code" as I've seen that suggested but it didn't seem to make a differences. Are there any other settings or values I need to adjust or other dependencies I need to update?
same issue
working now after updating.
I found a 'fix' but I'm not sure what the side effects might be. In the config.json file of the model find the entry that says "rope_scaling" and replace it with this
"rope_scaling": {
"factor": 8.0,
"type": "dynamic"
},
I honestly do not know what these values mean, I just fed in values the loader said it wanted and it seems to work.
I found a 'fix' but I'm not sure what the side effects might be. In the config.json file of the model find the entry that says "rope_scaling" and replace it with this
"rope_scaling": {
"factor": 8.0,
"type": "dynamic"
},I honestly do not know what these values mean, I just fed in values the loader said it wanted and it seems to work.
claude made the same suggestion it loads then crashes for me
Hi there! If you install the latest transformers version this should be fixed