Ultra HD merge
Hey, me and my buddy did an ULTRA merge of this, we finished today after 1 week of downloading and merging, and will report on Quality, if improvement is there we'll quant it fully and push it
for 3.0 (posted wrong thread)
What's an ULTRA merge?
@sophosympatheia its a technique where you can gain far better merge results by merging in f32. we did that, and if its not bias, the model improve quite a bit. Do you give us permission to upload the model and link to yours?
Interesting. You have my permission to upload and share your results, and I appreciate the link back to the model page. Can you link me to any additional information about ULTRA merging?
@sophosympatheia https://huggingface.co/Autumnlight/New-Dawn-Ultra-Llama-3-70B-32K-v1.0/tree/main Here is the main file, want me to upload / host q4k m for you so you can test it? From our own little testing we found it to be a bit better, requiring lower temperature and being more focused on system prompt (on 4q k m). If you approve I will add description and link yours + post recipe.
Sure, that would be great. Thanks!