Help
#4
by
damerajee
- opened
So I got inspired by you and I wanted to do something similar to this just for experimenting with it ,so I implemented Llamoe which is this model https://huggingface.co/damerajee/Llamoe-test , and I keep getting this errorRuntimeError: The size of tensor a (200) must match the size of tensor b (2048) at non-singleton dimension 3 can you fix this
So I look through your modeling code and it seems as if most of it is like similar to the Llama architecture ,so I copied mostly from your code but started changing stuff here and there , I really tried fixing the error , but this is the first time I'm coding a MOE model so any help please