Pruned MoEs (Mixtral-8x7B-Instruct-v0.1) - a Na0s Collection

Na0s 's Collections

Depth pruned and fine tuned Llama-3.1-8B

Medical Whisper

Pruned MoEs (Mixtral-8x7B-Instruct-v0.1)

Pruned MoEs (Mixtral-8x7B-Instruct-v0.1)

updated about 6 hours ago

Pruned experts from Mixtral-8x7B-Instruct-v0.1 with respect to the paper "A Provably Effective Method for Pruning Experts in Fine-tuned Sparse MoEs"

Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-1-experts

Text Generation • Updated Oct 6 • 12
Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-2-experts

Text Generation • Updated Oct 6 • 12
Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-3-experts

Text Generation • Updated Oct 6 • 12
Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-4-experts

Text Generation • Updated Oct 6 • 18
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-4-experts

Text Generation • Updated Oct 6 • 14
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-3-experts

Text Generation • Updated Oct 6 • 13
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-2-experts

Text Generation • Updated Oct 5 • 9
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-1-experts

Text Generation • Updated Oct 6 • 12
Na0s/Mixtral-8x7B-Instruct-v0.1-exhaustive-LoRA

Text Generation • Updated about 6 hours ago
Na0s/Mixtral-8x7B-Instruct-v0.1-LoRA-on-Gates

Text Generation • Updated Sep 26 • 13