kenshin commited on
Commit
d5578dc
β€’
1 Parent(s): 6e7490e

update model card again

Browse files
Files changed (1) hide show
  1. README.md +3 -5
README.md CHANGED
@@ -21,14 +21,12 @@ license: mit
21
 
22
  ### Adaptive Training Process
23
 
24
- <div style="width:100%; height:100%; overflow:hidden;">
25
- <iframe src="https://viewer.diagrams.net/?highlight=FFFFFF&nav=1&title=adaptive.drawio#R7V1bcxo5Fv4t%2B%2BCq3QdRul8e7TiZmdqZydRmK3N5mdIV9wTTXsCxM79%2BJaCBvgANNBgn2FU2qBu1kL5z1TlHV%2BTN%2FfN3I%2F1w91Pu%2FOAKQ%2Fd8RW6vMFaCxb%2Bp4cusQVAya%2BiPMjdrgsuGD9nfftaIitbHzPnxvG3WNMnzwSR7KDfafDj0dlJq06NR%2FlS%2BLeQDV2p40H1fa%2Fhg9aDe%2BmvmJnezVsngsv17n%2FXviicjOL9yr4ub5w3jO%2B3yp5Um8vaKvBnl%2BWT26v75jR%2BkuSvPy7s1VxcDG%2FnhpM0Hhs9Pk4%2B3b3%2F9pD%2F2b%2FD1u7754xZgPuvmsx48zr%2FxfLSTL8UUjPLHofOpF3RFbp7uson%2F8KBtuvoU1zy23U3uB%2FPLIR9O3uSDfBTfOx%2F042CSWrPBoGi9wsS49Bvb%2BwM9TgsE4%2BvF%2FKQ3D36U3fuJH6UnZcP%2BvPfxZJR%2FWiwEnj%2Fvnb7PBglfb%2FQgM6NscWN9IPGG%2FjC22DhpPl66%2BexHkywu9%2FX8wn3mXPri047ncETpQXkayCQ9RaUB1md%2FviCpQ%2F%2B80jRfje98Hr%2FQ6Eu8pbgKBeoROvvYnDwwEbP3T0uwUTq%2F524FaLzAmZ4DvL%2Fof4mB%2BGIOg10gAb9OSHS%2FgFjx8uJh2XLx0LEWD9GuF6%2B8TO9w%2Bt1AYBvpcXWV9qfEZjyVsNLB4i5YfLG4EvYIq62voj1WX2DCxbEWmLyWBe5gDTivLkF9%2Fo81%2B%2FrPn%2F%2F47ePg%2FdNn%2BeO%2Ff4fh%2FoYPAUKo6%2Blvwxxv6Vv0ju3NHA8Vgw3LXaXJ%2BvKPZwP5b%2F7QHU02SkzI2zFdBuXhqHj%2Fpx69D8%2FZb89kfEO%2F%2B1k9P74HahdMwPPARAuBOdDGD27ykfOj4rnDfOg3wKo%2F0i7zy8HPbz%2FWyouWuhIl6lj8gL1CdnDQ0h9DbWpPwcdaR9ywjHwh7qbmZTH7%2FH%2BPyWKLE0Hg9Ge1CTo90WAcTdYRGPtBNEjB%2FLnFPYRSEQTngFqm4x%2BlgGaaA60NJ0oIDxNYV3pcIqloTEOKz0jM9zreEFfjeXrr8kO8n%2F7%2F984X3yJOyuyLzK50qjEEoT1jFeym9ulPjVvMcbTUxiVcj8Mu9LgK2FRdh2MNOkTR1jnUmvS3Vw%2B1%2Fz1m9tNRwNYIns7Vl3XQXQf1Cnq7h2lUYXsvDNQmS7IE1HMCYMifvxL4rZX6pwMfg7zJ1D0W%2Bhp16p3srItO3YkeJlrqYcfTp5vU6RlVR3Ny2IbnWCSRg4YDaYIElDIMdHrFpITWU20YNx3xnF3ZYXBYhyAFMNBBQDW1QEnJAIVSa2KEsMF2NLS%2FHu8fxmsZ4qJ5NqlfH59kUz55BMYoTsoYm03OJvf8S8tlJNcA0Yzyp%2BFFMncFQErOAIBNonk3AHoWTGR6AjhJDKBKEmB8UIBBiRSBBvPgduSEdA0A3z5HkZ0GhY5jC6%2FghS%2Fku7af%2BtNuKvCs4MbaOBF1CzleIZEOiWtULBbqCFqP4lX1pQMQEirLxokkNQiiYtuwvEl5LAxuddhsxaDQEsGkHjhOCaDSYWCMpEBgQTXxEiq8q6KwFYP4DDC4C9Y6Yadb2eeSDFahfQTcUgkbjOoTI3er%2F2c795SQECocgFPxHTCLKq7yIEluz3DUJNmuZvVW5NJzQO6qIXbKbcgqjgTlPVbf6j8tkIjcbiH7obtOMVDLOVyZ%2FLL57J%2BzyW%2FzK%2Bn176m9x%2Bbvbp9Xbrv9UrwZxi%2Fy2%2BqblU%2Blt8uPTd8tPuf6%2FsN8lPFK1Jj%2B4wd6kuXDt8srN%2FZx9HkBpPK6ijoT4VArJJr4G74VKVhmVTsTNbt%2BHf8a5E%2FXw%2Bx%2BOrhtjGmcP46s37Rks%2FsmetT3kxbSLc3TRnyuqoAN2CvaRtPZ%2FexLo20C5PwJv%2BTZlBMVfgJeEf98wUaLTmZfff65JbBrXRFU6QrVuprNTq2rKY0svvoBZNNis%2FZrJZuyicLgXlRUJpsG%2Bth1K2cr2RQW71a6keys6AYVu8MLtyral24QL3dFWa2rI9MNbREVeKZ0UyYCtR8l7U4ocTbmug1%2FEbKhbaUNOi%2BqobhMNVztTTWq0hWqdXVsqmmxjXFkqtmDw9d0rTjEd1n65qeWXxWV7SRaGn6dWhopQn%2FnYOeI7Es3WJBSV5icnG62bj6Pi0yRwmRFW0ze3WInIhdyUkkPGNEQUOE10MxTEDTTKhDlHVd1c7myWbXd7sY1u7uF9x55BLXDAEKLATUmDs0SBHAcGuYqsJD09erQfs6vpjkp9i7tWkbUTi37f6zdDGpn7UcDelJmWO1dRk2%2BgTK7W3UPsPXkv5bUdwgvTOEWskI%2FcwyWLHxSp1%2BO15PqYQEYLXJzVoSHTU7fzG6UH0fl3I07MhvMEXoUrUe1Zd%2FkrNh3VOl7nKvlT3njR1Dao3hPfs5IZRuT1kTDkfk5O9xfHxxllikLlAscUC0lMEJTQCgWXmIiUIr6f2X%2Bepyo6%2BKvL3Ba5G0UVi5sCoI7rZuVHe6vh1RhrVUU2NSzaBIIDZRhHCDvFdGKERxn87Xtdl6Quwm5BL44bg8P39SBOWgYBVAKB2gQMvJerKPK6UMgkhOGO8ft0feZLrjdhFuO6iGAJ8Ztk9K7G253td0OwO0g1y5eH%2FqnhUl1fqbTPnr2DjmauOwsEAzVIQQbIESOBSF%2BqPNgu0QPJDiOFfBY%2BSjRqY0IUwIYnWQ5x0JuRdh654FphUiyh%2BfAKWkkjopI4E4CCjmOigg3cezSEoYctIHVx3XjQz7ysf9r57Kph22JcHN%2BDoPjoJyQSjZ4Q6o%2FEviUKF8fK31B%2Be4ovw4Jfa8Y5FUdpwvtgMEekdtxz0gPN0SgHg%2F64mAdQcYv54ViQFEebTIS%2F5gUES09Fc5GNUgksfbKvAmX6L%2BSTxeRsmZyDu4E3hS1tRt0bdqfoZYBx4kFlNIAJMcQOIikE8xhJd3FnfB1Qffl%2FQm8KW5qR%2BAKIRHEDiAvomCO0wq0VTQKZhKEQUFK0bkHl1x4bgNwtbchqFMA9%2BUdCoVSftGTO9WT%2F%2BPv8zilr05N7gLkpNiI3agUy8bCYEeD%2BdZ9ijYwn%2BFqZEv33U0mqdjmdRrgbEnHvX6e9wdeP2Tjns3vY7NNCUfvQnXCL4S01a3yLVMSgrQn2HZiEvy0xNSirNOOkRfzyL1ltN7vV8s4vlaRe54huhKft1o7ExKHeU2fSH7fZeTeVG9pCJQrxXYMTP60PgS2GQf7FXJcX5NhjTK1vGHvOOBWys7WCBPeNsKkKE92JhEmCKmqbkZ6RTXmnUNroxFa6kwo1GOiXVRJpBv9ZeW2h3TDeIeRi3mUz5LGZ112GrMiDvcyYRyswEGCIEI01aXkQFqOgCJGIGcsktMNz9dl8bQx1V8qUw8RxnqsghXWIy%2Bdryy2Ws%2F7rPyxwlgfh%2FN9zXPd0TxyMGiDViJoY%2BowbNBK2AbOfBCIZFMKzo4MSToRVWwJpEAGUIMRMM5I4JVFUXnFXMDO3d7fUuGF47Cwl6%2B3IA%2Bv%2BcGN5l5CBqDBAlCmabTroix0inCEnYlWUudZ60ffcUkL%2Beq8f8dzW9ehew4lF%2BTWnfLtctQTIlnkm9pDBGjwFkjqLMBxKqVQ1Aaqv0pF7pWA98gFb9bU%2FToxjA%2BPjDthQc78c3ITX%2Bp%2B7YlAKXCFk2LFGkt%2FbXAYdA%2FBw03iE0JwcqT6198qAgk%2FAwQeHj9xQgS6vH9BYIcIZNEwx3Dl56XRuN6tc8J6tev3vFpnKCnuBIwWWcAq5dYZCWQQONpmFnrKAxYBdkQSvbUE8dVXpT0hnYhIJy9LGuocy9SuReVA%2F%2F3lwqk7RCAl58WpVQdFa4MU0fhMznOpADVSA82tBUo5C4WywqVKABcfwCt2YFXT4Riqc9HTGv2qxXmOuxQQ%2BrpLz%2B1VH6nrChSty9WdW%2BEtpsosfOnM3bnMoyhnS3GKT1xwQu0WwXO2dNN9pdMOaeRlqqOeW5XHIt104SURXZENw6eujqp2Kzl0TlUeOxFRh5Aae5miwoUPYivdFKlHZ0I3ipbFDWb7hqKhCtkQURnMsYnmUon7tVXiLtzI24nmvGI4VaUO8KJI3e5EQysZZuTEkub11uH%2BRusJq7Z2zZkJGgRRWdJwXLHW2wc907KkocWJeScimkUdlVdINd%2BsqOFfh6hhtCtRQ4tKqyejmsOdv0YiKS1kgHrHASXBAGni2yCoRD7%2Bku4j%2Bc%2FizJ1vx%2Flby11%2Bee8vgvjC718bvxct%2Bf2CLZ0pw69ZBK%2BI4Xe8a%2FKayOa1urHaWuTnTjav1yRHsIPam3HYRuMAlA0IUGYQ0JoqoGggyhlpjd0WznTRk85cT6pneTBBevKl4%2BMRXB%2BdvPlAjqf58BLchvnoXg9m2KwicZ%2FTOqBTDhqFQLCRKqj0AkhtBGCeIMGcZbp81HFjAN9xDhJxHjmjvQNYKxgNG5XKbWANgrfQUc5YcH790LLGce02AqoMNpYbgA21gBoXRxDfA4ORNigQN0vhqozgg49gLRXMyBqiCFsFF86aXfZ5z0ne5diXNSi7gKkVmGb3%2FxAZ5lTfGHe3%2BrF5CoCds5oHPkw2ce%2BdcprxJk1rhbMXCa3pk%2FNBog6Yuqjsxi08myvsvIihW%2BXm9HinzO%2Fl6nR6fLeQ8C3COZb6%2BO8ldXyNcr5PFZeDyx9sjzhibXXn8wqdUIWdtgga5fuGTihW7orIWlfdac%2FD56fJx9u3v37SH%2Fs3%2BPpd3%2FxxC2QdnSumXD6a3OX9fKgHq2ZcGZXLe37M84c5sP7yk8mXOY%2FQj5O8DOsOAXkA%2FIpAthYRb21tt9bIastkGtcMN2mMlVWM6vNDehlN7sGXm1E0ANJX3GY9lE2NMMgevp%2B%2FntoRv%2BTjbG42j2ZktRAjP1auL8RJW7FT5yxHLFbAcPloKVxUzVnhILSBg9Bq4aHu1rRFNMbrW9MZA5iR%2BYkXuJwKLQqbbmWBiagv8OLw4%2B4XuIOkK06IDZoC7YgGNCgGjOMeeChFCErYqStkmx59YNlQoYRUhKBo9%2BiozAtigJESA6mY44wSHZX8vTO%2FmhXqLhR9YhwiChJAMUxpEIKkhBwIiJYYGo2EcQ02yLWN3FxPIvVgOHPcdKLBt9bVz7wOIlOVwAK5OKFvhc54AyPtogZiI5mRJv37QmanIjPoscFcxHHiZOrHGYtzJxHwGMXv4DlU3DbYx0N9IbRdCE1wfEJC63%2F66%2Fu3H%2F%2FOrvUv3%2F1EfnB59j0FnRTu3Zi4phlWKjAALXORbVsGFNEeSK65iCCTUNi9KeCoRx1zbqg0qZ4u43HkzKK0kRD%2FSEEpZc5p3CBrqkcdT6K2NByf%2F1HHx4G8Ks6gX8nF4U2%2B%2BoZSdgL1qntQncF%2Bt6DpFiV212%2FBdJpve0Dh2mpB3%2BlPTaEnV%2FW91Y15umfizam6EBkhPUoj5TKMBFGo0mFbz46s7IqmbiVfdlvutTsnD3%2FP7NvHXOU3P4Rfx4MRwpGfNQWSffM%2Byfd%2F6tH78Jz99kzGN%2FS7n9Xz43vQ%2BhjuM8uCrNm7RenWnQ%2FdrnSE1GmDYFj3NczXIBRtQ%2BimCuPThnihYMlXW4%2BMX1%2F%2BvFLs%2FEzKlTcG568J0Nkn0qi0v7%2BM41EbZcZ2p27bdMzC8X4m9CtZ2SEp961wznilBqBqm8O8a3lzxsoampj7ULuqbt7In1GTF61b%2FWtBl%2Beqf5UPVCgdm7BXxN%2BmoxZglbHtGyC4P0k37BI2QgOfVyICRuWtvdVinLsSNSZVoq5Y9d0J5eap3UuB7DYwdZM0rlNCRaC0jixdnklSKpvdPlFtX%2FooEdnJhf3%2B1Elba8zkrMiTl%2BWtQBWKah3%2BSiuCm7TTl9dL286JFx9Po94N4%2B2CxBtrj21Rrnf1Q21FawHrM0ErpqzHuVr%2BVNBLaY%2Fifct38LJvWVZdHscWLk2e5K6Fi%2BhUuryEOOlBTCrkluCymeDSu1%2F8KItrlOTIbTlnQjVod0WydpovPbR3%2BWi68LfZyNtOMibaBn2dGQWiqj8aVfj8GZLbxnk95EyffTZF9jvT5yGzn2LvQ%2F%2BUoPBtnuuTDIfKqWVU9JoOV4EsioEGEqiVdtpnO6SZee%2BWsvat2eOnsa33sadaMfDmJW9dhObMODjEZc7LiiqVO%2Bv8sOJk4wV9nkpl2poytJUda%2BKkwdqlyAwXhYJ3wFhBgUdWBGqEtmHb0TB2RgNJIoz65p9xcuO3gcW%2Ff10tk5GK83XTrau8cW2q0vzC7JGN%2BU2f9SjT8X8kfT15HPnxlvusflh3y2qGy4yS44uBn0zS5CVladivfzIfPdzp4bzL2Z4TTBIHZJGXDee9FXWbZ1cmo%2FiBEPsoepsqsjP850%2Flrp7ykSs%2FfNFXHK%2F5lMXuUp8zngPmFFS6byrDpn00zKFZ8DdQWcbpQXBpBVdf%2FGvlazhv81k2C5jcRfE89OP52LNhNsmKJ1TvXVnMjfetDKd0X%2BSeelKdOZeNHwb6S3H7IBumo4n%2Fkd0%2F5KOJHq6JAXmLr9TtlXyzY9jQljjbmnjqQPYvwncWTsSmCulE9IpztEuhP7zXRZ7LRiFwAAdSljsvlQKEKQyolSFyIAKB05xzbSwM6TDHCwe6cKALB3o5DqRk2fTgEC24yioLauI%2FuEePxH5Ik0%2BznvI%2Fo%2BGRLTGNu8lkTojv5ibkuNfP8%2F7A64ds3LP5fWy2CVHvQtWGWGVHQdsyN2q6abseJpF0XCogoNaAYm2BJkwA4a1AxkDMw7ajTfeIctxtjMZgQxTEIAglAY3MGRhvCYBIxFngWBi7IUm2q0EIiqm0BmghUz0rnV4JAbg03kPCuEq7a0ceBJEOGkcC4I5ErVmkQGePOSBKYeZNFFq2Iby540FgxjjR1AOBFIqCM06C4owAo7RBxFPEZdg19ntHX44QzgjOALZIAUpDFN3UMICcdZJhalE6iXDdCEwHI2DEBE%2BDBjCEOAJCOTCIGUAs5xQ7qgMVHUT%2Frivi8WHiH7YE5rZVgwSOWo9wDCgcTGQAXgEtVQBMQxdpnzJiS7g%2B4LiYxdAeTZuRccEJliwAYRSNZE8VkMFJwAiKjCDF8tMGsr9fnZf0oLbZ7E23NgX%2Br%2Fl4J80v6FDsQFZTWYndKypKnyBHoFlMb00SuIjpi5i%2BiOmLmN60EC3ybTSXNC63DQ4CKnnEHOYEOGatM4xbQ7o65rpDwd%2F9qBHvcc5Fu3I%2F29GjrDBUUoCMgImdOCCFxCBCShiJIrQo3fCUdjpG1H2QFul0PIgiz%2FIWA60iWLmC3irng%2FKu%2FpDhvjrGRfU4kuqBInJ7hC4PzkOn0kTi21Ge0LHcYInf8e6n3Pl0x%2F8B" width="100%" height="350" style="border:none; transform:scale(1);" title="Adaptive Training Process"></iframe>
26
- </div>
27
 
28
  ## πŸ’‘ Model Details
29
 
30
- - πŸ€” DynMoE-StableLM is a MoE model with **dynamic top-$k$ gating**, finetuned on [LanguageBind/MoE-LLaVA-StableLM-Stage2](https://huggingface.co/LanguageBind/MoE-LLaVA-StableLM-Stage2).
31
- - πŸš€ Our DynMoE-StableLM-1.6B has 3.2B parameters totally, but **only 1.8B are activated!** (averge top-$k$ = 1.25)
32
  - βŒ› With the DynMoE tuning stage, we can complete training on 8 A100 GPUs **within 40 hours.**
33
 
34
  ## πŸ‘ Acknowledgement
 
21
 
22
  ### Adaptive Training Process
23
 
24
+ ![](https://cdn.jsdelivr.net/gh/QAQdev/Pics@master/uPic/adaptive.png)
 
 
25
 
26
  ## πŸ’‘ Model Details
27
 
28
+ - πŸ€” DynMoE-StableLM is a MoE model with **dynamic top-k gating**, finetuned on [LanguageBind/MoE-LLaVA-StableLM-Stage2](https://huggingface.co/LanguageBind/MoE-LLaVA-StableLM-Stage2).
29
+ - πŸš€ Our DynMoE-StableLM-1.6B has totally 3.2B parameters, but **only 1.8B are activated!** (averge top-k = 1.25)
30
  - βŒ› With the DynMoE tuning stage, we can complete training on 8 A100 GPUs **within 40 hours.**
31
 
32
  ## πŸ‘ Acknowledgement