How to add new files without cloning the entire repo?
#1
by
ggerganov
- opened
@phymbert Now that we have Grok-1 in the repo, I'm not sure how to commit new files without having to download extra ~150GB of the grok shards. Is there some workaround?
If not, we might want to move the model to a separate repo, or demonstrate the sharding with something smaller.
- if using git: you can clone the repo using
GIT_LFS_SKIP_SMUDGE=1env var (only downloads pointers to large files, not the actual files) and you'll still be able to git commit as usual - alternatives to git:
- use our HTTP API:
huggingface-cli upload ... - or just drag'n'drop files from your browser to upload them (the hashing implem should be efficient, but ~150GB still might be a lot)
- use our HTTP API:
That being said, we do recommend (as a convention) splitting different models into different repos
That being said, we do recommend (as a convention) splitting different models into different repos
Noted, will split the models in different repos accordingly