Update README.md
Browse filesTo play with it in pytorch or GGML/.Net [start here](https://github.com/NethermindEth/Mpt-Instruct-DotNet-S)
## Training
Finetuned for CSharp [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct). Max context length is restricted to 1024 tokens.
'Loss': 0.256045166015625 on 300k CSharp-related records
'Loss': 0.095714599609375 on 50k specific short prompts
## Sources
data contained (most data was around 500 tokens long < 1000, except large code files):
- codeparrot/github-code C# ("mit", "Apache-2.0", "Bsd-3-clause", "Bsd-2-clause", "Cc0-1.0", "Unlicense", "isc")
- raw data Plain .cs files randomly cut at the 60-80% in the instruction, and we ask the network to continue last 40-20% (76k)
- documented static functions 72k
- SO 5q_5answer + 5q_5best (CC BY-SA 4.0) 70k
- Dotnet wiki (30k, rendered out from [github repo](https://github.com/microsoft/dotnet), see also removed, GPT-4 generated short question to each file)
- All NM Static Functions and Tests (from [nethermind client repo](https://github.com/NethermindEth/nethermind) documented and described via GPT-4 (4k)
- GPT-4 questions, GPT-3.5 answers for CSharp: Short Q->Code, Explain Code X > Step-By-Step (35k)
- GPT-4 questions, GPT-3.5 answers for nethermind client interface `IEthRpcModule `: Short Q->Code, Explain Code X -> Step-By-Step (7k)
## Contents
- HF compatible model
- GGML compatible quantisations (f16, q8, q5)