Create README.md

README.md

@@ -0,0 +1,61 @@
+---
+license: cc-by-nc-sa-4.0
+---
+# StellarX: A Base Language Model by Fortz
+
+StellarX is a powerful autoregressive language model designed for various natural language processing tasks. It has been trained on a massive dataset containing 810 billion tokens, curated by "redpajama," and is built upon the popular GPT-NeoX architecture. With approximately 4 billion parameters, StellarX offers exceptional performance and versatility.
+
+## Model Details
+
+- **Training Data:** StellarX is trained on a large-scale dataset provided by "redpajama," maintained by the group "togethercomputer." This dataset has been instrumental in shaping StellarX's language capabilities and general-purpose understanding.
+- **Model Architecture:** StellarX is built upon the GPT-NeoX architecture, which is inspired by GPT-3 and shares similarities with GPT-J-6B. The architecture incorporates key advancements in transformer-based language models, ensuring high-quality predictions and contextual understanding.
+- **Model Size:** StellarX consists of approximately 4 billion parameters, making it a highly capable language model for a wide range of natural language processing tasks.
+- **Carbon-Friendly and Resource-Efficient:** StellarX has been optimized for carbon efficiency and can be comfortably run on local devices. When loaded in 8 bits, the model requires only about 5GB of storage, making it more accessible and convenient for various applications.
+
+Licensing and Usage
+StellarX, developed by the Fortz group, is made available under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC-BY-NC-SA-4.0). This license ensures that you can utilize the model for research purposes and personal use without any restrictions, while also promoting the sharing and adaptation of the model under certain conditions.
+
+Research and Personal Use
+StellarX can be freely used for research purposes, allowing you to explore its capabilities, conduct experiments, and develop novel applications. Whether you're a student, researcher, or hobbyist, the model's availability under the CC-BY-NC-SA-4.0 license empowers you to unlock the potential of StellarX for your own non-commercial projects.
+
+Commercial Usage
+For commercial usage of StellarX, an additional licensing arrangement must be established. If you intend to leverage the model for any commercial purpose, such as integrating it into a product or service, you are required to reach an agreement with the Fortz group. This agreement will specify the terms, including the agreed-upon percentage or licensing fee to be paid for the commercial use of StellarX.
+
+To initiate discussions regarding commercial usage, please contact the Fortz group at the provided email address or through the designated channels mentioned earlier. They will be able to provide you with further information and guide you through the process of establishing a licensing arrangement tailored to your specific requirements.
+
+Importance of Licensing Compliance
+It is crucial to respect the licensing terms to ensure the fair usage and continued development of StellarX. The revenue generated from commercial licensing supports the efforts of the Fortz group in advancing the model and making it more widely accessible.
+
+Note on CC-BY-NC-SA-4.0
+Under the CC-BY-NC-SA-4.0 license, you are allowed to modify and adapt StellarX, incorporating it into your own projects. However, any derivative work or modifications should also be shared under the same license terms, ensuring the continued openness and collaborative spirit of the project.
+
+Please review the complete text of the CC-BY-NC-SA-4.0 license to familiarize yourself with its provisions and requirements. It is essential to comply with the terms of the license to respect the intellectual property rights and contributions of the Fortz group and the wider community involved in developing StellarX.
+## GPT-NeoX and Model Selection
+
+GPT-NeoX-20B, a sibling model to StellarX, is a 20 billion parameter autoregressive language model trained on the Pile using the GPT-NeoX library. StellarX draws inspiration from the architectural advancements and performance of GPT-NeoX models. While the specifics of StellarX's architecture and parameters may differ, it benefits from the proven capabilities of GPT-NeoX and its suitability for diverse natural language processing tasks.
+
+## Training and Evaluation
+
+StellarX's training dataset comprises a comprehensive collection of English-language texts, covering various domains, thanks to the efforts of "redpajama" and the wider "togethercomputer" group. The training process involved batch sizes of approximately 3.15 million tokens, leveraging tensor parallelism and pipeline parallelism for efficient distribution across GPUs.
+
+Evaluation of StellarX's performance has demonstrated its competence across different natural language tasks. Although this description provides a brief summary, the accompanying StellarX paper presents more detailed evaluations, comparing StellarX to other models on tasks such as OpenAI's LAMBADA, SciQ, PIQA, TriviaQA, and ARC Challenge.
+
+## Limitations and Considerations
+
+StellarX, like its sibling models, is intended primarily for research purposes. It provides a powerful foundation for extracting useful features and insights from the English language. While StellarX can be further fine-tuned and adapted for deployment, users should conduct their own risk and bias assessments before using it as a basis for downstream tasks.
+
+It's important to note that StellarX is not intended for direct deployment without supervision. It is not designed for human-facing interactions, unlike models like ChatGPT, which have been fine-tuned using reinforcement learning from human feedback to better understand human instructions and dialogue.
+
+Furthermore, StellarX is limited to the English language and cannot be used for translation or text generation in other languages.
+
+Lastly, users should be aware of potential biases and limitations inherent in
+
+ the training dataset. The Pile, used to train StellarX, contains texts known to include profanity, offensive content, and documented biases related to gender, religion, and race. Curation of StellarX's outputs and appropriate disclosure to the audience about the artificially generated nature of the text are strongly recommended.
+
+## Community and Support
+
+To inquire about StellarX and receive support, you can join the Fortz group's Discord server and engage in discussions in the #release-discussion channel. It is recommended to explore the existing documentation and resources available for GPT-NeoX-20B to familiarize yourself with the model before seeking assistance on Discord. For general correspondence, you can reach out to [email protected].
+
+## Summary
+
+StellarX, a base language model developed by the Fortz group, offers impressive language capabilities and flexibility. Trained on an extensive dataset and built upon the GPT-NeoX architecture, StellarX excels in various natural language processing tasks. Its carbon-friendly and resource-efficient design makes it accessible for local device deployment. Researchers and enthusiasts can freely explore StellarX for research purposes and personal use, while commercial users should adhere to the licensing terms.