Beckett Dillon PRO
Severian
AI & ML interests
I make music, teach machines, study nature, and build things.
Articles
Organizations
Severian's activity
posted
an
update
about 1 month ago
Post
1791
I'm excited to share a really cool milestone in my AI/LLM journey.
Brief backstory: Before diving into AI, I spent over a decade working in ecological fields such as the conservation corps, biodynamic farming, and natural habitat restoration. This background instilled in me a deep concern about the environmental impact of scaling AI without sustainable practices.
Driven by this concern, I've spent months planning and experimenting to make my AI work more eco-friendly. I'm thrilled to announce that I've successfully transitioned my entire operation to run on 100% sustainable solar power!
My current setup includes multiple linked Mac Pro tower desktops and custom code built from open-source libraries. While it's a bit experimental, this configuration is working great for my needs. All my LLM research, development, and client services now run exclusively on solar energy.
I'm curious if anyone else here has experimented with renewable energy for their LLM work?
For those interested in more details, I've written a brief blog post about this journey here https://medium.com/@betalabsllm/powering-the-future-be-ta-labs-revolutionary-100-solar-powered-ai-operation-444433e61d43
Brief backstory: Before diving into AI, I spent over a decade working in ecological fields such as the conservation corps, biodynamic farming, and natural habitat restoration. This background instilled in me a deep concern about the environmental impact of scaling AI without sustainable practices.
Driven by this concern, I've spent months planning and experimenting to make my AI work more eco-friendly. I'm thrilled to announce that I've successfully transitioned my entire operation to run on 100% sustainable solar power!
My current setup includes multiple linked Mac Pro tower desktops and custom code built from open-source libraries. While it's a bit experimental, this configuration is working great for my needs. All my LLM research, development, and client services now run exclusively on solar energy.
I'm curious if anyone else here has experimented with renewable energy for their LLM work?
For those interested in more details, I've written a brief blog post about this journey here https://medium.com/@betalabsllm/powering-the-future-be-ta-labs-revolutionary-100-solar-powered-ai-operation-444433e61d43
- 1 reply
posted
an
update
2 months ago
Post
3018
GraphRAG-Ollama-UI
I've been working on a local version of Microsoft's GraphRAG that uses Ollama for everything. It's got a new interactive UI built with Gradio that makes it easier to manage data, run queries, and visualize results. It's not fully featured or set up to harness the entire GraphRAG library yet but it allows you to run all the standard commands for Indexing/Processing and chatting with your graph. Some key features:
Uses local models via Ollama for LLM and embeddings
3D graph visualization of the knowledge graph using Plotly
File management through the UI (upload, view, edit, delete)
Settings management in the interface
Real-time logging for debugging
https://github.com/severian42/GraphRAG-Ollama-UI
I've been working on a local version of Microsoft's GraphRAG that uses Ollama for everything. It's got a new interactive UI built with Gradio that makes it easier to manage data, run queries, and visualize results. It's not fully featured or set up to harness the entire GraphRAG library yet but it allows you to run all the standard commands for Indexing/Processing and chatting with your graph. Some key features:
Uses local models via Ollama for LLM and embeddings
3D graph visualization of the knowledge graph using Plotly
File management through the UI (upload, view, edit, delete)
Settings management in the interface
Real-time logging for debugging
https://github.com/severian42/GraphRAG-Ollama-UI
- 1 reply
posted
an
update
3 months ago
Post
1150
Mixture of Agents now in MLC/LMStudio/Ollama
I've been a bit obsessed with the recent MoA paper and its implementation. I've noticed a HUGE upgrade in the final output and it seems to really be a great way to harness the power of a team of different LLMs. The downside is that it can be a bit slow to generate responses with the bigger models (but worth it if you want to wait). I wanted to get faster results so I made an MLC version and it actually works out great! Much quicker and the responses definitely are better than compared to just running one.
I'm going to keep working on seeing how it can be further integrated (API endpoints, RAG, synthetic data generation, etc) and will share the stuff that I can get to work decently enough :)
https://github.com/severian42/MoA-MLC-Chat
https://github.com/severian42/MoA-Ollama-Chat
https://github.com/severian42/MoA-LMStudio-Chat
I've been a bit obsessed with the recent MoA paper and its implementation. I've noticed a HUGE upgrade in the final output and it seems to really be a great way to harness the power of a team of different LLMs. The downside is that it can be a bit slow to generate responses with the bigger models (but worth it if you want to wait). I wanted to get faster results so I made an MLC version and it actually works out great! Much quicker and the responses definitely are better than compared to just running one.
I'm going to keep working on seeing how it can be further integrated (API endpoints, RAG, synthetic data generation, etc) and will share the stuff that I can get to work decently enough :)
https://github.com/severian42/MoA-MLC-Chat
https://github.com/severian42/MoA-Ollama-Chat
https://github.com/severian42/MoA-LMStudio-Chat
posted
an
update
4 months ago
Post
1602
Jamba GGUF!
Finally, thanks to the awesome work of the brilliant mind of Github user compilade (https://github.com/compilade) Jamba is now beginning to be supported in llama.cpp (just CPU inference at the moment). So far there are a few different versions I have been able to convert, mainly the Jamba-Bagel, Jamba-Claude, 900M Jamba-Small and a 1B Jamba
Severian/jamba-gguf-665884eb2ceef24c1a0547e0
Finally, thanks to the awesome work of the brilliant mind of Github user compilade (https://github.com/compilade) Jamba is now beginning to be supported in llama.cpp (just CPU inference at the moment). So far there are a few different versions I have been able to convert, mainly the Jamba-Bagel, Jamba-Claude, 900M Jamba-Small and a 1B Jamba
Severian/jamba-gguf-665884eb2ceef24c1a0547e0
replied to
davanstrien's
post
4 months ago
Awesome work! Here is a tool I crafted and use myself for synthetic datasets, maybe it could be of some use for your project: https://github.com/severian42/Vodalus-Expert-LLM-Forge
posted
an
update
4 months ago
Post
1494
Craft Your Own Expert LLM - Using 100% Open-Source/Private/Free/Awesome Tools
Hey everyone! After seeing a lot of people's interest in crafting their own datasets and then training their own models, I took it upon myself to try and build a stack to help ease that process. I'm excited to share a major project I've been developing—the Vodalus Expert LLM Forge.
https://github.com/severian42/Vodalus-Expert-LLM-Forge
This is a 100% locally LLM-powered tool designed to facilitate high-quality dataset generation. It utilizes free open-source tools so you can keep everything private and within your control.
Why Open Source?
I decided to open source the Vodalus Expert LLM Forge to empower individuals and organizations everywhere to generate their own high-quality data. By making these tools freely available, I hope this community can start crafting their own models with little to no money and/or experience, helping to improve data quality and innovation across the board. While I'm releasing this tool for free, I've also completed an extensive tutorial/course with lots of videos and instructions that guide you through each step of maximizing the potential of this stack. This course is available for purchase at ko-fi.com/s/076479f834 and is designed to enhance your experience and results with the Vodalus Expert LLM Forge.
What’s included in the Vodalus Expert LLM Forge?
- Data Generation: Harness RAG (through AnythingLLM if you are set up properly) and Wikipedia to create datasets via local language models.
- Model Training & Fine-Tuning: Tutorials and Jupyter notebooks to customize models to your specific needs.
- Quantization: Optimize models for performance with our quantization guides.
If this project aids your work, please consider supporting it through a donation at my ko-fi.com/severian42. Your support helps sustain my further LLM developments and experiments, always with a focus on using those efforts to give back to the LLM community.
Hey everyone! After seeing a lot of people's interest in crafting their own datasets and then training their own models, I took it upon myself to try and build a stack to help ease that process. I'm excited to share a major project I've been developing—the Vodalus Expert LLM Forge.
https://github.com/severian42/Vodalus-Expert-LLM-Forge
This is a 100% locally LLM-powered tool designed to facilitate high-quality dataset generation. It utilizes free open-source tools so you can keep everything private and within your control.
Why Open Source?
I decided to open source the Vodalus Expert LLM Forge to empower individuals and organizations everywhere to generate their own high-quality data. By making these tools freely available, I hope this community can start crafting their own models with little to no money and/or experience, helping to improve data quality and innovation across the board. While I'm releasing this tool for free, I've also completed an extensive tutorial/course with lots of videos and instructions that guide you through each step of maximizing the potential of this stack. This course is available for purchase at ko-fi.com/s/076479f834 and is designed to enhance your experience and results with the Vodalus Expert LLM Forge.
What’s included in the Vodalus Expert LLM Forge?
- Data Generation: Harness RAG (through AnythingLLM if you are set up properly) and Wikipedia to create datasets via local language models.
- Model Training & Fine-Tuning: Tutorials and Jupyter notebooks to customize models to your specific needs.
- Quantization: Optimize models for performance with our quantization guides.
If this project aids your work, please consider supporting it through a donation at my ko-fi.com/severian42. Your support helps sustain my further LLM developments and experiments, always with a focus on using those efforts to give back to the LLM community.
posted
an
update
4 months ago
Post
1623
Vodalus Expert LLM Forge - Dataset Crafting and Efficient Fine-Tuning Using Only Free Open-Source Tools
Hey everyone! After my last post getting a sense of people's interest in crafting their own datasets, I'm excited to share a major project I've been developing—the Vodalus Expert LLM Forge.
https://github.com/severian42/Vodalus-Expert-LLM-Forge
This is a 100% locally LLM-powered tool designed to facilitate high-quality dataset generation. It utilizes free open-source tools so you can keep everything private and within your control. After considerable thought and debate (this project is the culmination of my few years of learning/experimenting), I've decided to open-source the entire stack. My hope is to elevate the standard of datasets and democratize access to advanced data-handling tools. There shouldn't be so much mystery to this part of the process.
Why Open Source?
My hope is to empower individuals everywhere to generate their own high-quality data. By making these tools freely available, I hope this community can start crafting their own models with little to no money and/or experience, helping to improve data quality and innovation across the board. While I'm releasing this tool for free, I'm also near completion on an extensive tutorial/course that guides you through each step of maximizing the potential of this stack. This course will be available for purchase soon and is designed to enhance your experience and results with the Vodalus Forge; more details soon
If this project aids your work, please consider supporting it through a donation on my https://ko-fi.com/N4N4XZ2TZ. Your support helps sustain my further LLM developments and experiments, always with a focus on using those efforts to give back to this community
Hey everyone! After my last post getting a sense of people's interest in crafting their own datasets, I'm excited to share a major project I've been developing—the Vodalus Expert LLM Forge.
https://github.com/severian42/Vodalus-Expert-LLM-Forge
This is a 100% locally LLM-powered tool designed to facilitate high-quality dataset generation. It utilizes free open-source tools so you can keep everything private and within your control. After considerable thought and debate (this project is the culmination of my few years of learning/experimenting), I've decided to open-source the entire stack. My hope is to elevate the standard of datasets and democratize access to advanced data-handling tools. There shouldn't be so much mystery to this part of the process.
Why Open Source?
My hope is to empower individuals everywhere to generate their own high-quality data. By making these tools freely available, I hope this community can start crafting their own models with little to no money and/or experience, helping to improve data quality and innovation across the board. While I'm releasing this tool for free, I'm also near completion on an extensive tutorial/course that guides you through each step of maximizing the potential of this stack. This course will be available for purchase soon and is designed to enhance your experience and results with the Vodalus Forge; more details soon
If this project aids your work, please consider supporting it through a donation on my https://ko-fi.com/N4N4XZ2TZ. Your support helps sustain my further LLM developments and experiments, always with a focus on using those efforts to give back to this community
posted
an
update
5 months ago
Post
3588
Create and Train Your Own Expert LLM: Generating Synthetic, Fact-Based Datasets with LMStudio/Ollama and then fine-tuning with MLX and Unsloth
Hey everyone!
I know there are tons of videos and tutorials out there already but I've noticed a lot of questions popping up in community posts about using synthetic datasets for creative projects and how to transform personal content into more factual material. In my own work doing enterprise-level SFT and crafting my open-source models, I've enhanced a Python framework originally shared by the creator of the Tess models. This improved stack utilizes local language models and also integrates the Wikipedia dataset to ensure that the content generated is as accurate and reliable as possible.
I've been thinking of putting together a comprehensive, step-by-step course/guide on creating your own Expert Language Model. From dataset preparation and training to deployment on Hugging Face and even using something like AnythingLLM for user interaction. I'll walk you through each phase, clarifying complex concepts and troubleshooting common pitfalls.
Let me know if this interests you!
Most of the datasets and models I've made have been using these scripts and my approach
Hey everyone!
I know there are tons of videos and tutorials out there already but I've noticed a lot of questions popping up in community posts about using synthetic datasets for creative projects and how to transform personal content into more factual material. In my own work doing enterprise-level SFT and crafting my open-source models, I've enhanced a Python framework originally shared by the creator of the Tess models. This improved stack utilizes local language models and also integrates the Wikipedia dataset to ensure that the content generated is as accurate and reliable as possible.
I've been thinking of putting together a comprehensive, step-by-step course/guide on creating your own Expert Language Model. From dataset preparation and training to deployment on Hugging Face and even using something like AnythingLLM for user interaction. I'll walk you through each phase, clarifying complex concepts and troubleshooting common pitfalls.
Let me know if this interests you!
Most of the datasets and models I've made have been using these scripts and my approach
- 16 replies
posted
an
update
5 months ago
Post
1663
New model and dataset! The Llama-3-IMPACTS-2x8B-64k-MLX (and upcoming GGUF) model is a cutting-edge large language model trained on the I.M.P.A.C.T.S dataset, which encompasses scenarios from biomimicry, climate change, and theoretical astrobiology.
https://huggingface.co/Severian/Llama-3-IMPACTS-2x8B-64k-MLXF
Severian/IMPACTS
https://huggingface.co/Severian/Llama-3-IMPACTS-2x8B-64k-MLXF
Severian/IMPACTS