2025 textgen issues

[deandreong]

First off I’m on Ubuntu 24.04.2 cpu only version of text gen I have qdrant client 1.15.0 and docker 28.3.2 currently memoir is behaving weirdly it doesn’t really save memories except for like 2 times where it has stored single prompts during testing I’m not sure why to be honest because as of right now saving isn’t working as for injection it’s also not working but I did have the full version installed and will now be switching to the portable v 3.8? I’m largely confused by the set up process though I’m going to redownload the extension aswell from GitHub

[deandreong]

I’m having this error pop up do I proceed regardless or is it such that I’m installing the requirements to system and not text gen webui I had a similar problem my first time but I had to end up installing the requirements to a folder in web ui

[deandreong]

also i am on v3.8 the latest one click installer now of oobabooga webui

[deandreong]

Here we can see a difference in the older version which required portabl_env/python.exe -m pip install - r so what is my next course of action here I’m not super technical yet with Linux but I’d guess installing requirements to one of these file in the portable_env folder

[brucepro]

First off, is your system externally managed? Second, be sure to grab the text gen for linux. https://github.com/oobabooga/text-generation-webui/releases/download/v3.8/textgen-portable-3.8-linux-cpu.zip
I will do an install on my mini100 ubuntu server and verify it works for me. What llm are you using so I can test? When I tested it before I tested on windows, so might be a different way to install the extension on the linux one. Give em a bit and I will let you know.

[deandreong]

First off, is your system externally managed? Second, be sure to grab the text gen for linux. https://github.com/oobabooga/text-generation-webui/releases/download/v3.8/textgen-portable-3.8-linux-cpu.zip I will do an install on my mini100 ubuntu server and verify it works for me. What llm are you using so I can test? When I tested it before I tested on windows, so might be a different way to install the extension on the linux one. Give em a bit and I will let you know.

thanks for getting back to me i am using mythomist-7b.Q8_0.gguf from hugging face currently. i have verified im using cpu version for linux the next thing is im not sure what you mean by externally managed. i do not think so but i need to educate my self on that concept are you referring to my os? pc or configuration of textgen or something else to the best of my knowledge it shouldnt be because im running everything locally on my pc i did configure ubuntu to a usb though as a live bootable os with rufus and a casper-rw persistence file if that changes anything? but when i did try to install the requirements pip it did give me that externally managed warning perhaps because it was installing to my system python? idk i guess my ubuntu doesnt like that im not completly sure why but its in regards to potential conflict with my system python it does just look like a warning though. is it possible that i shouldve been running text gen in a virtual environment this whole time

[brucepro]

I don't think it would really matter on that setup. You might find the portable actually runs a bit better. Does textgen work well without Memoir+ running? Also mythomist is ancient now in LLM terms you could prolly run one of the small mistrals and enjoy it more depending on what you are using it for. I will install textgen on my mini server tomorrow and let you know. Most likely just an environment issue.

[brucepro]

another option in your environment is to not run the pip install requirements and just manually install each of the missing packages. But if you want to wait, I will write up my research tomorrow

[deandreong]

yes it does run without it fine(that was full though havent tested portable yet) im mostly only on mythomist because i was running mythomax 13b because i seen most results say its like the king of uncensored models and i wanted to see if my pc would handle less paramaters better mostly just speed with output and input of prompts so i switched to 7b perhaps there could be better uncensored models out now though but i only found out about your models like this year so i dont know? and thanks again for your help perhaps i will wait because

[brucepro]

I will grab the gguf from theblokes huggingface, but keep in mind that gguf has undergone some improvements and I am not 100% that those old quants would be compatible with modern llamacpp, so something to consider is testing a more modern model or creating new quants from the https://huggingface.co/Gryphe/MythoMist-7b repo. Or just grabbing a newer llm to test. Thedrummer is prolly the closest nsfw type model as he is known for his stuff on r/localllama https://huggingface.co/bartowski/TheDrummer_Cydonia-24B-v4-GGUF/blob/main/TheDrummer_Cydonia-24B-v4-IQ4_NL.gguf
A q4 would be pretty close to your 7b and be smarter.

[brucepro]

Also and just to be clear, I don't make models. I made Memoir+, Credit to where it is due.