Skip to content

Chatbot llama‐cpp

Jump to bottom
Woolverine94 edited this page Feb 22, 2025 · 25 revisions

chatbot

Informations

  • Module : Llama-cpp
  • Function : Chat with an AI using llama-cpp-python
  • Input(s) : Input text
  • Output(s) : Output text
  • HF models pages :

-[ 👌 Standard ]-: bartowski/Meta-Llama-3.1-8B-Instruct-GGUF, bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF, bartowski/Marco-o1-GGUF, bartowski/nvidia_AceInstruct-7B-GGUF, cognitivecomputations/Dolphin3.0-Llama3.1-8B-GGUF, bartowski/Llama-3.1-Tulu-3-8B-DPO-GGUF, prithivMLmods/GWQ-9B-Preview2-Q5_K_M-GGUF, bartowski/cybertron-v4-qw7B-MGS-GGUF, mradermacher/Lucie-7B-Instruct-GGUF, bartowski/Falcon3-10B-Instruct-GGUF, bartowski/Llama-3.1-8B-Open-SFT-GGUF, bartowski/DRT-o1-7B-GGUF, bartowski/moxin-chat-7b-GGUF, bartowski/Teuken-7B-instruct-research-v0.4-GGUF, bartowski/EuroLLM-9B-Instruct-GGUF, Triangle104/granite-3.2-8b-instruct-preview-Q5_K_M-GGUF, bartowski/Ministral-8B-Instruct-2410-GGUF, anthracite-org/magnum-v4-9b-gguf, mradermacher/reflection-llama-3.1-8B-Solshine-Full-GGUF, bartowski/Llama-3.1-Storm-8B-GGUF, mlabonne/Hermes-3-Llama-3.1-8B-lorablated-GGUF, allenai/OLMoE-1B-7B-0125-Instruct-GGUF, bartowski/xLAM-7b-r-GGUF, Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF, bartowski/Tiger-Gemma-9B-v3-GGUF, bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUF, bartowski/Qwen2.5-7B-Instruct-1M-GGUF, bartowski/openchat-3.6-8b-20240522-GGUF, bartowski/OpenChat-3.5-0106_32K-PoSE-GGUF, NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF, bartowski/Mistral-7B-Instruct-v0.3-GGUF, arcee-ai/Arcee-Spark-GGUF, bartowski/internlm2_5-7b-chat-1m-GGUF, bartowski/L3-8B-Everything-COT-GGUF, NousResearch/Hermes-2-Pro-Mistral-7B-GGUF,

-[ 👍 🚀 Tiny ]-: bartowski/Phi-3.1-mini-4k-instruct-GGUF, bartowski/Phi-3.5-mini-instruct_Uncensored-GGUF, bartowski/AGI-0_Art-Skynet-3B-GGUF, bartowski/Hermes-3-Llama-3.2-3B-GGUF, bartowski/Llama-3.2-3B-Instruct-uncensored-GGUF, bartowski/Nemotron-Mini-4B-Instruct-GGUF, bartowski/gemma-2-2b-it-abliterated-GGUF, bartowski/2B_or_not_2B-GGUF, dranger003/MambaHermes-3B-GGUF, bartowski/gemma-1.1-2b-it-GGUF, TheBloke/phi-2-GGUF, bartowski/SmolLM2-1.7B-Instruct-GGUF, TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF,

-[ 👏 🐢 High-end ]-: bartowski/Mistral-Nemo-Instruct-2407-GGUF, bartowski/cognitivecomputations_Dolphin3.0-R1-Mistral-24B-GGUF, bartowski/krutrim-ai-labs_Krutrim-2-instruct-GGUF, bartowski/Mistral-Small-24B-Instruct-2501-GGUF, bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF, bartowski/Qwentile2.5-32B-Instruct-GGUF, bartowski/QwQ-32B-Preview-GGUF, bartowski/SuperNova-Medius-GGUF, bartowski/Mistral-Small-Instruct-2409-GGUF, microsoft/phi-4-gguf, bartowski/Qwen2.5-14B-Instruct-GGUF, cognitivecomputations/dolphin-2.9.3-mistral-nemo-12b-gguf, TheBloke/mixtralnt-4x7b-test-GGUF, TheBloke/Mixtral_7Bx2_MoE-GGUF, DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF,

-[ 🧑‍💻 Code ]-: bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF, bartowski/codegeex4-all-9b-GGUF, bartowski/Qwen2.5.1-Coder-7B-Instruct-GGUF, bartowski/OpenCoder-8B-Instruct-GGUF, bartowski/llama-3-sqlcoder-8b-GGUF, TheBloke/CodeLlama-13B-Instruct-GGUF,

-[ 🧑‍⚕️ Specialized ]-: AdityaXPV/Llama-3-8b-Instruct-law-sage-v0.1-GGUF, bartowski/Llama-3.1-WhiteRabbitNeo-2-8B-GGUF, AlicanKiraz0/SenecaLLM-Q4_K_M-GGUF, QuantFactory/Medical-Llama3-v2-GGUF, mradermacher/Bio-Medical-Llama-3.1-8B-GGUF, prithivMLmods/Llama-Doctor-3.2-3B-Instruct-GGUF, bartowski/HuatuoGPT-o1-8B-GGUF, bartowski/internlm_OREAL-DeepSeek-R1-Distill-Qwen-7B-GGUF, bartowski/Qwen2-Math-7B-Instruct-GGUF, bartowski/Tulu-MathLingo-8B-GGUF, bartowski/Peach-9B-8k-Roleplay-GGUF, Lewdiculous/Kunoichi-DPO-v2-7B-GGUF-Imatrix, bartowski/oxy-1-small-GGUF, bartowski/deepthought-8b-llama-v0.01-alpha-GGUF,

-[ 📜 Legacy ]-: TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF, TheBloke/CollectiveCognition-v1.1-Mistral-7B-GGUF, LoneStriker/Starling-LM-7B-beta-GGUF, bartowski/gemma-1.1-7b-it-GGUF, TheBloke/Yarn-Mistral-7B-128k-GGUF, TheBloke/zephyr-7B-beta-GGUF, mlabonne/AlphaMonarch-7B-GGUF, mlabonne/NeuralBeagle14-7B-GGUF, TheBloke/MetaMath-Cybertron-Starling-GGUF, TheBloke/una-cybertron-7B-v2-GGUF, TheBloke/neural-chat-7B-v3-3-GGUF, TheBloke/Starling-LM-7B-alpha-GGUF,

Help

  • Usage :
    • Type your request in the Input textbox field
    • (optional) modify settings to use another model, change context size or modify maximum number of tokens generated.
    • Click the Generate button to generate a response to your input, using the chatbot history to keep a context.
    • Click the Continue button to complete the last reply.

  • Models :
    • You could place llama-cpp compatible .gguf models in the directory biniou/models/llamacpp. Restart Biniou to see them in the models list.
    • You can also copy/paste in the Model dropdown menu a HF repo ID (e.g : TheBloke/some_model-GGUF) from this list. You must also set manually prompt and system templates according to the model page.

Examples

  • Max tokens : 512
  • Context size : 1024
  • Prompt : Give me a list of 10 originals birthday gifts ideas for my wife.
  • Output(s) :

  1. A customized photo album with pictures of the two of you throughout your relationship. This can be created online or by hiring a professional photographer to do it.

  2. An evening at a spa where she can relax and unwind. Many spas offer couples packages for birthdays, so consider booking one for her.

  3. A handwritten letter expressing your love and appreciation for everything she does. It doesn't have to be long or complex; just make sure it comes from the heart.

  4. A gift certificate to her favorite restaurant or a new food experience, such as cooking classes or a food tasting event.

  5. A unique piece of jewelry designed specifically for her. You can find many online stores that offer customized jewelry at affordable prices.

  6. A framed print of your favorite photograph together. This is another way to showcase your love and memories together in a tangible form.

  7. A weekend getaway to somewhere special. Whether it's a cozy cabin in the woods or a beachfront hotel, surprise her with a trip she'll never forget.

  8. If money is tight, consider giving her a gift card from her favorite store or a gift certificate for something you know she wants. Personalized gifts don't have to break the bank!

  9. Finally, consider spending the day doing an activity that she loves but you haven't had time to do together lately. It could be painting, hiking, golfing – whatever makes her happy.

  10. Lastly, don't forget about thoughtful gifts like a new book, music album, or tickets to an upcoming event that she's been wanting to see. Remember, it's the thought that counts!

Clone this wiki locally