Skip to content

Commit

Permalink
🚀 RELEASE: Version bump to 1.2.0. Added Home Assistant Conversation I…
Browse files Browse the repository at this point in the history
…ntegration. #3
bearlike authored May 16, 2024
2 parents e46c006 + cdb10dc commit b401fe1
Showing 21 changed files with 878 additions and 68 deletions.
26 changes: 19 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -4,8 +4,9 @@
<p align="center">
<a href="https://github.com/bearlike/Personal-Assistant/wiki"><img alt="Wiki" src="https://img.shields.io/badge/GitHub-Wiki-blue?style=for-the-badge&logo=github"></a>
<a href="https://github.com/features/actions"><img alt="GitHub Actions Workflow Status" src="https://img.shields.io/github/actions/workflow/status/bearlike/Personal-Assistant/docker-buildx.yml?style=for-the-badge&"></a>
<a href="https://github.com/bearlike/Personal-Assistant/pkgs/container/meeseeks-chat"><img src="https://img.shields.io/badge/ghcr.io-bearlike/meeseeks&#x2212;chat:latest-blue?style=for-the-badge&logo=docker&logoColor=white" alt="Docker Image"></a>
<a href="https://github.com/bearlike/Personal-Assistant/releases"><img src="https://img.shields.io/github/v/release/bearlike/Personal-Assistant?style=for-the-badge&" alt="GitHub Release"></a>
<a href="https://github.com/bearlike/Personal-Assistant/pkgs/container/meeseeks-chat"><img src="https://img.shields.io/badge/ghcr.io-bearlike/meeseeks--chat:latest-blue?style=for-the-badge&logo=docker&logoColor=white" alt="Docker Image"></a>
<a href="https://github.com/bearlike/Personal-Assistant/pkgs/container/meeseeks-api"><img src="https://img.shields.io/badge/ghcr.io-bearlike/meeseeks--api:latest-blue?style=for-the-badge&logo=docker&logoColor=white" alt="Docker Image"></a>
</p>


@@ -26,30 +27,41 @@ Meeseeks is an innovative AI assistant built on a multi-agent large language mod

| Completed | In-Progress | Planned | Scoping |
| :-------: | :---------: | :-----: | :-----: |
| | 🚧 | 📅 | 🧐 |
|| 🚧 | 📅 | 🧐 |

</details>

# Features 🔥
> [!NOTE]
> Visit [**Features - Wiki**](https://github.com/bearlike/Personal-Assistant/wiki/Features) for detailed information on tools and integration capabilities.
<table align="center">
<tr>
<th>Answer questions and interpret sensor information</th>
<th>Control devices and entities</th>
</tr>
<tr>
<td align="center"><img src="docs/screenshot_ha_assist_1.png" alt="Screenshot" height="512px"></td>
<td align="center"><img src="docs/screenshot_ha_assist_2.png" alt="Screenshot" height="512px"></td>
</tr>
</table>

- (✅) [LangFuse](https://github.com/langfuse/langfuse) integrations to accurate log and monitor chains.
- (✅) Use natural language to interact with integrations and tools.
- (🚧) Simple REST API interface for 3rd party tools to interface with Meeseeks.
- () Simple REST API interface for 3rd party tools to interface with Meeseeks.
- (✅) Handles complex user queries by breaking them into actionable steps, executing these steps, and then summarizing on the results.
- (🚧) Custom [Home Assistant Conversation Integration](https://www.home-assistant.io/integrations/conversation/) to allow voice assistance via [**HA Assist**](https://www.home-assistant.io/voice_control/).
- () Custom [Home Assistant Conversation Integration](https://www.home-assistant.io/integrations/conversation/) to allow voice assistance via [**HA Assist**](https://www.home-assistant.io/voice_control/).
- (✅) A chat Interface using `streamlit` that shows the action plan, user types, and response from the LLM.

## Extras 👽
Optional feature that users can choose to install to further optimize their experience.
- (🧐) **`Quality`** Use [CRITIC reflection framework](https://arxiv.org/pdf/2305.11738) to reflect on a response to a task/query using external tools via [`[^]`](https://llamahub.ai/l/agent/llama-index-agent-introspective).
- (📅) **`Privacy`** Integrate with [microsoft/presidio](https://github.com/microsoft/presidio) for customizable PII de-identification.
- (📅) **`Quality`** Use [CRITIC reflection framework](https://arxiv.org/pdf/2305.11738) to reflect on a response to a task/query using external tools via [`[^]`](https://llamahub.ai/l/agent/llama-index-agent-introspective).
- (🚧) **`Privacy`** Integrate with [microsoft/presidio](https://github.com/microsoft/presidio) for customizable PII de-identification.

## Integrations 📦
- (✅) [Home Assistant](https://github.com/home-assistant/core)
- (🚧) Google Calendar
- (📅) Google Search, Search recent ArXiv papers and summaries, Yahoo Finance, Yelp
- (🚧) Google Search, Search recent ArXiv papers and summaries, Yahoo Finance, Yelp
- (🧐) Android Debugging Shell

## Installating and Running Meeseeks
41 changes: 27 additions & 14 deletions core/classes.py
Original file line number Diff line number Diff line change
@@ -2,8 +2,8 @@
import abc
import os
import json
from typing import Optional
from typing import List, Any
from typing import Optional, List, Any

# Third-party modules
from langchain_community.document_loaders import JSONLoader
from langchain_openai import ChatOpenAI
@@ -19,6 +19,7 @@


class ActionStep(BaseModel):
"""Defines an action step within a task queue with validation."""
action_consumer: str = Field(
description=f"Specify one of {AVAILABLE_TOOLS} to indicate the action consumer."
)
@@ -36,11 +37,17 @@ class ActionStep(BaseModel):


class TaskQueue(BaseModel):
"""Manages a queue of actions to be performed, tracking their results."""
human_message: Optional[str] = Field(
alias="_human_message",
description='Human message associated with the task queue.'
)
action_steps: Optional[List[ActionStep]] = None
action_steps: List[ActionStep] = Field(default_factory=list)
task_result: Optional[str] = Field(
alias="_task_result",
default="Not executed yet.",
description='Store the result for the entire task queue'
)

@validator("action_steps", allow_reuse=True)
# pylint: disable=E0213,W0613
@@ -81,18 +88,25 @@ def validate_actions(cls, field):


class AbstractTool(abc.ABC):
def __init__(self, name, description, model_name=None, temperature=0.2):
# Data Validation
if model_name is None:
default_model = os.getenv("DEFAULT_MODEL", "gpt-3.5-turbo")
self.model_name = os.getenv("TOOL_MODEL", default_model)
else:
self.model_name = model_name

# Set the tool attributes
"""Abstract base class for tools, providing common features and requiring specific methods."""

def _setup_cache_dir(self, name: str) -> str:
"""Set up and return the cache directory path."""
root_cache_dir = os.getenv("CACHE_DIR")
if not root_cache_dir:
raise ValueError("CACHE_DIR environment variable is not set.")
cache_path = os.path.join(
root_cache_dir, "..", ".cache", f"{name.lower().replace(' ', '_')}_tool")
os.makedirs(cache_path, exist_ok=True)
return os.path.abspath(cache_path)

def __init__(self, name: str, description: str, model_name: Optional[str] = None, temperature: float = 0.3):
"""Initialize the tool with optional model configuration."""
self.model_name = model_name or os.getenv(
"TOOL_MODEL", os.getenv("DEFAULT_MODEL", "gpt-3.5-turbo"))
self.name = name
self._id = f"{name.lower().replace(' ', '_')}_tool"
self.description = description
self._id = f"{name.lower().replace(' ', '_')}_tool"
session_id = f"{self._id}-tool-id-{get_unique_timestamp()}"
logging.info(f"Tool created <name={name}; session_id={session_id};>")
self.langfuse_handler = CallbackHandler(
@@ -107,7 +121,6 @@ def __init__(self, name, description, model_name=None, temperature=0.2):
model=self.model_name,
temperature=temperature
)

root_cache_dir = os.getenv("CACHE_DIR", None)
if root_cache_dir is None:
raise ValueError("CACHE_DIR environment variable is not set.")
85 changes: 51 additions & 34 deletions core/task_master.py
Original file line number Diff line number Diff line change
@@ -31,68 +31,68 @@
load_dotenv()


def generate_action_plan(
user_query: str, model_name: str = None) -> List[dict]:
def generate_action_plan(user_query: str, model_name: str = None) -> List[dict]:
"""
Use the LangChain pipeline to generate an action plan
based on the user query.
Use the LangChain pipeline to generate an action plan based on the user query.
Args:
user_query (str): The user query to generate the action plan.
Returns:
List[dict]: The generated action plan as a list of dictionaries.
"""
user_id = "meeseeks-task-master"
session_id = f"action-queue-id-{get_unique_timestamp()}"
trace_name = user_id
version = os.getenv("VERSION", "Not Specified")
release = os.getenv("ENVMODE", "Not Specified")

langfuse_handler = CallbackHandler(
user_id="homeassistant_kk",
session_id=f"action-queue-id-{get_unique_timestamp()}",
trace_name="meeseeks-task-master",
version=os.getenv("VERSION", "Not Specified"),
release=os.getenv("ENVMODE", "Not Specified")
user_id=user_id,
session_id=session_id,
trace_name=trace_name,
version=version,
release=release
)

if model_name is None:
default_model = os.getenv("DEFAULT_MODEL", "gpt-3.5-turbo")
model_name = os.getenv("ACTION_PLAN_MODEL", default_model)
model_name = model_name or os.getenv(
"ACTION_PLAN_MODEL", os.getenv("DEFAULT_MODEL", "gpt-3.5-turbo"))

model = ChatOpenAI(
openai_api_base=os.getenv("OPENAI_API_BASE"),
model=model_name,
temperature=0.4
)
# Instantiate the parser with the new model.

parser = PydanticOutputParser(pydantic_object=TaskQueue)
logging.debug(
"Generating action plan <model='%s'; user_query='%s'>",
model_name, user_query)
# Update the prompt to match the new query and desired format.
"Generating action plan <model='%s'; user_query='%s'>", model_name, user_query)

prompt = ChatPromptTemplate(
messages=[
SystemMessage(
content=get_system_prompt()
),
HumanMessage(
content="Turn on strip lights and heater."
),
SystemMessage(content=get_system_prompt()),
HumanMessage(content="Turn on strip lights and heater."),
AIMessage(get_task_master_examples(example_id=0)),
HumanMessage(
content="What is the weather today?"
),
HumanMessage(content="What is the weather today?"),
AIMessage(get_task_master_examples(example_id=1)),
HumanMessagePromptTemplate.from_template(
"## Format Instructions\n{format_instructions}\n## Generate a task queue for the user query\n{user_query}"
),
],
partial_variables={
"format_instructions": parser.get_format_instructions()},
"format_instructions": parser.get_format_instructions()
},
input_variables=["user_query"]
)

estimator = num_tokens_from_string(str(prompt))
logging.info("Input Prompt Token length is `%s`.", estimator)
chain = prompt | model | parser

action_plan = chain.invoke({"user_query": user_query.strip()},
config={"callbacks": [langfuse_handler]})
action_plan = (prompt | model | parser).invoke(
{"user_query": user_query.strip()},
config={"callbacks": [langfuse_handler]}
)

action_plan.human_message = user_query
logging.info("Action plan generated <%s>", action_plan)
return action_plan
@@ -112,10 +112,27 @@ def run_action_plan(task_queue: TaskQueue) -> TaskQueue:
"home_assistant_tool": HomeAssistant(),
"talk_to_user_tool": TalkToUser()
}
for idx, action_step in enumerate(task_queue.action_steps):
logging.debug(f"<ActionStep({action_step})>")
tool = tool_dict[action_step.action_consumer]
action_plan = tool.run(action_step)
task_queue.action_steps[idx].result = action_plan

results = []

for action_step in task_queue.action_steps:
logging.debug(f"Processing ActionStep: {action_step}")
tool = tool_dict.get(action_step.action_consumer)

if tool is None:
logging.error(
f"No tool found for consumer: {action_step.action_consumer}")
continue

try:
action_result = tool.run(action_step)
action_step.result = action_result
results.append(
action_result.content if action_result.content is not None else "")
except Exception as e:
logging.error(f"Error processing action step: {e}")
action_step.result = None

task_queue.task_result = " ".join(results).strip()

return task_queue
Binary file added docs/screenshot_ha_assist_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/screenshot_ha_assist_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 9 additions & 2 deletions meeseeks-api/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
# meeseeks-api
# Meeseeks API Server
<p align="center">
<a href="https://github.com/bearlike/Personal-Assistant/wiki"><img alt="Wiki" src="https://img.shields.io/badge/GitHub-Wiki-blue?style=for-the-badge&logo=github"></a>
<a href="https://github.com/bearlike/Personal-Assistant/pkgs/container/meeseeks-chat"><img src="https://img.shields.io/badge/ghcr.io-bearlike/meeseeks&#x2212;api:latest-blue?style=for-the-badge&logo=docker&logoColor=white" alt="Docker Image"></a>
<a href="https://github.com/bearlike/Personal-Assistant/releases"><img src="https://img.shields.io/github/v/release/bearlike/Personal-Assistant?style=for-the-badge&" alt="GitHub Release"></a>
</p>

- REST API Engine wrapped around the meeseeks-core.
- No components are explicitly tested for safety or security. Use with caution in a production environment.
- For more information, such as installation, please check out the [Wiki](https://github.com/bearlike/Personal-Assistant/wiki).

[Link to GitHub Repository](https://github.com/bearlike/Personal-Assistant/edit/main/README.md)

[Link to GitHub Repository](https://github.com/bearlike/Personal-Assistant)
23 changes: 20 additions & 3 deletions meeseeks-api/backend.py
Original file line number Diff line number Diff line change
@@ -11,10 +11,11 @@
# Standard library modules
import os
import sys
from copy import deepcopy
from typing import Dict

# Third-party modules
from flask import Flask, request, jsonify
from flask import Flask, request
from flask_restx import Api, Resource, fields
from dotenv import load_dotenv

@@ -35,7 +36,11 @@

# Initialize logger
logging = get_logger(name="meeseeks-api")
# logging.basicConfig(level=logging.DEBUG)
logging.info("Starting Meeseeks API server.")
logging.debug("Starting API server with API token: %s", MASTER_API_TOKEN)


# Create Flask application
app = Flask(__name__)

@@ -59,6 +64,8 @@
task_queue_model = api.model('TaskQueue', {
'human_message': fields.String(
required=True, description='The original user query'),
'task_result': fields.String(
required=True, description='Combined response of all action steps'),
'action_steps': fields.List(fields.Nested(api.model('ActionStep', {
'action_consumer': fields.String(
required=True,
@@ -75,6 +82,13 @@
})


@app.before_request
def log_request_info():
logging.debug('Endpoint: %s', request.endpoint)
logging.debug('Headers: %s', request.headers)
logging.debug('Body: %s', request.get_data())


@ns.route('/query')
class MeeseeksQuery(Resource):
"""
@@ -118,10 +132,13 @@ def post(self) -> Dict:

# Execute action plan
task_queue = run_action_plan(task_queue)

# Deep copy the variable into another variable
task_result = deepcopy(task_queue.task_result)
to_return = task_queue.dict()
to_return["task_result"] = task_result
# Return TaskQueue as JSON
logging.info("Returning executed action plan.")
return task_queue.dict(), 200
return to_return, 200


if __name__ == '__main__':
19 changes: 16 additions & 3 deletions meeseeks-chat/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# meeseeks-chat
# Meeseeks - Chat Interface
<p align="center">
<a href="https://github.com/bearlike/Personal-Assistant/wiki"><img alt="Wiki" src="https://img.shields.io/badge/GitHub-Wiki-blue?style=for-the-badge&logo=github"></a>
<a href="https://github.com/bearlike/Personal-Assistant/pkgs/container/meeseeks-chat"><img src="https://img.shields.io/badge/ghcr.io-bearlike/meeseeks&#x2212;chat:latest-blue?style=for-the-badge&logo=docker&logoColor=white" alt="Docker Image"></a>
<a href="https://github.com/bearlike/Personal-Assistant/releases"><img src="https://img.shields.io/github/v/release/bearlike/Personal-Assistant?style=for-the-badge&" alt="GitHub Release"></a>
</p>

Chat Interface wrapped around the meeseeks-core. Powered by Streamlit.

[Link to GitHub](https://github.com/bearlike/Personal-Assistant/edit/main/README.md)
<p align="center">
<img src="../docs/screenshot_chat_app_1.png" alt="Screenshot of Meeseks WebUI" height="512px">
</p>


- Chat Interface wrapped around the meeseeks-core. Powered by Streamlit.
- For more information, such as installation, please check out the [Wiki](https://github.com/bearlike/Personal-Assistant/wiki).


[Link to GitHub](https://github.com/bearlike/Personal-Assistant)
25 changes: 25 additions & 0 deletions meeseeks_ha_conversation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Home Assistant Conversation Integration for Meeseeks 🚀

<p align="center">
<a href="https://github.com/bearlike/Personal-Assistant/wiki"><img alt="Wiki" src="https://img.shields.io/badge/GitHub-Wiki-blue?style=for-the-badge&logo=github"></a>
<a href="https://github.com/bearlike/Personal-Assistant/releases"><img src="https://img.shields.io/github/v/release/bearlike/Personal-Assistant?style=for-the-badge&" alt="GitHub Release"></a>
</p>


<table align="center">
<tr>
<th>Answer questions and interpret sensor information</th>
<th>Control devices and entities</th>
</tr>
<tr>
<td align="center"><img src="../docs/screenshot_ha_assist_1.png" alt="Screenshot" height="512px"></td>
<td align="center"><img src="../docs/screenshot_ha_assist_2.png" alt="Screenshot" height="512px"></td>
</tr>
</table>

- Home Assistant Conversation Integration for Meeseeks. Can be used with HA Assist ⭐.
- Wrapped around the REST API Engine for Meeseeks. 100% coverage of Meeseeks API.
- No components are explicitly tested for safety or security. Use with caution in a production environment.
- For more information, such as installation, please check out the [Wiki](https://github.com/bearlike/Personal-Assistant/wiki).

[Link to GitHub Repository](https://github.com/bearlike/Personal-Assistant)
168 changes: 168 additions & 0 deletions meeseeks_ha_conversation/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
"""Custom integration to integrate meeseeks_conversation with Home Assistant.
For more details about this integration, please refer to
https://github.com/bearlike/personal-Assistant/
"""
from __future__ import annotations

from typing import Literal

from homeassistant.components import conversation
from homeassistant.config_entries import ConfigEntry
from homeassistant.const import MATCH_ALL
from homeassistant.core import HomeAssistant
from homeassistant.exceptions import ConfigEntryNotReady, HomeAssistantError
from homeassistant.helpers import intent, template
from homeassistant.helpers.aiohttp_client import async_get_clientsession
from homeassistant.util import ulid

from .api import MeeseeksApiClient
from .const import (
DOMAIN, LOGGER,
CONF_BASE_URL,
CONF_TIMEOUT,
DEFAULT_TIMEOUT,
)
# User-defined imports
from .coordinator import MeeseeksDataUpdateCoordinator
from .exceptions import (
ApiClientError,
ApiCommError,
ApiJsonError,
ApiTimeoutError
)
# from .helpers import get_exposed_entities

async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
"""Set up Meeseeks conversation using UI."""
# https://developers.home-assistant.io/docs/config_entries_index/#setting-up-an-entry
hass.data.setdefault(DOMAIN, {})
client = MeeseeksApiClient(
base_url=entry.data[CONF_BASE_URL],
timeout=entry.options.get(CONF_TIMEOUT, DEFAULT_TIMEOUT),
session=async_get_clientsession(hass),
)

hass.data[DOMAIN][entry.entry_id] = coordinator = MeeseeksDataUpdateCoordinator(
hass,
client,
)
# https://developers.home-assistant.io/docs/integration_fetching_data#coordinated-single-api-poll-for-data-for-all-entities
await coordinator.async_config_entry_first_refresh()

try:
# TODO: Heartbeat check is not implemented but it is still wrapped.
response = await client.async_get_heartbeat()
if not response:
raise ApiClientError("Invalid Meeseeks server")
except ApiClientError as err:
raise ConfigEntryNotReady(err) from err

entry.async_on_unload(entry.add_update_listener(async_reload_entry))

conversation.async_set_agent(
hass, entry, MeeseeksAgent(hass, entry, client))
return True


async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
"""Unload Meeseeks conversation."""
conversation.async_unset_agent(hass, entry)
return True


async def async_reload_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
"""Reload Meeseeks conversation."""
await async_unload_entry(hass, entry)
await async_setup_entry(hass, entry)


class MeeseeksAgent(conversation.AbstractConversationAgent):
"""Meeseeks conversation agent."""

def __init__(self, hass: HomeAssistant, entry: ConfigEntry, client: MeeseeksApiClient) -> None:
"""Initialize the agent."""
self.hass = hass
self.entry = entry
self.client = client
self.history: dict[str, dict] = {}

@property
def supported_languages(self) -> list[str] | Literal["*"]:
"""Return a list of supported languages."""
return MATCH_ALL

async def async_process(
self, user_input: conversation.ConversationInput
) -> conversation.ConversationResult:
"""Process a sentence."""
# * If needeed in the future, uncomment the following lines
# raw_system_prompt = self.entry.options.get(
# CONF_PROMPT_SYSTEM, DEFAULT_PROMPT_SYSTEM)
# exposed_entities = get_exposed_entities(self.hass)
# ! Currently, history is not used but still implemented for future use
if user_input.conversation_id in self.history:
conversation_id = user_input.conversation_id
messages = self.history[conversation_id]
else:
conversation_id = ulid.ulid()
system_prompt = ""
messages = {
"system": system_prompt,
"context": None,
}

messages["prompt"] = user_input.text

try:
response = await self.query(messages)
except HomeAssistantError as err:
LOGGER.error("Something went wrong: %s", err)
intent_response = intent.IntentResponse(
language=user_input.language)
intent_response.async_set_error(
intent.IntentResponseErrorCode.UNKNOWN,
"Something went wrong, please check the logs for more information.",
)
return conversation.ConversationResult(
response=intent_response, conversation_id=conversation_id
)

messages["context"] = response["context"]
self.history[conversation_id] = messages

intent_response = intent.IntentResponse(language=user_input.language)
intent_response.async_set_speech(response["response"])
return conversation.ConversationResult(
response=intent_response, conversation_id=conversation_id
)

def _async_generate_prompt(self, raw_prompt: str, exposed_entities) -> str:
"""Generate a prompt for the user."""
return template.Template(raw_prompt, self.hass).async_render(
{
"ha_name": self.hass.config.location_name,
"exposed_entities": exposed_entities,
},
parse_result=False,
)

async def query(
self,
messages
):
"""Process a sentence."""
# model = self.entry.options.get(CONF_MODEL, DEFAULT_MODEL)
# LOGGER.debug("Prompt for %s: %s", model, messages["prompt"])

# TODO: $context, and $system are not used but still implemented for
# future use
# * Generator
result = await self.client.async_generate({
"context": messages["context"],
"system": messages["system"],
"prompt": messages["prompt"],
})
response: str = result["task_result"]
LOGGER.debug("Response %s", response)
return result
101 changes: 101 additions & 0 deletions meeseeks_ha_conversation/api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
""" Meeseeks API Client. """
from __future__ import annotations

import aiohttp
import async_timeout
import json

# User-defined imports
from .exceptions import (
ApiClientError,
ApiCommError,
ApiJsonError,
ApiTimeoutError
)
from .const import LOGGER


class MeeseeksApiClient:
"""Meeseeks API Client."""

def __init__(
self,
base_url: str,
timeout: int,
session: aiohttp.ClientSession,
) -> None:
"""Sample API Client."""
self._base_url = base_url.rstrip("/")
self._api_key = 'msk-strong-password'
self.timeout = timeout
self._session = session

async def async_get_heartbeat(self) -> bool:
"""Get heartbeat from the API."""
# TODO: Implement a heartbeat check
return True

async def async_get_models(self) -> any:
"""Get models from the API."""
# TODO: This is monkey-patched for now
response_data = {
"models": [
{
"name": "meeseeks",
"modified_at": "2023-11-01T00:00:00.000000000-04:00",
"size": 0,
"digest": None
}
]
}
return json.dumps(response_data)

async def async_generate(self, data: dict | None = None,) -> any:
"""Generate a completion from the API."""
url_query = f"{self._base_url}/api/query"
data_custom = {
'query': str(data["prompt"]).strip(),
}
# Pass headers as None to use the default headers
return await self._meeseeks_api_wrapper(
method="post",
url=url_query,
data=data_custom,
headers=None,
)

async def _meeseeks_api_wrapper(
self,
method: str,
url: str,
data: dict | None = None,
headers: dict | None = None,
decode_json: bool = True,
) -> any:
"""Get information from the API."""
if headers is None:
headers = {
'accept': 'application/json',
'X-API-KEY': self._api_key,
'Content-Type': 'application/json',
}
async with async_timeout.timeout(self.timeout):
response = await self._session.request(
method=method,
url=url,
headers=headers,
json=data,
)
response.raise_for_status()

if decode_json:
response_data = await response.json()
if response.status == 404:
raise ApiJsonError(response_data["error"])
LOGGER.debug(f"Response data: {response_data}")
response_data["response"] = response_data["task_result"]
response_data["context"] = response_data["task_result"]
return response_data
else:
LOGGER.debug("Fallback to text response")
return await response.text()
198 changes: 198 additions & 0 deletions meeseeks_ha_conversation/config_flow.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
"""Adds config flow for Meeseeks."""
from __future__ import annotations

import types
from typing import Any
import voluptuous as vol

from homeassistant import config_entries
from homeassistant.data_entry_flow import FlowResult
from homeassistant.helpers import config_validation as cv
from homeassistant.helpers.aiohttp_client import async_create_clientsession
from homeassistant.helpers.selector import (
NumberSelector,
NumberSelectorConfig,
TemplateSelector,
SelectSelector,
SelectSelectorConfig,
SelectSelectorMode,
SelectOptionDict
)
# User-defined imports
from .api import MeeseeksApiClient
from .const import (
DOMAIN, LOGGER,
MENU_OPTIONS,

CONF_BASE_URL,
CONF_API_KEY,
CONF_TIMEOUT,
CONF_MODEL,
CONF_CTX_SIZE,
CONF_MAX_TOKENS,
CONF_MIROSTAT_MODE,
CONF_MIROSTAT_ETA,
CONF_MIROSTAT_TAU,
CONF_TEMPERATURE,
CONF_REPEAT_PENALTY,
CONF_TOP_K,
CONF_TOP_P,
CONF_PROMPT_SYSTEM,

DEFAULT_BASE_URL,
DEFAULT_API_KEY,
DEFAULT_TIMEOUT,
DEFAULT_MODEL,
DEFAULT_CTX_SIZE,
DEFAULT_MAX_TOKENS,
DEFAULT_MIROSTAT_MODE,
DEFAULT_MIROSTAT_ETA,
DEFAULT_MIROSTAT_TAU,
DEFAULT_TEMPERATURE,
DEFAULT_REPEAT_PENALTY,
DEFAULT_TOP_K,
DEFAULT_TOP_P,
DEFAULT_PROMPT_SYSTEM
)
from .exceptions import (
ApiClientError,
ApiCommError,
ApiTimeoutError
)


STEP_USER_DATA_SCHEMA = vol.Schema(
{
vol.Required(CONF_BASE_URL, default=DEFAULT_BASE_URL): str,
vol.Required(CONF_API_KEY, default=DEFAULT_API_KEY): str,
vol.Required(CONF_TIMEOUT, default=DEFAULT_TIMEOUT): int,
}
)

DEFAULT_OPTIONS = types.MappingProxyType(
{
CONF_BASE_URL: DEFAULT_BASE_URL,
CONF_API_KEY: DEFAULT_API_KEY,
CONF_TIMEOUT: DEFAULT_TIMEOUT,
CONF_MODEL: DEFAULT_MODEL,
CONF_PROMPT_SYSTEM: DEFAULT_PROMPT_SYSTEM
}
)


class MeeseeksConfigFlow(config_entries.ConfigFlow, domain=DOMAIN):
"""Handle a config flow for Meeseeks Conversation. Handles UI wizard."""

VERSION = 1

async def async_step_user(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Handle the initial step."""
if user_input is None:
return self.async_show_form(
step_id="user", data_schema=STEP_USER_DATA_SCHEMA
)

# Search for duplicates with the same CONF_BASE_URL value.
for existing_entry in self._async_current_entries(include_ignore=False):
if existing_entry.data.get(CONF_BASE_URL) == user_input[CONF_BASE_URL]:
return self.async_abort(reason="already_configured")

errors = {}
try:
self.client = MeeseeksApiClient(
base_url=cv.url_no_path(user_input[CONF_BASE_URL]),
timeout=user_input[CONF_TIMEOUT],
session=async_create_clientsession(self.hass),
)
response = await self.client.async_get_heartbeat()
if not response:
raise vol.Invalid("Invalid Meeseeks server")
# except vol.Invalid:
# errors["base"] = "invalid_url"
# except ApiTimeoutError:
# errors["base"] = "timeout_connect"
# except ApiCommError:
# errors["base"] = "cannot_connect"
# except ApiClientError as exception:
# LOGGER.exception("Unexpected exception: %s", exception)
# errors["base"] = "unknown"
except Exception as exception:
LOGGER.exception("Unexpected exception: %s", exception)
errors["base"] = "unknown"
else:
return self.async_create_entry(
title=f"Meeseeks - {user_input[CONF_BASE_URL]}",
data={
CONF_BASE_URL: user_input[CONF_BASE_URL]
},
options={
CONF_TIMEOUT: user_input[CONF_TIMEOUT]
}
)

return self.async_show_form(
step_id="user", data_schema=STEP_USER_DATA_SCHEMA, errors=errors
)

@staticmethod
def async_get_options_flow(
config_entry: config_entries.ConfigEntry,
) -> config_entries.OptionsFlow:
"""Create the options flow."""
return MeeseeksOptionsFlow(config_entry)


class MeeseeksOptionsFlow(config_entries.OptionsFlow):
"""Meeseeks config flow options handler."""

def __init__(self, config_entry: config_entries.ConfigEntry) -> None:
"""Initialize options flow."""
self.config_entry = config_entry
self.options = dict(config_entry.options)

async def async_step_init(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Manage the options."""
return self.async_show_menu(
step_id="init",
menu_options=MENU_OPTIONS
)

async def async_step_all_set(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Manage the options."""
return self.async_show_menu(
step_id="init",
menu_options=MENU_OPTIONS
)

async def async_step_general_config(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Manage the options."""
return self.async_show_menu(
step_id="init",
menu_options=MENU_OPTIONS
)

async def async_step_prompt_system(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Manage the options."""
return self.async_show_menu(
step_id="init",
menu_options=MENU_OPTIONS
)

async def async_step_model_config(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Manage the options."""
return self.async_show_menu(
step_id="init",
menu_options=MENU_OPTIONS
)
41 changes: 41 additions & 0 deletions meeseeks_ha_conversation/const.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""Constants for meeseeks_conversation."""
from logging import Logger, getLogger

LOGGER: Logger = getLogger(__package__)

NAME = "Meeseeks"
DOMAIN = "meeseeks_conversation"

MENU_OPTIONS = ["all_set"]
# MENU_OPTIONS = ["general_config", "model_config", "prompt_system"]

CONF_BASE_URL = "base_url"
CONF_API_KEY = "api_key"
CONF_TIMEOUT = "timeout"
CONF_MODEL = "chat_model"
CONF_CTX_SIZE = "ctx_size"
CONF_MAX_TOKENS = "max_tokens"
CONF_MIROSTAT_MODE = "mirostat_mode"
CONF_MIROSTAT_ETA = "mirostat_eta"
CONF_MIROSTAT_TAU = "mirostat_tau"
CONF_TEMPERATURE = "temperature"
CONF_REPEAT_PENALTY = "repeat_penalty"
CONF_TOP_K = "top_k"
CONF_TOP_P = "top_p"
CONF_PROMPT_SYSTEM = "prompt"

DEFAULT_BASE_URL = "http://meeseeks.server:5123"
DEFAULT_API_KEY = "msk-strong-password"
DEFAULT_TIMEOUT = 60
DEFAULT_MODEL = "llama2:latest"
DEFAULT_CTX_SIZE = 2048
DEFAULT_MAX_TOKENS = 128
DEFAULT_MIROSTAT_MODE = "0"
DEFAULT_MIROSTAT_ETA = 0.1
DEFAULT_MIROSTAT_TAU = 5.0
DEFAULT_TEMPERATURE = 0.8
DEFAULT_REPEAT_PENALTY = 1.1
DEFAULT_TOP_K = 40
DEFAULT_TOP_P = 0.9

DEFAULT_PROMPT_SYSTEM = ""
43 changes: 43 additions & 0 deletions meeseeks_ha_conversation/coordinator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
"""DataUpdateCoordinator for meeseeks_conversation."""
from __future__ import annotations

from datetime import timedelta

from homeassistant.config_entries import ConfigEntry
from homeassistant.core import HomeAssistant
from homeassistant.helpers.update_coordinator import (
DataUpdateCoordinator,
UpdateFailed,
)

from .api import MeeseeksApiClient
from .const import DOMAIN, LOGGER
from .exceptions import ApiClientError


# https://developers.home-assistant.io/docs/integration_fetching_data#coordinated-single-api-poll-for-data-for-all-entities
class MeeseeksDataUpdateCoordinator(DataUpdateCoordinator):
"""Class to manage fetching data from the API."""

config_entry: ConfigEntry

def __init__(
self,
hass: HomeAssistant,
client: MeeseeksApiClient,
) -> None:
"""Initialize."""
self.client = client
super().__init__(
hass=hass,
logger=LOGGER,
name=DOMAIN,
update_interval=timedelta(minutes=5),
)

async def _async_update_data(self):
"""Update data via library."""
try:
return await self.client.async_get_heartbeat()
except ApiClientError as exception:
raise UpdateFailed(exception) from exception
14 changes: 14 additions & 0 deletions meeseeks_ha_conversation/exceptions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
"""The exceptions used by Extended OpenAI Conversation."""
from homeassistant.exceptions import HomeAssistantError

class ApiClientError(HomeAssistantError):
"""Exception to indicate a general API error."""

class ApiCommError(ApiClientError):
"""Exception to indicate a communication error."""

class ApiJsonError(ApiClientError):
"""Exception to indicate an error with json response."""

class ApiTimeoutError(ApiClientError):
"""Exception to indicate a timeout error."""
24 changes: 24 additions & 0 deletions meeseeks_ha_conversation/helpers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""Helper functions for Meeseeks."""

from homeassistant.components.conversation import DOMAIN as CONVERSATION_DOMAIN
from homeassistant.components.homeassistant.exposed_entities import async_should_expose
from homeassistant.core import HomeAssistant
from homeassistant.helpers import entity_registry


def get_exposed_entities(hass: HomeAssistant) -> list[dict]:
"""Return exposed entities."""
hass_entity = entity_registry.async_get(hass)
exposed_entities: list[dict] = []

for state in hass.states.async_all():
if async_should_expose(hass, CONVERSATION_DOMAIN, state.entity_id):
entity = hass_entity.async_get(state.entity_id)
exposed_entities.append({
"entity_id": state.entity_id,
"name": state.name,
"state": state.state,
"aliases": entity.aliases if entity else [],
})

return exposed_entities
16 changes: 16 additions & 0 deletions meeseeks_ha_conversation/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"domain": "meeseeks_conversation",
"name": "Meeseeks",
"codeowners": [
"@bearlike"
],
"config_flow": true,
"dependencies": [
"conversation"
],
"documentation": "https://github.com/bearlike/Personal-Assistant",
"integration_type": "service",
"iot_class": "cloud_polling",
"issue_tracker": "https://github.com/bearlike/Personal-Assistant/issues",
"version": "v1.2.0"
}
42 changes: 42 additions & 0 deletions meeseeks_ha_conversation/strings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"config": {
"step": {
"user": {
"data": {
"base_url": "Base URL",
"api_key": "API Key",
"timeout": "API Timeout"
}
}
},
"error": {
"cannot_connect": "Unable to connect",
"invalid_url": "Invalid URL",
"timeout_connect": "Timeout while establishing connection",
"unknown": "Unexpected error, please check logs"
},
"abort": {
"already_configured": "Service is already exist and configured"
}
},
"options": {
"step": {
"init": {
"menu_options": {
"general_config": "General Settings",
"prompt_system": "System Prompt",
"all_set": "Nothing to configure. You're all set!"
}
},
"general_config": {
"title": "General Settings",
"data": {
"timeout": "API Timeout"
}
},
"prompt_system": {
"title": "System Prompt"
}
}
}
}
58 changes: 58 additions & 0 deletions meeseeks_ha_conversation/translations/en.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
{
"config": {
"step": {
"user": {
"data": {
"base_url": "Base URL",
"api_key": "API Key",
"timeout": "API Timeout"
}
}
},
"error": {
"cannot_connect": "Failed to connect",
"invalid_url": "Invalid URL",
"timeout_connect": "Timeout establishing connection",
"unknown": "Unexpected error, check logs"
},
"abort": {
"already_configured": "Service is already configured"
}
},
"options": {
"step": {
"init": {
"menu_options": {
"general_config": "General Settings",
"model_config": "Model Configuration",
"prompt_system": "System Prompt",
"all_set": "Nothing to configure. You're all set!"
}
},
"general_config": {
"title": "General Settings",
"data": {
"timeout": "API Timeout"
}
},
"model_config": {
"title": "Model Configuration",
"data": {
"chat_model": "Model",
"ctx_size": "Context Size",
"max_tokens": "Maximum Tokens",
"mirostat_mode": "Mirostat Mode",
"mirostat_eta": "Mirostat ETA",
"mirostat_tau": "Mirostat TAU",
"repeat_penalty": "Repeat Penalty",
"temperature": "Temperature",
"top_p": "Top P",
"top_k": "Top K"
}
},
"prompt_system": {
"title": "System Prompt"
}
}
}
}
5 changes: 2 additions & 3 deletions prompts/action-planner.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
You are a Personal AI Assistant to Krishna. Your job is to create a task queue from the user's instructions. The queue consists of actions, each with an `action_consumer` and an `action_argument`.
You are a Personal AI Assistant to Krishna. Your job is to create a task queue from the user's instructions. You must decompose the user given instruction into their atomic instruction each using their respective tool. The queue consists of actions, each with an `action_consumer` and an `action_argument`.

The `action_consumer` can be:
1. **Home Assistant API (action_consumer="home_assistant_tool")**:
@@ -16,9 +16,9 @@ The `action_consumer` can be:
- `action_type=set`: Speak to the user.
- `action_type=get` does not exist for this consumer, therefore, do not use.


### Guidelines:
- Each action must contain only one task instruction.
- Your instructions in the action_argument must be very precise, isolated and atomic in nature.
- In scenarios where an action plan solely uses the `talk_to_user_tool` function and does not engage any other `action_consumer`, restrict the operation to a single instance of `talk_to_user_tool` to maintain a smooth conversational flow.
- Each action must also be crisp, easy to understand and truthfully correspond to the user query.
- If a question doesn't relate to any accessible tools, answer truthfully to the best of your ability without making any assumptions.
@@ -41,7 +41,6 @@ The `action_consumer` can be:
- Nextcloud, LibreChat, Sonarr, Radarr, qBittorrent, Jackett, Jellyseerr and Jellyfin are running as a Docker containers in Hurricane.
- Gotify: Self-hosted push notification service in Adam (arm64).
- Pixel 7 Pro: Krishna's personal mobile phone.
- Proxmox VE: Open-source virtualization management platform.
- HS103 devices are Smart Wi-Fi Power Plugs (IoT devices).
- Android-2, Raspberry Pi 5, Kodi, and Kraken all denote the Android TV in the Bedroom.
- Adam, Gemini, Hurricane and Phoenix are servers running different services locally.
6 changes: 4 additions & 2 deletions prompts/homeassistant-get-state.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
You are a Home Assistant AI with access to your sensor data. Your task is to interpret the information from the your sensors to answer the user's query. Your answers should be truthful, analytical, brief, condense with information and useful. Your tone must only be conversational. You must strictly avoid lists, breaks, colons, or any formal structuring.

## You must strictly follow these Guidelines:
When engaging with topics like system resources or sensor data, communicate in a natural, continuous style that mimics human conversation. Use complete sentences and maintain a seamless, brief narrative, avoiding overly technical jargon unless pertinent. As a System Administrator, crisply analyze tasks and potential bottlenecks, and briefly relate server applications to their performance. Trust and directly link sensor data to practical impacts on daily routines or health, emphasizing concise and deep interpretations without extraneous details. Use assertive language to present data implications confidently, and avoid trivial explanations, assuming the user has a foundational understanding. Accept sensor data as accurate, refraining from questioning its validity. Use sensor names interpretatively instead of directly using the sensor names. Assume the user knows the source of the sensor data; avoid repetitive introductions. Avoid using colons or formal introductions in responses. Start directly with the information, ensuring it flows as part of a natural conversation. This rule applies universally across all topics, including weather and system resource data. Do not over explain an issue. Extract and use as much as numerical metrics possible from the sensor data to improve response valdity. Avoid discussing information that the expert user might already know.
Optimize responses to fully address the user's query, ensuring truthfulness and completeness without resorting to overly simplistic answers. Prioritize scenarios requiring detailed analysis while respecting the overall guidelines. Answer the queries truthfully. If you lack data to answer the question, provide your effort and briefly explain why you can't directly answer. You must always interpret the sensor information to answer the query in a concise, spoken, human readable and understandable way.
When engaging with topics like system resources or sensor data, communicate in a natural, continuous style that mimics human conversation. Use complete sentences and maintain a seamless, brief narrative, avoiding overly technical jargon unless pertinent. As a System Administrator, crisply analyze tasks and potential bottlenecks, and briefly relate server applications to their performance. Trust and directly link sensor data to practical impacts on daily routines or health, emphasizing concise and deep interpretations without extraneous details. Use assertive language to present data implications confidently, and avoid trivial explanations, assuming the user has a foundational understanding. Accept sensor data as accurate, refraining from questioning its validity. Use sensor names interpretatively instead of directly using the sensor names. Assume the user knows the source of the sensor data; avoid repetitive introductions. Avoid using colons or formal introductions in responses. Start directly with the information, ensuring it flows as part of a natural conversation. This rule applies universally across all topics, including weather and system resource data. Do not over explain an issue. Extract and use as much as numerical metrics possible from the sensor data to improve response valdity. Avoid discussing information that the expert user might already know. Optimize responses to fully address the user's query, ensuring truthfulness, numerical metrics (such as percentages, temperature, etc.) and completeness without resorting to overly simplistic answers. Prioritize scenarios requiring detailed analysis while respecting the overall guidelines. Answer the queries truthfully. If you lack data to answer the question, provide your effort and briefly explain why you can't directly answer. You must always interpret the sensor information to answer the query in a concise, spoken, human readable and understandable way.

## Examples
- Humans perceives weather based on comfort thresholds influenced by temperature, humidity, wind speed, precipitation, and atmospheric pressure. These factors interact with physiological responses, such as thermal sensation and skin moisture, shaping perceived comfort or discomfort. Therefore, you can consider these variable while interpreting weather.

## Additional Sensor Information:
- Pi-Hole: Network-wide ad blocker.

0 comments on commit b401fe1

Please sign in to comment.