Feat/add file upload UI

README.md

@@ -493,6 +493,65 @@ python examples/run_azure_openai.py
 python examples/run_ollama.py
 ```
 
+### Using with OpenRouter
+
+[OpenRouter](https://openrouter.ai/) provides access to a wide variety of models through a single API. OWL integrates with OpenRouter, including advanced features like API key pooling and rotation.
+
+**1. Setup OpenRouter API Key:**
+
+Get your API key from the [OpenRouter website](https://openrouter.ai/keys). Add it to your `.env` file:
+
+```
+OPENROUTER_API_KEY='your-key-here'
+```
+
+For enhanced reliability, you can provide a comma-separated list of keys. The system will automatically rotate through them and put failing keys on a temporary cooldown.
+
+```
+OPENROUTER_API_KEY='key1,key2,key3'
+```
+
+**2. Run via Web UI:**
+
+1.  Start the web application: `python owl/webapp.py`
+2.  From the "Select Function Module" dropdown, choose `run_openrouter`.
+3.  A new text box will appear. Enter the model identifier from OpenRouter (e.g., `mistralai/mistral-7b-instruct`).
+4.  Enter your question and click "Run".
+
+### Using with Local Models (Ollama)
+
+You can run OWL with local models served by [Ollama](https://ollama.com/). This allows you to run the entire system on your own machine, even without an internet connection.
+
+**1. Setup Ollama:**
+
+First, make sure you have Ollama installed and running. You can download it from the [Ollama website](https://ollama.com/).
+
+After installation, pull the models you want to use. For example, to get the Llama 3 and Llava (for vision) models, run the following commands in your terminal:
+
+```bash
+ollama run llama3
+ollama run llava
+```
+
+**2. Run via Web UI:**
+
+The easiest way to use a local model is through the web interface:
+
+1.  Start the web application: `python owl/webapp.py`
+2.  From the "Select Function Module" dropdown, choose `run_ollama`.
+3.  A new text box will appear. Enter the name of the text model you want to use (e.g., `llama3`). The vision model is currently defaulted to `llava`.
+4.  Enter your question and click "Run".
+
+**3. (Optional) Custom Server URL:**
+
+If your Ollama server is not running on the default `http://localhost:11434`, you can configure the URL by setting an environment variable:
+
+```bash
+export OLLAMA_API_BASE_URL="http://your-ollama-host:11434"
+```
+
+You can also add this line to your `.env` file.
+
 For a simpler version that only requires an LLM API key, you can try our minimal example:
 
 ```bash
@@ -535,6 +594,33 @@ Here are some tasks you can try with OWL:
 - "Summarize the main points from this research paper: [paper URL]"
 - "Create a data visualization for this dataset: [dataset path]"
 
+# 🚀 Advanced Usage
+
+## Autonomous Developer Agent
+
+OWL now includes an experimental autonomous developer agent that can perform upgrades and modify its own codebase. This "Daemon Developer" runs in a continuous loop to improve the application.
+
+**Capabilities:**
+- **Code Introspection**: Can list, read, and search through its own source code files.
+- **Self-Upgrade from Git**: Can check for remote `git` updates and apply them safely using a backup-test-restore workflow.
+- **Self-Modification**: Can work on a "backlog" of development tasks by programmatically modifying its own code.
+
+**How to Run:**
+
+You can start this agent from the Web UI:
+1.  Start the web application: `python owl/webapp.py`
+2.  From the "Select Function Module" dropdown, choose `run_developer_daemon`.
+3.  Click "Run".
+
+> **Warning**: This is a persistent process that will run indefinitely in your terminal. To stop it, you will need to press `Ctrl+C` in the terminal where you launched the web app.
+
+**Security Considerations:**
+This feature is highly experimental and grants the AI agent significant control over its own source code and execution environment. While safeguards are in place (restricting file writes and script execution to specific project directories), this capability carries inherent risks.
+
+As an additional security measure, high-risk tools (`write_file` and `run_upgrade_from_git`) now require **human-in-the-loop confirmation**. When the agent attempts to use these tools, it will print a security prompt in the terminal where the app is running and wait for you to type `yes` before proceeding.
+
+It is strongly recommended to run this agent in a sandboxed or containerized environment and to carefully review any action before approving it.
+
 # 🧰 Toolkits and Capabilities
 
 ## Model Context Protocol (MCP)
@@ -608,6 +694,7 @@ Key toolkits include:
 - **CodeExecutionToolkit**: Python code execution and evaluation
 - **SearchToolkit**: Web searches (Google, DuckDuckGo, Wikipedia)
 - **DocumentProcessingToolkit**: Document parsing (PDF, DOCX, etc.)
+- **CSVToolkit**: Read, write, and query data in CSV files.
 
 Additional specialized toolkits: ArxivToolkit, GitHubToolkit, GoogleMapsToolkit, MathToolkit, NetworkXToolkit, NotionToolkit, RedditToolkit, WeatherToolkit, and more. For a complete list, see the [CAMEL toolkits documentation](https://docs.camel-ai.org/key_modules/tools.html#built-in-toolkits).
 
@@ -659,10 +746,10 @@ python owl/webapp_jp.py
 
 ## Features
 
-- **Easy Model Selection**: Choose between different models (OpenAI, Qwen, DeepSeek, etc.)
-- **Environment Variable Management**: Configure your API keys and other settings directly from the UI
-- **Interactive Chat Interface**: Communicate with OWL agents through a user-friendly interface
-- **Task History**: View the history and results of your interactions
+- **Real-time Conversation Streaming**: Watch the agent conversation unfold in real-time in the "Conversation" tab.
+- **Easy Model Selection**: Choose between different models and providers (OpenAI, OpenRouter, Ollama, etc.).
+- **Environment Variable Management**: Configure your API keys and other settings directly from the UI.
+- **Full Log Viewer**: Access the detailed, raw logs in the "Full Logs" tab for debugging.
 
 The web interface is built using Gradio and runs locally on your machine. No data is sent to external servers beyond what's required for the model API calls you configure.
 

examples/data/sample_employees.csv

@@ -0,0 +1,8 @@
+EmployeeID,FirstName,LastName,Department,Salary
+101,John,Doe,Engineering,90000
+102,Jane,Smith,Marketing,75000
+103,Peter,Jones,Engineering,95000
+104,Mary,Johnson,Sales,80000
+105,David,Williams,Marketing,78000
+106,Emily,Brown,Engineering,120000
+107,Michael,Davis,Sales,82000

examples/run_csv_task.py

@@ -0,0 +1,102 @@
+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
+from dotenv import load_dotenv
+from camel.models import ModelFactory
+from camel.types import ModelType
+from camel.societies import RolePlaying
+from camel.logger import set_log_level
+
+# Assuming the new toolkit is in this path
+from owl.utils.csv_toolkit import CSVToolkit
+from owl.utils import run_society
+
+# Setup environment
+import pathlib
+base_dir = pathlib.Path(__file__).parent.parent
+env_path = base_dir / "owl" / ".env"
+load_dotenv(dotenv_path=str(env_path))
+
+set_log_level(level="INFO")
+
+
+def construct_society(question: str) -> RolePlaying:
+    """
+    Constructs a society of agents for the CSV processing task.
+    """
+    # Define the model for the agent
+    model = ModelFactory.create(
+        model_platform="openai",
+        model_type=ModelType.GPT_4O,
+        model_config_dict={"temperature": 0.2},
+    )
+
+    # Instantiate the toolkit
+    csv_toolkit = CSVToolkit()
+
+    # Get the tool functions from the toolkit instance
+    tools = [
+        csv_toolkit.read_csv,
+        csv_toolkit.write_csv,
+        csv_toolkit.query_csv,
+    ]
+
+    # Configure agent roles and parameters
+    user_agent_kwargs = {"model": model}
+    assistant_agent_kwargs = {"model": model, "tools": tools}
+
+    # Configure task parameters
+    task_kwargs = {
+        "task_prompt": question,
+        "with_task_specify": False,
+    }
+
+    # Create and return the society
+    society = RolePlaying(
+        **task_kwargs,
+        user_role_name="user",
+        user_agent_kwargs=user_agent_kwargs,
+        assistant_role_name="assistant",
+        assistant_agent_kwargs=assistant_agent_kwargs,
+    )
+
+    return society
+
+
+def main():
+    """Main function to run the CSV processing workflow."""
+
+    # The detailed task prompt that guides the agent
+    task_prompt = """
+    Your task is to process an employee data file. You must follow these steps:
+    1.  First, read the data from the CSV file located at `examples/data/sample_employees.csv`.
+    2.  Next, query this data to find all employees who are in the 'Engineering' department.
+    3.  Finally, take the filtered list of engineers and write it to a new CSV file named `engineers.csv`.
+    After you have written the new file, report that the task is complete and state how many engineers were found and saved.
+    """
+
+    # Construct and run the society
+    society = construct_society(task_prompt)
+    answer, chat_history, token_count = run_society(society)
+
+    # Output the final result from the agent
+    print("\n" + "="*30)
+    print("CSV Processing Task Final Report:")
+    print("="*30)
+    print(f"\033[94m{answer}\033[0m")
+    print("\nToken usage information:")
+    print(token_count)
+
+
+if __name__ == "__main__":
+    main()

examples/run_developer_daemon.py

@@ -0,0 +1,120 @@
+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
+import time
+import pathlib
+from dotenv import load_dotenv
+from camel.models import ModelFactory
+from camel.types import ModelType
+from camel.societies import RolePlaying
+from camel.logger import set_log_level
+
+from owl.utils.developer_toolkit import DeveloperToolkit
+from owl.utils import run_society
+
+# Setup environment
+base_dir = pathlib.Path(__file__).parent.parent
+env_path = base_dir / "owl" / ".env"
+load_dotenv(dotenv_path=str(env_path))
+
+set_log_level(level="INFO")
+
+
+def construct_society(question: str) -> RolePlaying:
+    """
+    Constructs a society of agents for the developer daemon task.
+    """
+    model = ModelFactory.create(
+        model_platform="openai",
+        model_type=ModelType.GPT_4O,
+        model_config_dict={"temperature": 0.2},
+    )
+
+    developer_toolkit = DeveloperToolkit()
+    tools = [
+        developer_toolkit.list_files,
+        developer_toolkit.read_file,
+        developer_toolkit.write_file,
+        developer_toolkit.check_for_git_updates,
+        developer_toolkit.run_tests,
+        developer_toolkit.run_upgrade_from_git,
+    ]
+
+    assistant_agent_kwargs = {"model": model, "tools": tools}
+    user_agent_kwargs = {"model": model}
+
+    return RolePlaying(
+        task_prompt=question,
+        user_role_name="user",
+        assistant_role_name="assistant",
+        user_agent_kwargs=user_agent_kwargs,
+        assistant_agent_kwargs=assistant_agent_kwargs,
+    )
+
+
+def main():
+    """Main function to run the developer daemon."""
+    print("Starting Daemon Developer Agent...")
+    print("This agent will run in a continuous loop to improve the codebase.")
+
+    # This backlog can be expanded with more development tasks.
+    development_backlog = [
+        "1. Add a new function to the `CSVToolkit` in `owl/utils/csv_toolkit.py` called `get_row_count` that reads a CSV and returns the number of rows (excluding the header).",
+        "2. Refactor the `SystemToolkit` to use the new `DeveloperToolkit`'s `_run_command` helper to reduce code duplication.",
+        # Add more tasks here in the future.
+    ]
+
+    while True:
+        print("\n" + "="*50)
+        print("Starting new development cycle...")
+        print("="*50)
+
+        task_prompt = f"""
+        Your goal is to continuously improve this application. Your workflow for this cycle is as follows:
+
+        **IMPORTANT NOTE:** Some of your tools, like `write_file` and `run_upgrade_from_git`, are high-risk and require human approval before they can execute. You must check the output of these tools carefully. If the output says 'Action cancelled by user', you must stop the current task and report the cancellation.
+
+        1.  **Check for External Updates:** First, call the `check_for_git_updates` tool to see if there are any new commits in the main repository.
+
+        2.  **Apply External Updates (if any):** If updates are available, call the `run_upgrade_from_git` tool to safely apply them. This tool handles backup, upgrade, testing, and restore automatically. Report the result of this process and your cycle is complete.
+
+        3.  **Work on Internal Tasks (if no external updates):** If no external updates are found, you must work on an internal development task. Here is the current backlog of tasks:
+            ---
+            {chr(10).join(development_backlog)}
+            ---
+            Choose the *first* task from this list that has not been completed.
+
+        4.  **Implement the Internal Task:**
+            a. Use your `list_files` and `read_file` tools to understand the current codebase related to the task.
+            b. Plan the necessary code changes.
+            c. Use the `write_file` tool to implement the changes.
+            d. After writing the code, use the `run_tests` tool to ensure you haven't broken anything.
+            e. Report a summary of the changes you made and the result of the tests. Your cycle is then complete.
+        """
+
+        # Construct and run the society for one cycle
+        society = construct_society(task_prompt)
+        answer, _, _ = run_society(society)
+
+        print("\n" + "-"*50)
+        print("Development Cycle Complete. Final Report from Agent:")
+        print(f"\033[94m{answer}\033[0m")
+        print("-" * 50)
+
+        # Wait for a few seconds before starting the next cycle
+        print("\nWaiting for 10 seconds before next cycle...")
+        time.sleep(10)
+
+
+if __name__ == "__main__":
+    main()