Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use AI to check if each instruction can archive it's objective. #10

Open
ngaut opened this issue Jun 25, 2023 · 0 comments
Open

Use AI to check if each instruction can archive it's objective. #10

ngaut opened this issue Jun 25, 2023 · 0 comments

Comments

@ngaut
Copy link
Owner

ngaut commented Jun 25, 2023

Prompt: Review each instruction, see if it can archive it's objective, review line by line, word by word, finally give me a yes or no answer.

An example:
Review each instruction, see if it can archive it's objective, review line by line, word by word, finally give me a yes or no answer.
{
"goal": "Rewrite the codebase in the project files to Go syntax",
"objective": "To rewrite the codebase in the project files to use Go syntax",
"start_seq": 10,
"hints_from_user": {
"previous_tasks": {
"seq6": "Install the required Go dependencies for the project"
},
"outcome_list": [
"The required Go dependencies have been installed for the project. To retrieve the list of installed dependencies, use the key 'go_dependencies.seq8.list'. The verification result of the installation can be retrieved using the key 'verification_result.seq9.bool'."
]
},
"thoughts": "Based on the user's hints, we can start by retrieving the list of installed Go dependencies and the verification result. Then, we can proceed with rewriting the project files to Go syntax.",
"instructions": [
{
"seq": 10,
"type": "RunPython",
"objective": "Retrieve the list of installed Go dependencies",
"args": {
"code": "installed_dependencies = []\nfor dep in @eval(jvm.list_values_with_key_prefix('go_dependencies.seq8')):\n installed_dependencies.append(dep)\n@eval(jvm.set('installed_dependencies.seq10.list', installed_dependencies))",
"code_review": "This code retrieves the list of installed Go dependencies from the database and saves it under the key 'installed_dependencies.seq10.list'.",
"timeout": 10,
"pkg_dependencies": []
}
},
{
"seq": 11,
"type": "If",
"objective": "Check the verification result of the Go dependency installation",
"args": {
"condition": "@eval(jvm.get('verification_result.seq9.bool'))"
},
"then": [
{
"seq": 12,
"type": "RunPython",
"objective": "Rewrite the project files to Go syntax",
"args": {
"code": "# Code to rewrite project files to Go syntax\n# TODO: Replace this code with the actual Go syntax rewriting code\n@eval(jvm.set('go_code_rewritten.seq12.bool', True))",
"code_review": "This code represents the placeholder for the actual Go syntax rewriting code. It sets the key 'go_code_rewritten.seq12.bool' to True.",
"timeout": 30,
"pkg_dependencies": []
}
}
],
"else": [
{
"seq": 13,
"type": "RunPython",
"objective": "Log the failure of Go dependency installation",
"args": {
"code": "error_message = 'Go dependency installation failed'\n@eval(jvm.set('error_message.seq13.str', error_message))",
"code_review": "This code logs the failure of Go dependency installation by setting the key 'error_message.seq13.str' to 'Go dependency installation failed'.",
"timeout": 5,
"pkg_dependencies": []
}
}
]
}
],
"criticism": "",
"end_seq": 13,
"overall_outcome": "The project files have been rewritten to Go syntax. To check if the rewriting was successful, use the key 'go_code_rewritten.seq12.bool'. If the Go dependency installation failed, retrieve the error message using the key 'error_message.seq13.str'."
}
ChatGPT

2 / 2

Looking at the instructions given, each appears to have a clear objective and uses a language model's task-oriented interface to achieve it.

Seq 10: This instruction retrieves the list of installed Go dependencies, which aligns with its objective. The Python code in 'args' executes correctly, assuming 'jvm' functions properly and returns the expected results.

Seq 11: This instruction checks the verification result of the Go dependency installation. This makes sense and appears to be written correctly. If the condition is met (i.e., the Go dependencies have been successfully installed), the program proceeds to the 'then' branch; otherwise, it goes to the 'else' branch.

Seq 12: This instruction is to rewrite the project files to Go syntax. However, the provided Python code is a placeholder and needs to be replaced with the actual code that rewrites the project files to Go syntax. Without the real code, this instruction can't achieve its objective. This is a significant issue.

Seq 13: This instruction logs the failure of Go dependency installation. The Python code correctly sets an error message if the condition from 'Seq 11' fails.

Finally, looking at the overall outcome, the result of whether the project files have been rewritten to Go syntax or the Go dependency installation has failed is clear and can be easily checked.

So, in its current state, the plan can't fully achieve its objective because of the placeholder code in instruction 'Seq 12'. Until that code is replaced with functional code that can actually rewrite the project files to Go syntax, the plan can't achieve its goal.

In conclusion: No.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant