[Feature request]: make code_helper use its standard input

[gautaz]

What do you need?

FWIU, code_helper is good at preparing a JSON input containing a code base to send to an AI.

Currently, it takes everything it finds in a given directory but some files depending on the ignore option (or the lack of it, its default being ".git,node_modules,vendor").
In order for this tool to be closer to the UNIX philosophy, I would have asked it to take the list of files to consider from its standard input.

Doing this would make scenarii like these ones possible:

git ls-files | code_helper | fabric ...
find . -name '*.py' | code_helper | fabric ...
find . | grep -E '.*.pth' | code_helper | fabric ...

It would also avoid the burden of supporting other options like --depth and all other features that users might need in the future.
All these needs would then just be a matter of providing the good input to code_helper by delegating it to other CLI commands.

Aside from this, I would perhaps rename code_helper to somewhat closer to its purpose, perplexity proposed these names:

• fs2json
• file2json
• file-json

I am pretty sure there are even better names reflecting the purpose of code_helper but I must admit I am not good at finding sound names.

NOTE: While reading the description of code_helper, I found myself comparing it to tar but instead of producing a binary archive, it delivers a "JSON archive".

[ksylvan]

Cool suggestions, @gautaz - I like it. Let me noodle on it and get back to you.

In particular, I really like the behavior you suggest, because you could do something like:

find src/some/subdirectory -type f -print  | \
  code_helper 'Make sure all the debug or error outputs are printing to stderr, not stdout.'  | \
  fabric -m $MODEL_GEMINI -p 'create_coding_feature'

Without having to use the complex file-ignore and search depth logic.

[gautaz]

I am glad that the suggestion seems sound :-).
If find is used, what could also be supported is an option like -0 or --null (the same option than the one from the xargs command) in order to pipe from commands like find . -print0.
This is useful when dealing with peculiar file names, but as it should not happen so frequently, there is probably no need to support it from the start.

[ksylvan]

How about a name like code2context or codepack

with options like -0 or --null (for the null separated) names.

In addition, I could add a --format flag with options json and xml, json being the default.

What do you think @gautaz ?

[gautaz]

Yes, both seem great, I would say that code2context seems closer to the task done by the executable as the result is part of the context provided to the AI.
As for --format, if xml is also supported by fabric, yes, that would be great 👍.