Context provider "@" autocomplete not in v3

[michaelchia]

Description

The autocomplete feature for context providers (@) in v3 does not yet have feature parity with v2.

Only slash-commands autocompletion was ported. It seems like the autocomplete feature was ported before the introduction of context provider.

Reproduce

Type '@' in chat input

Expected behavior

Not only should it autocomplete with the '@' opener,

1. it needs to be able to trigger autocomplete when '@' is entered at any point in the input.
2. it also optionally needs to trigger an autocomplete request to the backend ContextProvider object as the command argument (@cmd:argument) is being entered. For example "@file:dir/" will send the command "file" and the argument "dir/" to the backend where the logic in FileContextProvider.get_arg_options (more generally `BaseCommandContextProvider.get_arg_options) will determine what arg autocompletion options will be sent back. For @file it allows autocompletion of filepaths, for something like @var it would allow for autocompletion of variable names in the kernel.

The v2 logic can be found in https://github.com/jupyterlab/jupyter-ai/blob/v2.28.4/packages/jupyter-ai/src/components/chat-input.tsx.

[dlqqq]

@michaelchia Thanks for reporting this issue. I had noticed the same thing from testing, but forgot to include this in my list of known issues included with the release notes.

I've let this regression be introduced in v3.0.0a0 because I'm certain that we will fix it. @file has proven to be one of the most useful commands in Jupyter AI v2, and the @<variable-name> feature proposed in #1157 would also require input autocomplete for @ commands.

One of our priorities in the near future (next ~1 month) will be to design & implement a new input suggestions API for Jupyter Chat that can provide suggestions for both / commands and @ commands. This design hasn't even begun, but I will loop you in once we have an initial proposal. Your feedback will be very valuable to us, as you have been a significant contributor to this project. 🤗

[dlqqq]

Note that the backend for @file still works as it did in v2. If you ask What is @file:package.json? from the root of this repo, Jupyter AI will correctly answer that the file defines a Lerna monorepo.