Neovim Plugin and nvim-lspconfig integration

[Nold360]

Hi,

I really would love to test this with neovim, but i have no idea how to setup a custom LSP. Maybe using nvim-lspconfig

[SilasMarvin]

Hey @Nold360 . We were just having this discussion on Reddit yesterday and will hopefully have some example configurations in the repository and make a pr into nvim-lspconfig soon.

Here is an answer provided by Microbzz on reddit:

local lsp_ai_config = {
  -- Uncomment if using nvim-cmp
  -- capabilities = require('cmp_nvim_lsp').default_capabilities(),
  cmd = { 'lsp-ai' },
  root_dir = vim.loop.cwd(),
  init_options = {
    memory = {
      file_store = {}
    },
    models = {
      model1 = {
        type = "llama_cpp",
        repository = "mmnga/codegemma-1.1-2b-gguf",
        name = "codegemma-1.1-2b-Q8_0.gguf",
        n_ctx = 2048,
        n_gpu_layers = 999
      }
    },
    completion = {
      model = "model1",
      parameters = {
        fim = {
          start = "<|fim_prefix|>",
          middle = "<|fim_suffix|>",
          ["end"] = "<|fim_middle|>"
        },
        max_context = 2000,
        max_new_tokens = 32
      }
    }
  },
}

vim.api.nvim_create_autocmd({"BufEnter", "BufWinEnter"}, {
  callback = function() vim.lsp.start(lsp_ai_config) end,
})

You can swap out the value of init_options with whatever configuration you prefer. See the configuration section of the wiki for more info.

There is still an open discussion around getting ghost text working and potentially shipping our own neovim plugin for automatic inline completion.

[Myzel394]

For those of you who want a very rough pow you can use this snippet. This will use ClosedAI OpenAI's chat completion and can be called using <leader>co. This is pretty much just a work in progress, but maybe it will help someone :)

local lsp_ai_config = {
  -- Uncomment if using nvim-cmp
  -- capabilities = require('cmp_nvim_lsp').default_capabilities(),
  cmd = { 'lsp-ai' },
  root_dir = vim.loop.cwd(),
  init_options = {
    memory = {
      file_store = {}
    },
    models = {
      model1 = {
          type = "open_ai",
          chat_endpoint = "https://api.openai.com/v1/chat/completions",
          model = "gpt-4-1106-preview",
          auth_token_env_var_name = "OPENAI_API_KEY",
      }
    },
    completion = {
      model = "model1",
      parameters = {
          max_context = 2048,
          max_new_tokens = 128,
            messages = {
            {
              role = "system",
              content = "You are a chat completion system like GitHub Copilot. You will be given a context and a code snippet. You should generate a response that is a continuation of the context and code snippet."
            },
            {
              role = "user",
              content = "Context: {CONTEXT} - Code: {CODE}"
            }
          }
      }
    }
  },
}

vim.api.nvim_create_autocmd({"BufEnter", "BufWinEnter"}, {
  callback = function() vim.lsp.start(lsp_ai_config) end,
})

-- Register key shortcut
vim.keymap.set(
    "n", 
    "<leader>co", 
    function()
        print("Loading completion...")

        local x = vim.lsp.util.make_position_params(0)
        local y = vim.lsp.util.make_text_document_params(0)

        local combined = vim.tbl_extend("force", x, y)

        local result = vim.lsp.buf_request_sync(
            0,
            "textDocument/completion",
            combined,
            10000
        )

        print(vim.inspect(result))
    end,
    {
        noremap = true,
    }
)

I'd definitely wish a ghost-like text, just like copilot.vim does it. I'm not too familiar with LSPs, but #5 could be related to this

[SilasMarvin]

For those of you who want a very rough pow you can use this snippet. This will use ClosedAI OpenAI's chat completion and can be called using <leader>co. This is pretty much just a work in progress, but maybe it will help someone :)

local lsp_ai_config = {
  -- Uncomment if using nvim-cmp
  -- capabilities = require('cmp_nvim_lsp').default_capabilities(),
  cmd = { 'lsp-ai' },
  root_dir = vim.loop.cwd(),
  init_options = {
    memory = {
      file_store = {}
    },
    models = {
      model1 = {
          type = "open_ai",
          chat_endpoint = "https://api.openai.com/v1/chat/completions",
          model = "gpt-4-1106-preview",
          auth_token_env_var_name = "OPENAI_API_KEY",
      }
    },
    completion = {
      model = "model1",
      parameters = {
          max_context = 2048,
          max_new_tokens = 128,
            messages = {
            {
              role = "system",
              content = "You are a chat completion system like GitHub Copilot. You will be given a context and a code snippet. You should generate a response that is a continuation of the context and code snippet."
            },
            {
              role = "user",
              content = "Context: {CONTEXT} - Code: {CODE}"
            }
          }
      }
    }
  },
}

vim.api.nvim_create_autocmd({"BufEnter", "BufWinEnter"}, {
  callback = function() vim.lsp.start(lsp_ai_config) end,
})

-- Register key shortcut
vim.keymap.set(
    "n", 
    "<leader>co", 
    function()
        print("Loading completion...")

        local x = vim.lsp.util.make_position_params(0)
        local y = vim.lsp.util.make_text_document_params(0)

        local combined = vim.tbl_extend("force", x, y)

        local result = vim.lsp.buf_request_sync(
            0,
            "textDocument/completion",
            combined,
            10000
        )

        print(vim.inspect(result))
    end,
    {
        noremap = true,
    }
)

I'd definitely wish a ghost-like text, just like copilot.vim does it. I'm not too familiar with LSPs, but #5 could be related to this

Thank you for sharing this! To integrate fully with Neovim and provide good inline completion with ghost text I think we will need to write our own plugin. Write now it will pretty much mimic the functionality of copilot.vim but have more support for different backends for completion. This will change as we add new supported features to LSP-AI that we want Neovim to take advantage of like chatting with your code and semantic search over your code base.

If anyone sees this and is interested in writing a Neovim plugin, feel free to do it! I'm happy to help however I can. Our VS Code plugin is a really good place to start for the kind of functionality it should provide: https://github.com/SilasMarvin/lsp-ai/blob/main/editors/vscode/src/index.ts

[Robzz]

nvim-cmp just merged support for multi-line ghost text, so a cmp based setup should now be quite viable. I'll play with it some more and see if I can get a decent example config going.

Update: yeah that works, the main issue right now is that the window containing the completion is drawn below the cursor which hides the ghost text on the following lines, but there's a PR (#1955) addressing it. I'll play with that branch a bit.
Update 2: yeah it's not perfect, the windows does not always go above the cursor, but it kinda works. The first character of the prediction is also not displayed in ghost text, not sure if a config problem or a cmp bug. Anyway, it looks like this:

[Myzel394]

@Robzz we'd love to see how you did that! :)

[Robzz]

I'll be opening a draft PR in a bit. I would not recommend merging it until the default config is integrated in nvim-lspconfig (managing the LSP lifecycle by hand is annoying and exactly what nvim-lspconfig is here for) but at least it should give a place to point the more adventurous people that want to try it right now.

Edit: PR up, see #17

[SilasMarvin]

I'll be opening a draft PR in a bit. I would not recommend merging it until the default config is integrated in nvim-lspconfig (managing the LSP lifecycle by hand is annoying and exactly what nvim-lspconfig is here for) but at least it should give a place to point the more adventurous people that want to try it right now.

This is awesome!

One thing I'm still unsure of is how to handle default configs. Right now if a user passes empty initializationOptions to LSP-AI we error. We require they provide a memory object and models array.

We could absolutely provide a default on the server for memory, but its tough choosing the default models array as there are many options for model backends, and all are hardware or api key dependent.

For the VS Code plugin, I thought that making OpenAI with gpt-4o the default in the plugin settings would be a reasonable choice, but it honestly wasn't my favorite as it still requires users to set an OPENAI_API_KEY for the plugin to work.

We want to make it as easy as possible for everyone to get started using LSP-AI but I think it requires they make some initial decision on at least what backend they want to use which brings me back to being unsure on how to implement any default config.

[Myzel394]

We want to make it as easy as possible for everyone to get started using LSP-AI but I think it requires they make some initial decision on at least what backend they want to use which brings me back to being unsure on how to implement any default config.

... and ...

For the VS Code plugin, I thought that making OpenAI with gpt-4o the default in the plugin settings would be a reasonable choice, but it honestly wasn't my favorite as it still requires users to set an OPENAI_API_KEY for the plugin to work.

I think this is the best case for a default config. It's much more likely that a user will expose an OPENAI_API_KEY than have set up a local LLM already; especially we wouldn't know what model they're running. So I'm in favor of using openai as the default config.

[SilasMarvin]

We want to make it as easy as possible for everyone to get started using LSP-AI but I think it requires they make some initial decision on at least what backend they want to use which brings me back to being unsure on how to implement any default config.

... and ...

For the VS Code plugin, I thought that making OpenAI with gpt-4o the default in the plugin settings would be a reasonable choice, but it honestly wasn't my favorite as it still requires users to set an OPENAI_API_KEY for the plugin to work.

I think this is the best case for a default config. It's much more likely that a user will expose an OPENAI_API_KEY than have set up a local LLM already; especially we wouldn't know what model they're running. So I'm in favor of using openai as the default config.

I think you are probably right here. I do think they shouldn't be defaults on the server, they should be defaults for the config / plugin to send to the server. I don't want to have defaults for the server that might expose users codebases to third parties.

[Robzz]

I think you are probably right here. I do think they shouldn't be defaults on the server, they should be defaults for the config / plugin to send to the server. I don't want to have defaults for the server that might expose users codebases to third parties.

Agree'd. I think ideally the LSP part and LSP/IDE glue should have sensible defaults, but the backend/model config should probably be left for the user to pick. So as far as I can tell, that leaves only the memory key, which accepts only an empty file store for now ? In this case aside from registering the textDocument/generation action with nvim, there wouldn't be that much to do.

[SilasMarvin]

I think you are probably right here. I do think they shouldn't be defaults on the server, they should be defaults for the config / plugin to send to the server. I don't want to have defaults for the server that might expose users codebases to third parties.

Agree'd. I think ideally the LSP part and LSP/IDE glue should have sensible defaults, but the backend/model config should probably be left for the user to pick. So as far as I can tell, that leaves only the memory key, which accepts only an empty file store for now ? In this case aside from registering the textDocument/generation action with nvim, there wouldn't be that much to do.

Yes we can make the memory key have a default but I agree the model config should be controlled by the user. I just checked out your fork of nvim-lspconfig. Thank you for making that. I think for the init_options if we want to provide OpenAI gpt-4o as the default model we can use the example OpenAI chat config I have in the configuration section of the wiki: https://github.com/SilasMarvin/lsp-ai/wiki/Configuration#chat-3

Or maybe I am misunderstanding what is standard for nvim-lspconfig and its ok to provide a default that doesn't full work without the user providing more parameters?

[Robzz]

Or maybe I am misunderstanding what is standard for nvim-lspconfig and its ok to provide a default that doesn't full work without the user providing more parameters?

This I really don't know, maybe there are similar cases among the supported LSP servers in nvim-lspconfig, I haven't seen any but I only glanced over it and it's a long list. I haven't seen any AI LSP servers in there either since HuggingFace have gone the way of writing their own plugin instead for llm-ls so there's really no one to copy from. Asking on the nvim Matrix server is probably the easiest way to know for sure.