REPL: generate tab completion hints on a worker thread to not block typing

[IanButterworth]

Prevents slow tab completion hint generation from blocking typing.

Fixes #55434
Specifically the hinting part.

With julia now starting with an interactive thread by default, we can move the repl tab completion hint generation to a worker thread and enable the user to continue to type (on the repl task on the interactive thread) if generating the hint takes a long time. Showing old hints is cancelled if a new key has been pressed since.

Take this intentionally slowly compiling generated function, which needs to be generated to be introspected for completions. (thanks @MasonProtter for the example)

@generated function foo()
    sleep(5)
    (;apple=1, banana=2)
end

On master and 1.11 typing is blocked at foo(). while the generated function compiles for 5s.

On this PR typing can continue and at any point the user can wait 5s for apple to be completed properly.

Screen.Recording.2025-01-28.at.10.30.27.PM.mov

[IanButterworth]

One thing I don't understand is that a sleep acts differently to a busywait

With a busywait, the behavior is as expected. After typing foo() and continuing to type .app slowly, the completion appears after 5s from when the . was pressed.

julia> using StressTest

julia> @generated function foo()
           busywait(5)
           (;apple=1, banana=2)
       end;

julia> foo().apple

With a sleep every keypress in .app resets the 5s timer.

julia> @generated function foo()
           sleep(5)
           (;apple=1, banana=2)
       end;

julia> foo().apple

Looking at internals, this line returns earlier than 5s if a key is pressed, which it doesn't do for busywait

julia/stdlib/REPL/src/REPLCompletions.jl

Line 761 in 5b8b07b

Base.invoke_in_world(COMPLETION_WORLD[], CC.typeinf, interp, frame)

Is there already some cancellation on keypress behavior in REPLInterpreter? Or is it some IO bug with stdin and the timer?

[IanButterworth]

This is an important improvement to get into 1.12, and I think this is in good shape now to be tried out in the wild, so I will merge. Post-merge review would of course be appreciated.