Add search functionality to https://valkey.io/ website

[bbarani]

Is your feature request related to a problem? Please describe.

The current https://valkey.io/ website lacks search functionality causing discoverability issues. Adding search functionality to the website can greatly enhance the user experience by allowing users to quickly find the content they're looking for.

Describe the solution you'd like

Add a search functionality to https://valkey.io/ website and index the documentation, command reference and website content.

Describe alternatives you've considered

Leave it as is.

Additional context

Add any other context or screenshots about the feature request here.

[kris-ms]

@madolson @stockholmux Hi guys, if you've got nobody looking in to this internally currently, I'd be happy to have a look at getting this done.

I've done a bit of homework and it looks like we may need to come up with a more concrete solution for bringing the markdown and command data over from the valkey-doc and valkey repos.

(Mind that I only stuffed around with the elasticlunr integration built in to Zola itself.)

It looks like Zola (somewhat annoyingly) compiles its search index at buildtime prior to the page generation macros for commands/docs/topics being run.

The search term SORT_RO finds the path for the command page but does not yet have data for the body/title:

Whereas the search term "generally available" works just fine as the page exists prior to buildtime:

Which begs the questions:

1. Is just having a link to the command page if the search term matches a command good enough?
2. If not, do we swap out the current bash scripts for bringing docs/commands over for scripts that actually write the markdown files?
3. If we do write new scripts, do we use another scripting language for ease of maintenance later down the line?
4. Is there some other niftier, cleverer way of forcing the post-build data in to the search index generated by Zola?
5. Do we ignore the Zola search index all together and make our own after the fact? (Probably requires some buildtime shenanigans.)

Of course, if this is being looked in to already, this is all moot.

Hope to hear your thoughts, cheers!

[bbarani]

@kris-ms I did take a stab at using elasticlunr and search on the index generated by Zola but the search was limited to just website content since the documentation content was generated using a separate script. The script need to be tweaked for Zola to auto generate search index for documentation content.

My code is in this fork - https://github.com/bbarani/valkey-io.github.io. Feel free to re-use or suggest alternate solutions.

CC: @stockholmux @madolson @dmitrypol @zuiderkwast

[kris-ms]

Looks really nice @bbarani, seems we have run in to the same problems.

I also think that the scripts will need to be changed if we want to go the elasticlunr route so that the content already exists as markdown files before Zola builds the site.

[stockholmux]

I would definitely like a search on the site, but I do think we're wedded (for a variety of reasons) to having the separation the doc and command markdown which means the out-of-the-box Zola search isn't going to pick them up.

I'd like to retain the minimal set of requirements to build the site: so bringing in other scripting engines (python/ruby/perl/whatever) creates new problems on that front.

@bbarani and I (months back) did briefly chat about having a separate build-time integration of command and doc markdown. So, if you're just messing about with the site localhost - writing a blog, testing documentation, etc. you wouldn't have a localhost search. However, the GitHub action would invoke a separate process that directly brings in the markdown and reveals it to the index creation. Of course, if you were testing the search index you could build the site the other way locally, but you lose the instant reload, low friction, and quick build time of Zola's built in system. It's a little hacky but would get us to the end goal.

Two things come to mind on this strategy:

1. elasticlunr has limitations and downsides. This site isn't tiny now and could eventually grow and reveal those problems, so we need to see how that looks.
2. we'll effectively need to maintain two different builds, so it could introduce some different scenarios of things not quite working or testing revealing problems on one but not the other

I think both are manageable.

The other option which I've mused about is proposing a change to Zola to fix this issue. I'm not sure how that project would feel about it but I do think it verges on a bug that the search index process does miss valid parts of the template.