Python 3.12 docs website search results prioritizes "what's new" instead of actual topic

[Saba-Sabato]

(I initially posted this in the pythondotorg repo - where I was directed here. Copying the description from there verbatim)

Describe the bug
Searching for library modules on 3.12 returns first a list of "What's new in <version>" pages instead of the searched for module.

To Reproduce
Steps to reproduce the behavior:

1. Go to https://docs.python.org/3.12/index.html
2. Click on search box in top right corner, enter e.g. "asyncio" and hit enter
3. Scroll through results
4. See that the first result is "History and License" followed by 10 (!) "What’s New In Python <version>" results. The 11th result is the actual asyncio module.

Expected behavior
I would expect the asyncio module page to be the first result, or at the very least to appear very close to the top. This has been the case in previous versions of the website search.

Screenshots
(the list goes on a few more entries below the screenshot)

Compare with search from 3.11:

Desktop (please complete the following information):

• OS: MacOS
• Browser Chrome
• Version 123

Additional context
This happens for other topics too, not only modules.

[picnixz]

It might be an issue with Sphinx as well, so feel free to ping me. I remember we changed the search algorithm so this might have affected that, though I honestly don't know why the search would be different depending on the version.

[hugovk]

It sounds like Sphinx to me too: the 3.12 docs were created with Sphinx 7.3.7 and 3.11 with Sphinx 7.2.6 (the version is shown in the footer).

Should this be reported to https://github.com/sphinx-doc/sphinx?

[picnixz]

Definitely yes! I'll tag the relevant maintainers for that component.

[mi1acl]

I spent some time last night on this. Looks like there has been a change in the way Sphinx ranks the results.

I changed the searchtools.js generated by the build process. The first 10 rows get score of 100 which they clearly shouldn't. The title of the first 10 rows is considered to be 'asyncio' which is not correct. the actual asyncio gets a score of 15.

[picnixz]

Yes we changed the ranking because other pages were affected but it appears that we did not consider thoses cases. I think we should sorting the results differently depending on the length of the title and where the match is done (like, is it at the beginning or not).

Another possibility is that we incorrectly guessed the What's New asyncio because of the "title-like" section and handle it as the "principal" asyncio. I was expecting someone to open an issue meanwhile but I'll do it myself (I think it'd be best to continue the discussion on Sphinx' side).

[wlach]

Yes we changed the ranking because other pages were affected but it appears that we did not consider thoses cases. I think we should sorting the results differently depending on the length of the title and where the match is done (like, is it at the beginning or not).

Just to be clear, the original code was clearly not working as intended but it appears that the bugs actually produced more desirable behaviour in some cases. I think there's a number of things we can do here, see the issue above for more discussion.

[wlach]

For what it's worth, updating the docs to use the latest minor version (7.4.x) of Sphinx should improve the results here somewhat. Give it a try!

[AA-Turner]

The current docs have been built with 7.4.6: https://docs.python.org/3.12/

A search for me (on mobile):

Seems improved! (Closing the issue for now -- please report other search ranking issues to Sphinx, as we can't do much in here CPython.)

A

[picnixz]

By the way, remember to clear your browser cache (CTRL+SHIFT+R on FF) (I needed to do that otherwise I had the old results somewhat?)