Add detected file language to code search

[lafriks]

There are multiple things improved in code search:

• Detect code language for each indexed file and allow filter search by specifying language
• Show total file counts in each language for specified query
• Add commit ID the file is indexed for so View file would open actually indexed file version
• Add last indexed at field to file with time file was indexed at
• Ignore files in vendor directories

Screenshot:

Selected language filter:

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@266d09f). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master   #10256   +/-   ##
=========================================
  Coverage          ?   43.75%           
=========================================
  Files             ?      586           
  Lines             ?    81113           
  Branches          ?        0           
=========================================
  Hits              ?    35491           
  Misses            ?    41234           
  Partials          ?     4388

Impacted Files	Coverage Δ
models/issue.go	53.82% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 266d09f...79ddfd1. Read the comment docs.

[lunny]

I think a better UI is always list all the languages even you check one of the languages. And if you use one and hightlight one. Otherwise once you click the language, you cannot back to the previous UI or click another language.

[lafriks]

@lunny that's really not possible imho as bleve returns stats based on query in single request. Showing all languages would require execute separate search without language filter and would make things at least twice as slow

[guillep2k]

@lunny that's really not possible imho as bleve returns stats based on query in single request. Showing all languages would require execute separate search without language filter and would make things at least twice as slow

Can you use the laguage stats table for a list of languagues?

[lafriks]

@lunny that's really not possible imho as bleve returns stats based on query in single request. Showing all languages would require execute separate search without language filter and would make things at least twice as slow

Can you use the laguage stats table for a list of languagues?

No, code language stats are for repository, search can be global and filters are shown per-file ocurrance count

[guillep2k]

👍

[lunny]

I will not block this PR. But I think we still need that languages stat bar. We may need two requests.

[lafriks]

That can be improved later but it should be measured how much performance hit would it make

[lunny]

Or we could add a button to return to all languages. i.e. All languages >> <selected language>

[guillep2k]

@lafriks Maybe the language list can be actually asked to bleve itself? The indexer should know what languages are indexed at any given time, since it's metadata.

[lafriks]

@guillep2k language list is already returned by bleve using facets: https://github.com/go-gitea/gitea/pull/10256/files#diff-b34c4c9dfe1f4d61c3585819d795f655R352

To get language list for results for all languages would require doing separate request to bleve with other query without language condition just to get them

[lafriks]

@lunny @guillep2k ok, I added additional query to bleve to get language stats for query if filtering by language, see updates screenshot above

[lunny]

Pagination is not correct.

[lafriks]

@lunny I already fixed pagination today, or what do you mean?

[lunny]

When you select one language, I found there is still pagination problem.

[lafriks]

@lunny what problem exactly? Works for me just fine both when language is selected and when not

[lafriks]

[guillep2k]

Perhaps if you're on page 2 of C# and click on say JavaScript (which only got one page) the page number remains as 2 and that causes a problem? (didn't check myself).

[lafriks]

@guillep2k no, I specifically create url without page number when clicking on language to go to page one in such cases

[lunny]

@lafriks when you click that language only one page.

[lafriks]

@lunny fixed, I introduced bug when adding language stats with separate bleve query :)

[guillep2k]

Somehow the "Update Branch" button is not available ATM. I'm unable to update+merge.

[guillep2k]

Ping LG-TM

[mooror]

This is such a useful feature, I can't wait to upgrade to the 12 release! Big thank you to @lafriks and everyone else who contributed to this feature.

Keep up the great work guys,
Benjamin