Highlighting array field - Also return non-matching entries

[panmari]

I have an array field with the entries [foo, foobar, bar] and search for foo. The highlighting then returns for that field

[<em>foo</em>, <em>foo</em>bar]

I would like it to return

[<em>foo</em>, <em>foo</em>bar, bar]

I did try to set no_match_size as described on http:https://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-highlighting.html but that didn't work. Is there any way to make elasticsearch behave the way I want?

[nik9000]

I don't believe it has an option to do that right now. I don't think it'd be too hard to build though.

[panmari]

What would be better: To respect the setting of no_match_size for every single array entry or introduce a new setting parameter?

[leeho123]

Hi, did this ever get fixed? I'm relying on this functionality returning non-matched entries for my application.

[prashanttct07]

Hi Team,
Is there any plan to fix this in ES 5.0

[mouafa]

+1 here

[edeak]

+1

[cosmin-marginean]

+1

[hmottestad]

Would also like this. Or some other way to find out what fields in the original document actually got highlighted when having nested documents with arrays.

[hvelucha]

+1

[ashitpupu]

any fixes available for this ??? My work would hugely depend on this.
If any fixes or script available, please let me know.

[vcollignon]

+1

[markisme]

Waiting for any fixes...

[Abduvakilov]

+1

[alex-kuck]

+1

[jimczi]

cc @elastic/es-search-aggs

[shwetaskatdare]

+1

[grantharper]

+1

[musukvl]

+1
I have an ordered list of paragraphs in my documents, so It is very handy to store it like an array and display it directly from the highlight section.
Currently, I need to merge _source lines and highlight lines in code. It looks terrible: I'm removing highlighting tags and match _source with highlight.

I also faced with the need to get non-matching entries because arrays are the simplest way to implement relations. I know sub items order so I don't need to use nested objects. For example, I can just store usernames like an arrays:

[ "1", "2", "3" ] 
[ "Alice", "John Doe", "Bob" ] 

instead of using objects with ID:

[{id:"1", name: "Alice"}, {id:"2", name: "John Doe"}, {id: "3", name:"Bob"} ] 

[jimczi]

We index multi-valued fields as if it was a single value, each entry is appended and separated with a custom separator. This means that the offsets that we index are relative to the concatenated text.
This is needed because we don't know which value matched the query but only the offsets in the original text. In this context no_match_size refers to the situation where none of the value matches the query so we use the first value to populate the response. Though I think we should be able to return all values if the number of fragments is set to 0 (which means highlight the whole text). Currently only values that match the query are returned but if a single value matches then we should return all values. For the reason stated above this is not a low hanging fruit but it should be doable.
I don't have time to work on this at the moment so I'll mark this issue with the adoptme tag and will come back to it if nobody is interested.

[otherBoy]

Any update on the newest version?

[mingyitianxia]

Any update on the newest version?

[robmartin-scibite]

This would be really useful. Would make things a lot easier for a project Im working on. Any update as to whether we are likely to see it implemented any time soon?

[jun915tae]

Waiting for any fixes

[kazykenov]

+1

[nemphys]

Any news on this one? It's quite old and I suppose a lot of people would be happy to see it implemented.

[belousevgen]

+1
Our project also depends on this feature. Any news on it?

I really don't see how I can use Highlight feature, until this is resolved.
It is very handy to display array of values directly from the highlight section. But since it is missing non-matched entities - this is not the case for me.

I thought I can manually merge source and highlight, but if you have simple array of strings - it's not possible to merge.

if source: ["foo", "bar", "foo bar"] (3 records)
and highlights: ["<em>foo</em>", "<em>foo</em>bar"] (2 records)

As a work around I could matched it by index, if highlights returns "null" for non-matched records like
["<em>foo</em>", null, "<em>foo</em>bar"]

Is there another workaround at this point?

[ghost]

+1

[Selroy46]

If you have a nested object, there is a way to find out what element was highlighted if you use inner_hits in nested query.

For example if you have document like this:

{
  "id": 1,
  "title": "foo",
  "comments": [{"id": 1, "text": "foo"}, {"id": 2, "text": "bar"}, {"id": 3, "text": "foobar"}]
}

You can make a query like this:

{
  "nested": {
    "path": "comments",
    "query": {"term": {"text": "foobar"}}
    "inner_hits": {
      "_source": {"includes": "comments.text"},
      "highlight": {"fields": {"comments.text": {"number_of_fragments": 0}}}
    }
  }
}

Now when you run a search you will find "inner_hits" field in the document, somewhere after "_source". There will be something like this, you can find index of highlighted element in _nested.offset field.

"inner_hits": {
        "comments": {
            "hits": {
                "hits": [
                    {
                        "_nested": {"field": "comments", "offset": 2},
                        "_source": {"text": "foobar"},
                        "highlight": {"comments.text": ["<em>foobar</em>"]}
                    }
                ]
            }
        }
}

It is not perfect, but I hope it will help someone.

[bhaveshpatel640]

+1

[nwaughachukwuma]

My workaround

Use a regex to remove the pre/post tags and compare against the data in _source. This takes care of all cases - highlight or not

Example

Source: { 'path.to.arrayField': [foo, foobar, bar]}
Highlight: [foo, foobar]

export function removeHighlightTags(
  text: string,
  preTag = '<em>',
  postTag = '</em>',
) {
  return text
    .replace(new RegExp(preTag, 'g'), '')
    .replace(new RegExp(postTag, 'g'), '')
    .trim()
}

// fetch data
const {hits} = await es_client.search(...)

// let's handle a single hit
const hit = hits.hits[0]
const highlights = hit.highlight.['path.to.arrayField.arrayItem']

const result = hit.arrayField.map((item) => {
    return highlights.find((h) => removeHighlightTags(h) === item.trim()) || item
})

console.log(result) // -->  [<em>foo</em>, <em>foo</em>bar, bar]

[jsphstls]

I've created a similar issue where highlighted array items could be indicated by path as an alternative to the original proposal of including non-highlighted items within highlights.