Layout Analysis

[alejandrojcastaneira]

First of all thanks, for the great work!
My question, could these models be adapted to the task of Layout Analysis, so that we could use them in datasets like Publaynet?
In this sense, the models would need to output the probabilities of each pixel belonging to a given class, instead of the possible tags for each token.

[jpWang]

Hi,
since LiLT does not introduce image information yet, it cannot directly output the probabilities of each pixel. However, maybe you can first use the OCR engine to get the OCR result and then classify each token into the categories of pixels contained in its corresponding box. In this way, it can assist the traditional visual model in dealing with layout analysis tasks.

[mllife]

Anyone has any update on this?