Make imgmath_use_preview work also for svg output #6310

jfbu · 2019-04-16T14:46:19Z

Currently, inline mathematics in svg format is not well aligned with surrounding text.

This PR improves the situation.

Relates: #5301

jfbu

This gives somewhat satisfactory result in my testing of inline math with depth (for example anything using (, ) or letters p, q,...). But it is frustrating to lose the precision of the dvisvgm depth indication.

Consider this a bit of WIP. Suggestions welcome on how to improve this.

jfbu · 2019-04-16T14:47:49Z

sphinx/ext/imgmath.py

@@ -76,7 +76,7 @@ class InvokeError(SphinxError):
 '''

 DOC_BODY_PREVIEW = r'''
-\usepackage[active]{preview}
+\usepackage[active,tightpage]{preview}


This is needed for dvisvgm to gather "depth" information. I hope it does not affect dvipng usage (with its -Ttight option).

In fact adding this option did change the dvipng console output and thus broke imgmath for it because the regex did not find matches anymore. Fixed in second commit.

We probably don't need to use -T tight option of dvipng anymore.

The tightpage option does not play well with dvipng. In my latest commit I am thus using it only for dvisvgm. As per my previous comment, the -T tight option of dvipng remains required.

jfbu · 2019-04-16T14:49:09Z

sphinx/ext/imgmath.py

@@ -85,7 +85,7 @@ class InvokeError(SphinxError):
 '''

 depth_re = re.compile(br'\[\d+ depth=(-?\d+)\]')
-
+depthsvg_re = re.compile(br'.*, depth=(.*)pt')


Here is some example output from dvisvgm when the dvi was produced with tightpage option of preview.

$ dvisvgm --no-fonts testsvg pre-processing DVI file (format version 2) processing page 1 computing extents based on data set by preview package (version 11.91) width=103.354867pt, height=9.225002pt, depth=3.075001pt graphic size: 103.354867pt x 12.300003pt (36.325081mm x 4.322957mm) output written to testsvg.svg 1 of 1 page converted in 0.253939 seconds

Perhaps the regex can be made better?

jfbu · 2019-04-16T14:52:27Z

sphinx/ext/imgmath.py

+ for line in stderr.splitlines(): # not stdout !
+ matched = depthsvg_re.match(line)
+ if matched:
+ depth = round(float(matched.group(1)) * 100 / 72.27) # assume 100ppi


as shown above the depth may have fractional digit. The unit is the TeX point (72.27pt=1in). We could convert it to HTML "pt" unit (72pt=1in) but due to shared code with "png" format, we have to return an integral number of pixels. Thus I round it to an integer here and assume 100ppi.

Clearly any advice welcome on how to better handle this. Probably I have to change more of imgmath to handle a non integer depth. Should we pass it over to html output using pt unit or if we use px how to best convert? The dvipng returns an integral number of px but I don't know how it does the conversion internally from TeX dimensions to px unit.

+1: enough to me.

I think in future more separate handling of dvipng and dvisvgm could be considered. Because here I coerce to an int mainly due to shared code between the two.

Also, dvisvgm can generate its own hashing for naming svg files.

jfbu · 2019-04-16T14:54:32Z

sphinx/ext/imgmath.py

+
+ depth = None
+ if builder.config.imgmath_use_preview:
+ for line in stderr.splitlines(): # not stdout !


The dvisvgm output (as shown above) goes to stderr... whereas the dvipng used stdout...

codecov · 2019-04-16T14:58:58Z

Codecov Report

Merging #6310 into master will decrease coverage by 0.04%.
The diff coverage is 20.68%.

@@            Coverage Diff             @@
##           master    #6310      +/-   ##
==========================================
- Coverage   84.18%   84.14%   -0.05%     
==========================================
  Files         274      274              
  Lines       39289    39313      +24     
  Branches     5831     5838       +7     
==========================================
+ Hits        33074    33078       +4     
- Misses       4900     4920      +20     
  Partials     1315     1315

Impacted Files	Coverage Δ
sphinx/ext/imgmath.py	`48.21% <20.68%> (-3.79%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2757458...48ad83d. Read the comment docs.

jfbu · 2019-04-16T15:07:39Z

I think the usage of preview package to get the needed depth info should not be optional (whether for PNG or SVG). The preview package is old package, we can assume it to be available in Sphinx user LaTeX installation (I will check on Ubuntu xenial). And inline math without this depth correction basically looks very bad (be it with PNG or SVG).

jfbu

I forgot to comment on lines 238-240. For png it seems Sphinx hardcodes the depth inside the png file (for reuse). Probably this must be extended to SVG case. As I did not see immediately how to handle this (and wondering if the whole thing might become obsolete once we let dvipng or dvisvgm handle many equations at once instead of one per one) I put together this PR in this WIP state, sorry.

jfbu · 2019-04-16T15:32:46Z

On xenial I got this:


/usr/share/texmf/tex/latex/preview/preview.sty | preview-latex-style
-- | --
/usr/share/xemacs21/xemacs-packages/etc/auctex/latex/preview.sty | xemacs21-basesupport

I propose we add preview-latex-style as dependency at some later Sphinx release so that imgmath_use_preview defaults to True. Display mathematics does not need it but inline math snippets is basically broken without it for PNG and SVG.

jfbu · 2019-04-16T16:13:33Z

Unfortunately the tightpage option of preview package breaks PNG rendering of display math: equations get shifted to the right. This might be caused by whitespace in LaTeX source which we did not see because of dvipng -Ttight but under the tightpage option of preview package, dvipng acts differently.

jfbu · 2019-04-16T16:20:58Z

I will update this PR. The dvipng and dvisvgm must be separated more one reason being that there is bad interaction of tightpage option of preview package with dvipng in case of display math: the display gets shifted to the right whether or not we use the -Ttight option of dvipng. Thus, we must separate more the code branches for PNG and SVG cases.

edit I have done that now.

jfbu · 2019-04-16T19:24:18Z

I forgot to comment on lines 238-240. For png it seems Sphinx hardcodes the depth inside the png file (for reuse). Probably this must be extended to SVG case.

Done now.

jfbu · 2019-04-23T13:30:55Z

Squashed and rebased on master tip after merge of #6308

tk0miya

Sorry for late. LGTM with nits.

tk0miya · 2019-04-23T15:49:58Z

sphinx/ext/imgmath.py

+ with open(filename, 'r') as f:
+ for line in f:
+ pass
+ return int(line[11:-4])


I feel this is a bit fragile. Could you replace this by regexp?

I think you mean to use a regex for the parsing of very last line (cf write_svg_depth()), and we can pass over silently other lines?

tk0miya · 2019-04-23T15:58:51Z

sphinx/ext/imgmath.py

+ for line in stderr.splitlines(): # not stdout !
+ matched = depthsvg_re.match(line)
+ if matched:
+ depth = round(float(matched.group(1)) * 100 / 72.27) # assume 100ppi


+1: enough to me.

tk0miya · 2019-04-23T16:00:44Z

doc/usage/extensions/math.rst

@@ -105,8 +105,9 @@ built:
 Unfortunately, this only works when the `preview-latex package`_ is
 installed. Therefore, the default for this option is ``False``.

- Currently this option is only used when ``imgmath_image_format`` is
- ``'png'``.
+ .. versionchanged:: 2.1.0


I've written versionchnaged:: 2.1 (not 2.1.0) in another PR. I don't know what is best.

I will use 2.1 too. I still need to adjust a bit to our new branch/release model.

jfbu

I will update, thanks for reviewing.

jfbu · 2019-04-24T08:48:04Z

doc/usage/extensions/math.rst

@@ -105,8 +105,9 @@ built:
 Unfortunately, this only works when the `preview-latex package`_ is
 installed. Therefore, the default for this option is ``False``.

- Currently this option is only used when ``imgmath_image_format`` is
- ``'png'``.
+ .. versionchanged:: 2.1.0


I will use 2.1 too. I still need to adjust a bit to our new branch/release model.

jfbu · 2019-04-24T08:51:06Z

sphinx/ext/imgmath.py

+ with open(filename, 'r') as f:
+ for line in f:
+ pass
+ return int(line[11:-4])


I think you mean to use a regex for the parsing of very last line (cf write_svg_depth()), and we can pass over silently other lines?

jfbu · 2019-04-24T08:52:50Z

sphinx/ext/imgmath.py

+ for line in stderr.splitlines(): # not stdout !
+ matched = depthsvg_re.match(line)
+ if matched:
+ depth = round(float(matched.group(1)) * 100 / 72.27) # assume 100ppi


I think in future more separate handling of dvipng and dvisvgm could be considered. Because here I coerce to an int mainly due to shared code between the two.

Also, dvisvgm can generate its own hashing for naming svg files.

tk0miya

Great work! LGTM :-)

jfbu · 2019-04-25T21:28:00Z

Thanks for reviewing :-) merging...

This reverts commit 80abe49.

jfbu added the type:enhancement enhance or introduce a new feature label Apr 16, 2019

jfbu added this to the 2.1.0 milestone Apr 16, 2019

jfbu commented Apr 16, 2019

View reviewed changes

jfbu force-pushed the imgmath_correct_svg_baselines branch from 878d3eb to 28ee88e Compare April 16, 2019 15:09

jfbu commented Apr 16, 2019

View reviewed changes

This was referenced Apr 16, 2019

SVG output is not as nice as MathJax #5301

Open

For "math into SVG" conversion, run latex+dvisvgm only once for all equations at once #6281

Open

Make imgmath_use_preview work also for svg output

3edcff7

jfbu force-pushed the imgmath_correct_svg_baselines branch from 2d2101b to 3edcff7 Compare April 23, 2019 13:29

tk0miya approved these changes Apr 23, 2019

View reviewed changes

tk0miya reviewed Apr 23, 2019

View reviewed changes

jfbu commented Apr 24, 2019

View reviewed changes

More robust code for getting saved "depth" of svg inline math snippet

ea28bd9

jfbu force-pushed the imgmath_correct_svg_baselines branch from 5ac9715 to ea28bd9 Compare April 24, 2019 09:56

Reorganize documentation of the imgmath configuration variables

48ad83d

tk0miya approved these changes Apr 25, 2019

View reviewed changes

jfbu merged commit dd9598f into sphinx-doc:master Apr 25, 2019

jfbu deleted the imgmath_correct_svg_baselines branch April 25, 2019 21:31

jfbu added a commit that referenced this pull request Apr 25, 2019

Update CHANGES for PR #6310

9af8d96

jfbu mentioned this pull request May 4, 2019

imgmath: Replace utf8x by utf8 in templates for latex snippets #6339

Merged

fcolas mentioned this pull request Jul 24, 2019

#6310 Doesn't seem released in 2.1.* #6602

Closed

jfbu added a commit that referenced this pull request Jul 30, 2019

Fix CHANGES entry for #6310 merge

80abe49

jfbu mentioned this pull request Aug 1, 2019

hf-tikz not rendering correctly #6610

Open

jfbu restored the imgmath_correct_svg_baselines branch August 1, 2019 15:26

jfbu added a commit that referenced this pull request Aug 1, 2019

Add Python3 type annotation to cherry-picked #6310 for 2.2.0 release

2d5d290

jfbu added a commit that referenced this pull request Aug 1, 2019

Revert "Fix CHANGES entry for #6310 merge"

eeca5ee

This reverts commit 80abe49.

jfbu added a commit that referenced this pull request Aug 1, 2019

Update CHANGES for cherry-picked #6310

5a86a71

jfbu added a commit that referenced this pull request Aug 1, 2019

Add Python3 type annotation to cherry-picked #6310 for 2.2.0 release

bfe921b

jfbu added a commit that referenced this pull request Aug 1, 2019

Update CHANGES for cherry-picked #6310

763908c

jfbu added a commit that referenced this pull request Aug 1, 2019

Fix CHANGES after 2.0 branch merge into master (refs: #6310, #6602)

f2165a3

jfbu added a commit that referenced this pull request Aug 1, 2019

Fix error in cherry-picking #6310 to 2.0

365574f

jfbu deleted the imgmath_correct_svg_baselines branch August 1, 2019 16:18

github-actions bot locked as resolved and limited conversation to collaborators Aug 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make imgmath_use_preview work also for svg output #6310

Make imgmath_use_preview work also for svg output #6310

jfbu commented Apr 16, 2019

jfbu left a comment

jfbu Apr 16, 2019

jfbu Apr 16, 2019

jfbu Apr 16, 2019

jfbu Apr 16, 2019

jfbu Apr 16, 2019

jfbu Apr 16, 2019

tk0miya Apr 23, 2019

jfbu Apr 24, 2019

jfbu Apr 16, 2019

codecov bot commented Apr 16, 2019 •

edited

Loading

jfbu commented Apr 16, 2019

jfbu left a comment

jfbu commented Apr 16, 2019

jfbu commented Apr 16, 2019

jfbu commented Apr 16, 2019 •

edited

Loading

jfbu commented Apr 16, 2019

jfbu commented Apr 23, 2019

tk0miya left a comment

tk0miya Apr 23, 2019

jfbu Apr 24, 2019

tk0miya Apr 23, 2019

tk0miya Apr 23, 2019

jfbu Apr 24, 2019

jfbu left a comment

jfbu Apr 24, 2019

jfbu Apr 24, 2019

jfbu Apr 24, 2019

tk0miya left a comment

jfbu commented Apr 25, 2019

Make imgmath_use_preview work also for svg output #6310

Make imgmath_use_preview work also for svg output #6310

Conversation

jfbu commented Apr 16, 2019

jfbu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2019 • edited Loading

Codecov Report

jfbu commented Apr 16, 2019

jfbu left a comment

Choose a reason for hiding this comment

jfbu commented Apr 16, 2019

jfbu commented Apr 16, 2019

jfbu commented Apr 16, 2019 • edited Loading

jfbu commented Apr 16, 2019

jfbu commented Apr 23, 2019

tk0miya left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jfbu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tk0miya left a comment

Choose a reason for hiding this comment

jfbu commented Apr 25, 2019

codecov bot commented Apr 16, 2019 •

edited

Loading

jfbu commented Apr 16, 2019 •

edited

Loading