Cleaner output is nested incorrectly if safelist and input document have differing cases

[Balf]

@jhy We're running into an issue which might be related to #2040. In version 1.16.1 we used to code snippet below to clean SVG's and it worked fine. In 1.16.2 the same code breaks, because the cleaned SVG is invalid. Self closing elements are not parsed properly. How should we do this in 1.16.2?

String svg = "...see below for the source svg";
Safelist safeList = Safelist.none();

String[] lowercaseElementsArray = Arrays.stream(DEFAULT_SVG_ELEMENTS)
            .map(String::toLowerCase)
            .toList()
            .toArray(new String[0]);

String[] lowercaseAttributesArray = Arrays.stream(DEFAULT_SVG_ATTRIBUTES)
            .map(String::toLowerCase)
            .toList()
            .toArray(new String[0]);

safelist.addTags(lowercaseElementsArray);
safelist.addAttributes(":all", lowercaseAttributesArray);

String cleanedSvg = Jsoup.clean(svg, safeList)

SVG Source

<svg width="200" height="200" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink">
    <filter id="feOffset" x="-40" y="-20" width="100" height="200">
        <feOffset in="SourceGraphic" dx="60" dy="60" />
        <feGaussianBlur in="SourceGraphic" stdDeviation="5" result="blur2" />
        <feMerge>
            <feMergeNode
                    in="blur2"></feMergeNode>
            <feMergeNode></feMergeNode>
            <feMergeNode in="SourceGraphic" />
        </feMerge>
    </filter>
    <filter id="noise2" x="0" y="0" width="100%" height="100%">
        <feTurbulence baseFrequency="0.05" />
    </filter>
    <clipPath id="myClip2"          clipPathUnits="objectBoundingBox">
        <circle cx=".5" cy=".5" r=".35" />
    </clipPath>
    <filter id="convolveMatrix2" x="0" y="0" width="100%" height="100%">
        <feConvolveMatrix
                kernelMatrix="-1 0 0 0 0 0 0 0 1" />
    </filter>
    <rect
            x="40"
            y="40"
            width="100"
            height="100"
            style="stroke: #000000; fill: green; filter: url(#feOffset);" />
    <rect
            x="40"
            y="40"
            width="100"
            height="100"
            style="stroke: #000000; fill: green;" />
</svg> ```

Expected result

<svg width="200" height="200" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink">
 <filter id="feOffset" x="-40" y="-20" width="100" height="200">
  <feOffset in="SourceGraphic" dx="60" dy="60"></feOffset>
  <feGaussianBlur in="SourceGraphic" stdDeviation="5" result="blur2"></feGaussianBlur>
  <feMerge>
   <feMergeNode in="blur2"></feMergeNode>
   <feMergeNode></feMergeNode>
   <feMergeNode in="SourceGraphic"></feMergeNode>
  </feMerge>
 </filter> <filter id="noise2" x="0" y="0" width="100%" height="100%">
  <feTurbulence baseFrequency="0.05"></feTurbulence>
 </filter> <clipPath id="myClip2" clipPathUnits="objectBoundingBox">
  <circle cx=".5" cy=".5" r=".35"></circle>
 </clipPath> <filter id="convolveMatrix2" x="0" y="0" width="100%" height="100%">
  <feConvolveMatrix kernelMatrix="-1 0 0 0 0 0 0 0 1"></feConvolveMatrix>
 </filter> <rect x="40" y="40" width="100" height="100" style="stroke: #000000; fill: green; filter: url(#feOffset);"></rect> <rect x="40" y="40" width="100" height="100" style="stroke: #000000; fill: green;"></rect>
</svg>

Actual result

<svg width="200" height="200" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink">
 <filter id="feOffset" x="-40" y="-20" width="100" height="200">
  <feOffset in="SourceGraphic" dx="60" dy="60">
   <feGaussianBlur in="SourceGraphic" result="blur2">
    <feMerge>
     <feMergeNode in="blur2">
      <feMergeNode>
       <feMergeNode in="SourceGraphic">
       </feMergeNode>
       <filter id="noise2" x="0" y="0" width="100%" height="100%">
        <feTurbulence>
        </feTurbulence>
        <clipPath id="myClip2">
         <circle cx=".5" cy=".5" r=".35"></circle>
         <filter id="convolveMatrix2" x="0" y="0" width="100%" height="100%">
          <feConvolveMatrix>
          </feConvolveMatrix>
          <rect x="40" y="40" width="100" height="100" style="stroke: #000000; fill: green; filter: url(#feOffset);"></rect>
          <rect x="40" y="40" width="100" height="100" style="stroke: #000000; fill: green;"></rect>
         </filter>
        </clipPath>
       </filter>
      </feMergeNode>
     </feMergeNode>
    </feMerge>
   </feGaussianBlur>
  </feOffset>
 </filter>
</svg>

[jhy]

Hi @Balf - can you also provide the details for // The default svg tags are added to the safe list? I want to check that I'm looking down the right path here.

[Balf]

(JH: Snipped inline code; moved to this gist)

[jhy]

Thanks for reporting this, fixed!

The issue wasn't about self-closing tags so much as a problem with case normalization. The tail hit (here - line 168) was testing against the nodename not the normalname. The head was already using the normalname. This difference had no impact in earlier versions of jsoup because through this cleaner path, the parsed tags would all be lowercased. But now with SVG foreign elements preserving case, the bug manifested.

Fixed by using the normalname, and also ensuring that the input to the safelist is normalized.

Similarly for attribute names - the input names were not normalized and so by adding lower-cased versions of the attributes, they were removed from the output.

I also noticed an issue that the self-closing style of the input for tags was not preserved in the output - have fixed that too.

The actual output is now:

<svg width="200" height="200" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink">
 <filter id="feOffset" x="-40" y="-20" width="100" height="200">
  <feOffset in="SourceGraphic" dx="60" dy="60" />
  <feGaussianBlur in="SourceGraphic" stdDeviation="5" result="blur2" />
  <feMerge>
   <feMergeNode in="blur2" />
   <feMergeNode />
   <feMergeNode in="SourceGraphic" />
  </feMerge>
 </filter> <filter id="noise2" x="0" y="0" width="100%" height="100%">
  <feTurbulence baseFrequency="0.05" />
 </filter> <clipPath id="myClip2" clipPathUnits="objectBoundingBox">
  <circle cx=".5" cy=".5" r=".35" />
 </clipPath> <filter id="convolveMatrix2" x="0" y="0" width="100%" height="100%">
  <feConvolveMatrix kernelMatrix="-1 0 0 0 0 0 0 0 1" />
 </filter> <rect x="40" y="40" width="100" height="100" style="stroke: #000000; fill: green; filter: url(#feOffset);" /> <rect x="40" y="40" width="100" height="100" style="stroke: #000000; fill: green;" />
</svg>

Which renders correctly.

As a feature thought - I wonder if it would be useful to include an enforceCase option in the cleaner, which would enforce the case of tags and elements as they are set in the safelist? That way one could normalize to a specific case. Possibly also useful for XML cleaning (though we need to update the Cleaner to support full documents, not just the body fragment, for that to be more useful).

[Balf]

@jhy Thank you! It might be nice to have such an option, it allows a bit more flexibility