CCClippingNode with CCLabelTTF - problem with CCRenderTexture

[richardgroves]

Hi,

Summary: Doing a 'direct' render of a CCRenderTexture when the node to be rendered is a CClippingNode with a stencil defined by a CCLabelTTF fails. Other stencil node types work, and adding the render texture to the scene makes it render properly. This usage worked fine on 3.0 but failed when I updated to 3.1.

I posted this on the forums: https://forum.cocos2d-swift.org/t/problem-with-ccclippingnode-with-cclabelttf-as-stencil-into-ccrendertexture/14689 which shows how to reproduce the issue

I've recreated the problem in a fork of the 3.1 code:

https://github.com/richardgroves/cocos2d-iphone

The change made to the CCRendererTest:
richardgroves@2007790

The image here shows the result: The text shown is the render path where the CCRenderTexture is added to the scene, the solid block of colour is where it is renderer in the setup code with [begin] [visit] [end] with a text label stencil.

[ksjogo]

#913 is a problem with RenderTextures and Effects as well.

[ksjogo]

There have been some change like 4b62c1a
Is it fixed now?

[richardgroves]

No - seeing exactly the same problem with v3.2.1

I suspect the issue discussed in #936 could be a major part of the problem, but no idea why just CCLabelTTF is failing compared to other node types used as the stencil.

[richardgroves]

This is still broken in v3.3

[programmarchy]

This is also breaking using a CCNodeColor as a stencil. In my case, both the stencil node and clipping node are inside a CCEffectNode.

[slembcke]

Finally looked into this, and I have no idea how it ever worked before. O_o

CCClippingNode replaces shaders on everything that it uses for a mask, and it's really making the assumption that everything is being shaded like a regular sprite (texture alpha*sprite alpha). This is the way it has always worked, and I think my eyes nearly popped out of my head when I first saw the code for it. (What? You can't do that!) Unfortunately since GLES 2 doesn't support alpha testing there really is no other option.

Depending on if a label is styled or not it uses different texture formats to save memory (RGBA vs just alpha). Alpha-only labels need a special shader to render them, and I'm guessing that is what doesn't work. We may have changed the rules for creating alpha only labels? I dunno. Try setting the fontColor property to something other than white. That's the easiest way to force an RGBA label texture.

I'll dig a bit more on this. Maybe there is some sketchy shader hack I can do to make it work better. Unfortunately the whole thing has always been based on sketchy shader hacks.

[programmarchy]

In my case (not using a label - see #913), I tried swapping in a regular sprite for the CCNodeColor that I was using for a stencil, but no luck. Which makes me think this might be related to the clipping occurring inside of a CCEffectNode instead of alpha testing...

I've been poking around a bit in CCClippingNode.m and CCEffectNode.m and wondering if CCEffectNode is somehow overriding the stencil buffer?

[richardgroves]

Hi @slembcke

Thanks for looking into this. I now remember seeing that shader replacement code in CCClippingNode when I was trying to work out how it worked and being 'concerned' that it was a bit of a brutal hack.

Forcing a full colour texture for the CCLabelTTF doesn't help in my case. The image in the original comment shows the problem: the clip node with the label as the stencil is working when added to the scene normally (the 'TESTING TEXT' in the middle of the screen) but not when used in the RenderTexture begin/visit/end getUIImage -> sprite onto screen path that draws the second block at the bottom left. But that path does work for a normal sprite. So what change can there be in the begin/visit/end path of the CCRenderTexture vs the standard draw path?

[retested on 3.3.5 from latest develop branch and still the same]

This code replacing the 'setupClippingNodeTest' method in CCRendererTest in the tests project demonstrates the issue:

// Test CCClippingNode.
-(void)setupClippingNodeTest
{
self.subTitle = @"ClippingNode test - modified to show render texture problem.";

CGSize size = [CCDirector sharedDirector].designSize;
CCRenderTexture *parent = [CCRenderTexture renderTextureWithWidth:size.width
                                                           height:size.height
                                                      pixelFormat:CCTexturePixelFormat_RGBA8888
                                               depthStencilFormat:GL_DEPTH24_STENCIL8];
parent.positionType = CCPositionTypeNormalized;
parent.position = ccp(0.5, 0.5);
parent.autoDraw = YES;
parent.clearColor = [CCColor blackColor];
parent.clearDepth = 1.0;
parent.clearStencil = 0;
parent.clearFlags = GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT;

CCNodeGradient *grad = [CCNodeGradient nodeWithColor:[CCColor redColor] fadingTo:[CCColor blueColor] alongVector:ccp(1, 1)];

// Swap these two lines around and it all works as expected - just the CCLabelTTF that shows a problem - seems to be fixed on 3.3
// for the direct draw - but the capture the CCRenderTexture image and make a sprite of it is still broken for text, but not the sprite image....
//CCNode *stencil = [CCSprite spriteWithImageNamed:@"Sprites/grossini.png"];
CCLabelTTF *stencil = [CCLabelTTF labelWithString:@"TESTING TEXT" fontName:nil fontSize:20.0f];
//stencil.fontColor = [CCColor colorWithRed:0.59 green:0.99 blue:0.99]; // Force the CCLabelTTF texture to use the full colour mode - doesn't help on 3.3

stencil.position = ccp(size.width/2, size.height/2);
stencil.scale = 1.0;
//[stencil runAction:[CCActionRepeatForever actionWithAction:[CCActionRotateBy actionWithDuration:1.0 angle:90.0]]];

CCClippingNode *clip = [CCClippingNode clippingNodeWithStencil:stencil];
[parent addChild:clip];
clip.alphaThreshold = 0.0;
[clip addChild:grad];

[self.contentNode addChild:parent]; // Normal - just add the CCRenderTexture into the scene and let it draw normally

// Render the CCRenderTexture directly and make an image from it - put that on the screen on top of the normal version
{
    // CHANGE: Render the clipped gradient/stencil into the CCRenderTexture here rather than on normal scene draw and then
    // turn the resulting image into a sprite and add that to the scene
    [parent beginWithClear:0.0 g:0.0 b:0.0 a:0.0 depth:1.0 stencil:0];
    [clip visit];
    [parent end];

    // Capture the CCRenderTexture image and make a sprite from it to add to the scene
    UIImage* parentImage = [parent getUIImage];
    CCTexture* texture = [[CCTexture alloc] initWithCGImage:parentImage.CGImage contentScale:parentImage.scale];

    CCSprite *sprite = [CCSprite spriteWithTexture:texture];
    sprite.scale = 1.0;
    sprite.position = ccp(size.width/4.0, size.height/4.0);
    //[sprite runAction:[CCActionRepeatForever actionWithAction:[CCActionRotateBy actionWithDuration:1.0 angle:90.0]]];

    [self.contentNode addChild:sprite z:2];
}

}

[richardgroves]

So I've finally worked out what is going wrong here!

@slembcke - you were spot on with those shader shenanigans: that's the root cause of it. The reason it was broken for CCLabelTTF was that in the visit method of the label it does a lazy update of the texture if anything has changed: which it will have in my example having been just created... As part of the update texture code it resets the shader - thus wiping out the change the CCClippingNode made.

So to make it work I've had to force it to build the texture before the visit call to the clip node. Asking the label for the content size is enough (there is no explicit method for it) and that has fixed it. Interestingly it works with or without a full colour texture: somehow the stencil render path/shader override must be able to handle the A8 texture in the basic labels (default r/g/b being 1 if not in the texture?).

So to change the code above to work simply insert this above the [parent beginWithClear... line

CGSize cs = stencil.contentSize; // Forces the CCLabelTTF to update the texture and thus set the shader
cs.width = 0; // Do something silly to remove the warning about unused variable

and it works.

I can't immediately see that there is any sensible solution to problems like this but maybe a big chunk of warning in the CCClippingNode documentation for when people hit problems would be helpful.