Add pointer events to interactive image

[frankvp11]

The reason that I am submitting this pull request is that because I found a need to have access to the events that arise from what I recently learned are pointers.
Basically, with the on_mouse you can access the position that the user clicked on. However, given my project https://github.com/frankvp11/ModelMaker, I have been trying to access what* they are clicking on when they click. I have yet to test it with my project, but from what I can tell, it doesn't break the functionality of on_mouse, and would certainly be a nice feature for those of us (albeit very few) working with the SVG side of things with the interactive images.
This can also be considered as an example for how the user can simply use SVG's inside the interactive image, since from a discord discussion we determined it wasn't too clear.
If there are any improvements needed, please let me know.
If you don't like the idea, or have perhaps a better way of implementing this (or getting my project to work lol) it'd be much appreciated.
When testing this, go to this route: documentation/interactive_image#svg_content and place your mouse over one of the two black circles. I apologize in advance for the immense amount of spamming that pops up - this is due to the amount of events that arise. I couldn't exactly think of a better way to deal with this while still showing off the feature.

[frankvp11]

To kind of give more background as to why I'd like this (perhaps even as just an example or to keep this as a dead PR if need be) is because I thought it'd be interesting (and thus useful) if we could determine when the mouse enters an svg. Seeing if the mouse coordinates are over an SVG is almost trivial, however determining if the mouse enters / leaves is less so trivial, even less so when you have hundreds of these SVG shapes. This system allows you to do something super simple to access that, as can be determined in this video. Watch as my mouse moves over the shape it can determine if the mouse enters / leaves, if the click is down or up, etc. In case you were wondering, aside from the code provided in this commit, below is all the code necessary to run this example. I'd appreciate it if you would consider it. 😄
Screencast from 2024-03-21 09:02:45 PM.webm

from nicegui import ui 

class Image:
    def __init__(self, image_url):
        self.image_url = image_url
        self.content = """ 
            <circle  transform="scale(1, 1) rotate(0,0,0) translate(0, 0) skewX(1) skewY(1)"  cx="100" cy="50" r="50" fill="black" fill-opacity="1" stroke="black" stroke="black" stroke-width="1" pointer-events="all"/>
                        """ 
        self.image = ui.interactive_image(image_url,  on_pointer = self.emit_events)
        self.image.style("border: 1px solid black;")
        self.image.content = self.content

    def emit_events(self, event):
        if event.type == "mousemove":
            print("Mouse Move")
        elif event.type == "mousedown":
            print("Mouse Down")
        elif event.type == "mouseup":
            print("Mouse Up")
        elif event.type == "mouseleave":
            print("Mouse Leave")
        elif event.type == "mouseenter":
            print("Mouse Enter")

image = Image("result.svg")



ui.run()

[rodja]

Thanks for the pull request @frankvp11. I like the general idea but wonder if the events are not better added in the same style as in #2716...

[falkoschindler]

Hi @frankvp11,

Thanks for the pull request. If I understand correctly, you basically want to receive events from the <svg> element instead of the <img> element.

In its current form, I see two major flaws in the implementation:

• The code is very repetitive.
• It doesn't allow the users to specify which event they are interested in. It simply sends all events to the server, causing quite some traffic, even if SVG events are not needed at all.

I agree with @rodja that the approach from #2716 should work nicely. This way the user should be able to register SVG events like this:

ui.interactive_image(...).on('svg:mouseenter', ...)

[frankvp11]

I'm trying to make it so that this is possible, however I am having trouble determining how we can access the SVG events in this way.
In #2716, you have access to the this.chart.on functionality, as seen here:
this.chart.on(event, (e) => this.$emit(`chart:${event}`, e));
I'm not entirely sure of how I can replicate this functionality. Besides that though, I definitely agree with what you said about it being repetitive and having high traffic. I've added the following commit, which certainly doesn't work, but I've added it so that you can see where I'm at / where I'm stuck at.
Edit: Also, to answer your first question, yes, you were correct, I want to be able to access the SVG events instead of the image events

[falkoschindler]

Great! I added the event listener registration, but somehow the server doesn't receive the event:

ui.interactive_image(content='''
    <circle cx="125" cy="80" r="50" fill="black" pointer-events="all" />
    <circle cx="250" cy="80" r="50" fill="black" pointer-events="all" />
''', size=(800, 600), cross=True).classes('w-64 bg-blue-50').on('svg:pointerenter', lambda e: print(e))

The onPointerEvent method is called though.

[frankvp11]

I figured that out. You were sending it as something similar to this:
"svg:{event:...}", instead of "svg:event_type", and receiving it wrong too - it was supposed to be svg:pointermove and not pointer:mousemove

[frankvp11]

Are there any more changes required ?

[frankvp11]

Actually I've noticed on bug in testing this.
For some reason on occasion (most typically found with pointermove) the clientWidth and clientHeight fields are 0, making the image_x and image_y (obviously) infinity. I'll start looking into fixes for this

Edit: I think there might be a better way to get what I'm after. I was looking around and noticed that if you do this in Javascript, you can actually access the events of the raw SVG element itself, which makes me wonder if I can create a mixin for the SVG's
https://stackoverflow.com/questions/31057699/mouseover-on-svg-circles#:~:text=%3Ccircle%20id%3D%22circle2%22%20cx%3D%22150%22%20cy%3D%2250%22%20r%3D%2220%22%20fill%3D%22green%22%20onmouseover%3D%22evt.target.setAttribute(%27fill%27%2C%20%27blue%27)%3B%22%20onmouseout%3D%22evt.target.setAttribute(%27fill%27%2C%27green%27)%3B%22/%3E

Edit 2:
@falkoschindler I think it'd be a good idea to leave the PR here, (in reference to previous edit). I'll continue looking towards perhaps basing something off of the ui.html SVG as described here: http:https://172.19.0.3:8080/documentation/section_audiovisual_elements#svg

[falkoschindler]

Thanks for looking into it, @frankvp11. Let me know when it's a good time for a final review. 🙂

[frankvp11]

Alright. I'm bringing / have brought this to the discord so if you wanted to follow it, as it'd allow me to expand more on my thinking and ideas for this / current issues

https://discord.com/channels/1089836369431498784/1220463393060950016/1220730161058680862

[frankvp11]

In terms of functionality, this PR should be complete. It allows the user to access the SVG events, which was the intention from the start. I think it would be good to start the final review.

[falkoschindler]

Hi @frankvp11, while working on the demo for the documentation, I noticed that the image coordinates are incorrect. It seems like the coordinates change depending on the browser's zoom level. And there's an obvious typo using event.x for x and y. Can you, please, have another look?

[frankvp11]

I noticed the typo earlier this morning, I haven't had a chance to getting around to fixing that yet. I didn't however notice that the coordinates are/were wrong. I believe using just the offsetX and offsetY should work, but I'll double check and get back to you with a new commit soon 👍

[frankvp11]

If you test the version before the commit where I "removed debug statements", you should see that it appears to be working properly. In that version, it would place a red circle in the image_x and image_y position that you clicked in. Upon testing it regularly (by reading the notify box) it appears to be working on max and min zoom.
Also, thanks for making the actual demo for it. I am not the type of person who is great at designing 🤣

[frankvp11]

Are there any more steps necessary for merging this PR?

[falkoschindler]

I just added the element_id to the event arguments so that you can identify which SVG element has been interacted with.

Apart from that I only wonder how we could keep the crosshair visible while hovering nested SVG elements. I guess it disappears because the image registers a "leave" event which is handled in onCrossEvents. On the other hand this could as well be a feature: It combines image interactions with a crosshair and SVG interactions without. Let's leave it like this and - if necessary - improve on it later.