d3_widget function

[jjallaire]

I had an idea for an extension to this package that I wanted to run by you. The general notion would be to create a generic d3 htmlwidget that could be used for including an arbitrary d3 script as a widget. I was further thinking that we could build some authoring support for this in RStudio so that "Sourcing" the widget script would result in it rendering in the Viewer pane.

Here is generally what I am thinking:

A d3_widget() function that would work something like this:

d3_widget(d3_nest(as.data.frame(Titanic)), "titanic.js")

Under the hood this would convert the "titanic.js" script into a proper htmlwidget. This means that it would package up the data specified in the first argument within the x variable and then call the code in "titanitc.js" within the render method. Implementing resize generically might be a bit tricky but it seems like if educated folks to write their scripts using some conventions and/or helper functions this could be done.

So what would "titanic.js" look like? I think it can have arbitrary JS code and have references to two variables that would be bound externally (i.e. by the wrapper we create):

x (for full access to the htmlwidget options) and data (a convenience accessor for x.data which would point to e.g. the data created by d3_nest().

To provide support for rapid iterative development, we could allow the user to put a special comment at the top of the script that indicates R code to execute to prime the data when it's previewed in RStudio, e.g.

// !preview d3_nest(as.data.frame(Titanic))

The counterpoint to all of this would be that users should just write a full htmlwidget when they want to use d3, but it seems like there is an awful lot of scaffolding required and we don't in that case have the quick preview/iteration we'd have for this.

Does this seem like a potentially good idea or are there some subtleties I'm missing? If this is something we could work on relatively soon we could add the requisite features to RStudio IDE in time for the next release.

cc @rich-iannone @jcheng5

[timelyportfolio]

I very much like the idea, and I think if it works in d3r then it might also be worthwhile with reactR and vueR.

1. So, would titanic.js be a template that gets populated with data and options much like what we did in rCharts and--dang that was a long time ago-- clickme but now with the benefits of htmltools? Or would titanic.js be standalone JavaScript that exports functions that we expect to .call() or .bind() in JavaScript from the htmlwidgets initialize, renderValue(), and resize()?

2. Wouldn't we also want to potentially pass through other arguments to createWidget(), such as elementId, sizingPolicy, width, height, etc.?

3. While d3_nest produces JavaScript in a standard format, we also will need to determine who is responsible for converting R data into proper JavaScript form? Will this be the responsibility of the user or will this be handled in R or JavaScript somehow?

4. Would you like me to write some code to as proof of concept?

5. Shiny?

I really appreciate the input and ideas and of course outside of this all that RStudio is doing. Thanks!!!

[timelyportfolio]

@jjallaire @rich-iannone as an extension this would allow a very efficient method of storing scripts without needing a package and set of dependencies for each, so within d3r or another package we could have a large collection of user-contributed designs that could be rapidly deployed.

[timelyportfolio]

I bet @nstrayer might enjoy this discussion.

[nstrayer]

Oh this would be cool. Especially with the rapid prototyping idea. The biggest pain point for me currently is moving between a basic index.html template for building the vis to the htmlwidget one. I would be willing to pitch in on development in some way if it would help.

[jjallaire]

I was thinking that it would be nice if the JavaScript could be as close as possible to idiomatic d3 examples (e.g. https://github.com/d3/d3/wiki/Gallery) rather than explicitly implementing initialize, render, and resize (users can always build a widget if they want to do that!)

So the body of the script could reference an x and a data variable (where data is just and alias for x.data) and that would be provided to the script at runtime by the d3_widget wrapper. RStudio would also bind the data element by executing the !preview R code during development time. RStudio couldn't provide x of course but the script could be written to have defaults for x.

I'm not trying to create an entirely new way to define htmlwidgets here. Rather, I'm trying to make it easy to write idiomatic d3 code and have it automatically work with our htmlwidgets infrastructure.

As for Shiny, it seems like there are 1 of 2 modalities:

1. User rebuilds the entire visualization whenever data changes; or

2. User implements d3 update semantics and visualization dynamically transitions.

Not exactly sure how to distinguish these cases within the tooling. Maybe the answer is that the user writes a render function after all?

[nstrayer]

I think the way that observable notebooks work is nice. They inject a DOM variable that contains an svg and canvas element and also a width variable. It reevaluates the code everytime the window size changes. If the vis is built in the normal 'd3 way' they handle rurunning the code well.

[jjallaire]

Yes, this is exactly the sort of thing I have in mind!

…

On Thu, Mar 15, 2018 at 6:47 PM Nick Strayer ***@***.***> wrote: I think the way that observable notebooks work ***@***.***/introduction-to-html> is nice. They inject a DOM variable that contains an svg and canvas element and also a width variable. It reevaluates the code everytime the window size changes. If the vis is built in the normal 'd3 way' they handle rurunning the code well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXx72GuIjWx8IA2VcoTZsW6x51uRnOks5tewu2gaJpZM4SsYS6> .

[jjallaire]

To put a finer point on it, I think the JavaScript should have the following variables injected:

• data
• svg
• width
• height

When running under the d3_widget function these would be inject via it's scaffolding. When previewing in RStudio the data variable would be provided by executing the code contained in the preview magic comment.

To be clear, this is NOT intended as an easier way to develop htmlwidgets (there are other changes we could make there to provide the same incremental preview workflow). Rather, the idea is that we want to make it easy to add a custom d3 visualization to an R Markdown document or Shiny application without requiring that the author write a package and implement all of the conventional htmlwidget scaffolding.

Think of it this way: when a user want to add a custom ggplot2 visualization to an R Markdown or Shiny application we don't ask them to write an R package and implement a bunch of component interfaces, rather, they just write the R code and use it inline. I want the same thing to be possible for d3.

@jcheng5 you might have some thoughts about the particular requirements of setting this up for incremental updates within the Shiny applications.

[jcheng5]

@jjallaire I was envisioning the same, except with el (just a generic div) instead of svg. Interesting that Observable gives you both svg and canvas without you asking. Maybe it should default to svg, but you could easily tell it you want canvas or div instead? (More commenting magic?)

@nstrayer I assume the Observable notebook also reevaluates the code every time the data changes?

[jjallaire]

My only reason for the svg is that I recall from working on networkD3 that you sometimes need to manipulate the height and width of the svg element directly in order to get it to size correctly. But of course we could just do this after the fact (if the div has a single embedded svg element then we could just tweak it's size as necessary).

[timelyportfolio]

I'd suggest we provide el (generic div or really any element) as @jcheng5 suggests and then expect the code to specify/create svg or canvas. Nearly all d3 examples create a svg or canvas so I don't think this would be unreasonably burdensome. However, if we do this, then we lose some control over the chart element and what we might automatically do with it.

How should we prototype? Happy to put something out there as a first draft, but also happy to review what someone else creates.

[jjallaire]

The issue is that we want to handle resizing for the user and if we don't have access to the svg element then it might be harder to handle resizing for the user. I think we should start with svg (along with width, height, and data) then add the ability to opt in to canvas and/or el later.

I'll take a shot at a prototype next week and ping back this thread when it's ready.

[nstrayer]

@jcheng5 Yes, they do re-evaluate the code entirely. I've found the more I write d3 the less of a burden this becomes though. I always underestimate the speed of JS and find it can handle blowing away the whole vis and redrawing it fairly easily.

I don't think observable by default generates the SVG and canvas, they just lazily create references to them under the DOM variable.

[timelyportfolio]

ramnathv/htmlwidgets#305 likely relevant to this discussion

[jjallaire]

We've been working on the d3_widget idea discussed above. Here's the work in progress: https://rstudio.github.io/r2d3/

@timelyportfolio The data argument of r2d3() should seamlessly accept the output of the d3r json conversion functions (let me know if you find otherwise).

We've built some smarts about this into RStudio IDE and R Notebook so if you are giving it a test drive I strongly recommend you install the RStudio Daily Build: https://dailies.rstudio.com.

cc @javierluraschi

[timelyportfolio]

Going to close since r2d3 takes care of all this. Thanks!