Fast Style Transfer #38
Conversation
There was a problem hiding this comment.
This is so amazing @yining1023!!! I left a few review comments. Also, similarly to what we did with LSTMs, could we add a README with documentation about training your own model?
| * @param inputImg HTMLImageElement of input img | ||
| * @return Array3D containing pixels of output img | ||
| */ | ||
| outputImgData = net.predict(inputImg); |
There was a problem hiding this comment.
It's tricky b/c I think the library should be able to be "p5 independent" and work with plain JS or other JS frameworks, however, for beginners using p5 it would be nice to be able to pass in a p5.Image or valid p5.Element/p5.MediaElement and receive a p5.Image back.
There was a problem hiding this comment.
I totally agree. I added another function called array3DToP5Image(imageData) which takes an Array3D and returns a p5.Image. I left this array3DToP5Image in the example code. I was going to include it in the p5ml library, but in order to do that, I need to add p5.js as a dependency to p5.ml. And I'm not sure if p5ml should depend on p5.js. I think this is a question that worth discussion. So I left the function in the example code for now.
| for (let i = 0; i < width; i++) { | ||
| for (let j = 0; j < height; j++) { | ||
| k = (i + j * height) * 3; | ||
| let r = Math.round(255 * data[k + 0]); |
There was a problem hiding this comment.
I believe p5 has a round() function so Math is not necessary. Also, would floor() and 256 be more correct here? Tiny minor point! And perhaps irrelevant if we refactor based on my other comment above.
|
|
||
| function setup() { | ||
| createCanvas(252, 252); | ||
| net = new p5ml.TransformNet(modelLoaded, 'udnie', 'models/udnie/'); |
There was a problem hiding this comment.
Is there an option to use this with preload()? Also I think the convention is to have the callback last? Is it possible to load more than one style model into the same p5ml.TransformNet instance? Is that way a name is required? Would it be simpler to just create multiple p5ml.TransformNet instances with different model directories?
There was a problem hiding this comment.
It seems like p5's preload() only waits for its native methods, like loadImage(). I think using a callback might be more generic, and works outside of p5.js.
It's possible to load more than one style model in one instance, but it's simpler to create multiple p5ml.TransformNet instance, and a name is not required in this case. so it will be net1 = new p5ml.TransformNet('models/udnie/', callback) and net2 = new p5ml.TransformNet('models/wave/', callback).
|
Thank you!! I left comments under each of your comments. About the documentation to train your model with other styles, I want to train one myself and then write something. For now, all models are from deeplearn.js 's demo. I changed the method to be |
…ay3d to a HTML image element
|
Looks great to me, thank you! @cvalenzuela any comments before I merge? |
|
I also added an image utility to convert array3d to HTML image element: |
|
Looks good to me!, I will merge and add a documentation and example page in the website |
This PR -
TransformNetclass, andTransformNet.setStyle()andTransformNet.predict()methodsI will create a demo of Fast Style Transfer with a few other styles and host it on my GitHub, because it is a large folder of all the models with different styles.
You can check out a demo here.
Simple example of Fast Style Transfer:
