-
Notifications
You must be signed in to change notification settings - Fork 900
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update to deeplearn 0.5.1 and general refactor #97
Conversation
updates dev and non dev dependencies to match dl 0.5.0 and resolve the potential security vulnerability in the ssri package
add webpack-cli
This commit adds support the the new dl api. It also updates the squeezneet dependency and mobilenet and removes the pre image process bc dl has a new method to handle that now.
Besides updating the KNN Image Classifier this commit also changes the its api. Now you can construct the classifier and pass it a optinal HTML videl elt (or in p5 a video.elt) and then you can call addImageFromVideo() to train it on that specific video and predictFromVideo() to predict from that video. This allows for the user to create a video of any size and ml5 will handle the resize. If the video elt is not given, then addImage() and predictFromImage() can still be called givin image or video as the first argument
Updating the LSTM generator to the new dl 0.5 api. updating how cells are generated and how they are combined in multiRNNCell.
Updated Word2Vec to match dl 0.5
Updating the TransformNet method to dl.05. This commit also changes the name of this method to FastStyle instead of TransformNet and the predict() methods is now called transfer(). The class has been redesign so the transfer methods just handles outputing the desire result. Some static methods have also been addded. The transfer() method now returns a HTMLImage element instead of raw pixels, so no need to do array3DtoImage in the sketch.
Removing YOLO and Neural Network class since they are not ready for use yet. YOLO needs some updates from dl and Neural Network needs to be updated to be more efficient (it was just a toy example). I'm effectively removing them from this series of commit and moving it to a WIP branch
…l5. This updates MobileNet depe
Adding a callback to the LSTM methods in order to know when the model is loaded and ready to use
…ith all the previous commits
src/FastStyle/index.js
Outdated
constructor(model, callback) { | ||
this.ready = false; | ||
this.variableDictionary = {}; | ||
this.timesScalar = dl.tensor(150); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason why we are calling tensor()
here over scalar()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! A couple comments (posted one with this review and one as a single comment by accident.)
@@ -19,7 +17,7 @@ class Word2Vec { | |||
.then(response => response.json()) | |||
.then((json) => { | |||
Object.keys(json.vectors).forEach((word) => { | |||
this.model[word] = Array1D.new(json.vectors[word]); | |||
this.model[word] = tensor(json.vectors[word]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docs specify tensor1d()
, tensor2d()
, etc. are preferred.
In general, we recommend using this method as it makes code more readable
This probably comes from the codebase being in typsecript but maybe makes sense to carry over for readability?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, I'll update all reference to scalars.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated!
…leTransfer Updating the scalar notation to tensor in LSTM and Word2Vec. This also changes FastStyle name to StyleTransefer.
Woohoo! |
Update to deeplearn 0.5.1 and general refactor
Major updates!
Following the proposed Boston Milestone, this PR update ML5 to match the new deeplearn.js API (0.5.1) and introduces new updated to the api too!
I tried to follow @dariusk and left as much comments as possible in each of the individual commits.
Some major changes are introduce to the KNN Image Classifier changing slightly the internal api to make it more accessible: See feat(KNUpdate the KNN Image Classifier to match the dl 0.5
TransformNet has also been updated and renamed FastStyle. See refactor(TransformNet): update the TransformNet to dl 0.5 which I think describes better what this class does. Also, the predict() function has been renamed to transfer()
The Neural Network was removed until we resolve #47 and I also remove the unfinished YOLO into a separate branch.
This PR related to: #57, #89 , #78, #70, #67 and #27.
The ml5-examples repo has another PR that addresses all this changes!
Some comments: