This project is a tool to setup and utilize the standalone version of Tensorflow Embedding Projector.
Explore the docs »
View Demo
·
Report Bug
·
Request Feature
~ t-Distributed Stochastic Neighbor Embedding (t-SNE) is a dimensionality reduction technique that is particularly well suited for the visualisation of large datasets. The technique is implementable using Barnes-Hut approximations, allowing it to be used on large real-world datasets. We used it on data sets containing up to 30 million examples.
~ It is a build-in tool of Google's Machine Learning framework, TensorFlow . This project is all about an implementation of tool to use the standalone prototype you can find here
Table of Contents
Whilst learning tensorflow I stumbled upon the concept of embedding, which I found quite interesting in it's course taking the motivation through it, here I tried to create an efficient tool that will help you visualize your embeddings more efficientky using the TF embedding tool.Certainly for us, there seems to be an excellent tool that is part of Tensorboard. This is meant to be used as Tensorboard , but in my opinion, tensorboard is too cumbersome, and unless you are already using Tensorflow, it is too much of a fuss to use. As a direct consequence, we can only use the self - contained version. The creators did not appear to have written any documentation for the standalone version, but it is pretty straightforward, and Then I will walk you through the pathways in this editorial. To export data from the model, we wrote a few wrappers. The specifics of the task will be included at the end of the story. Follow through to get started!
This section is the list of any major frameworks/libraries used to bootstrap the subsequent project. Leave any add-ons/plugins for the acknowledgements section. Here are a few assets to work with :
Following are the instructions on setting up your project locally. To get a local copy up and running, follow these simple steps.
This is an example of how to list things you need to use the software and how to install them.
- pip
python -m pip install --upgrade pip
- npm
npm install npm@latest -g
- opencv-python
pip install opencv-pythonInstall TensorFlow with pip TensorFlow 2 packages are available * tensorflow —Latest stable release with CPU and GPU support (Ubuntu and Windows) * tf-nightly —Preview build (unstable). Ubuntu and Windows include GPU support. Older versions of TensorFlow For TensorFlow 1.x, CPU and GPU packages are separate:
- tensorflow==1.15 —Release for CPU-only
- tensorflow-gpu==1.15 —Release with GPU support (Ubuntu and Windows) System requirements
- Python 3.7–3.9
- Python 3.9 support requires TensorFlow 2.5 or later.
- Python 3.8 support requires TensorFlow 2.2 or later.
-
Clone the repo
git clone https://github.com/DrCybernotix/EmbedroW.git
After cloning just open the project folder in any IDE, (Recommend: Pycharm or Visual Studio) Run the test.py first or either you can do this from the terminal.
python launch.py
For terminal :
cd (paste the directory location where you inported the repository) python launch.py -
Install packages
npm install
Now, you might have got the output pop-up in your browser, if not copy paste this Port ID in your browser at : https://localhost:8000, or wahtever the IDE you have run into has output you.(If you have run this project through your defined IDE). Eihter run this command for redirected output through the shell
python -m SimpleHTTPServer
Useful examples of how the model actually works and what does it do when provided costum inputs
For more examples, please refer to the (coming soon!) Documentation
The Projector accepts as input a NxD tensor, where N is the number of samples (or embeddings) and D is the dimension of each sample. The tensor is kept in a file (raw float bytes for tsv). A sample is represented as a point on the plot. We can add metadata to a sample, a picture (called asprite, or labels / class id or names ).
A example sprite image:
We designed a method that would handle everything for you; simply call it with your data. BOOM! Tensor is represented as binary bytes.
write_image_embeddings(root, title, feats, labels, imgs, sprite_size)
'''
:param root: root dir of `Embedding Project` tool
:param title: name of the tensor
:param feats: embedding tensor NxDim
:param labels: labels for each sample NxNumClasses
:param [optional] imgs: images in format NHWC
:param [optional] sprite_size: image sprite size
:param mode: 'w' -- write, 'w+' -- update or append, '+' -- append
'''
Now how do you import your tensors in .tsv format?
write_tsv_embeddings(prefix, feats, labels=None):🔑Note: .tsv files can be loaded in the web browser in real time, on-line.
This tool features an intuitive way of visualizing any vector array in tensor format using a small dependency stack. It is intended to be independent of any library. Furthermore, it employs a static file system, allowing you to broadcast your findings without the need for a private server.
- Add Changelog
- Add back to top links
- Add Additional Templates w/ Examples
- Add "components" document to easily copy & paste sections of the readme
See the open issues for a full list of proposed features (and known issues).
Contributions are what make the open source community such an amazing place to learn, inspire, and create! Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b EmbedroW/main/requirements.txt) - Commit your Changes (
git commit -m 'oss_data') - Push to the Branch (
git push origin EmbedroW/main) - Open a Pull Request
Distributed under the MIT License. See LICENSE.txt for more information.
Twitter/Email - @DrCybernotix
Project Link: EmbedroW
The list of resources I found helpful whilst makingg this project and would like to give credit to :