Termite Version 1!

.gitignore

@@ -0,0 +1,9 @@
+.DS_Store
+*.pyc
+closure-compiler.jar
+/lib
+/client-template/*.min.js
+/client-template/d3.v3.js
+/client-template/data
+/mallet-2.0.7
+/stmt-0.4.0/tmt-0.4.0.jar

LICENSE

@@ -0,0 +1,24 @@
+Copyright (c) 2013, Leland Stanford Junior University
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+ * Neither the name of the <organization> nor the
+ names of its contributors may be used to endorse or promote products
+ derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README

@@ -0,0 +1,158 @@
+README for Termite, a topic model visualization tool.
+
+---------------
+ INFORMATION
+---------------
+Termite is a visualization tool for inspecting the output of statistical
+topic models, and based on the techniques described in the following publication:
+
+Termite: Visualization Techniques for Assessing Textual Topic Models
+Jason Chuang, Christopher D. Manning, Jeffrey Heer
+Computer Science Dept, Stanford University
+http:https://vis.stanford.edu/papers/termite
+
+This tool is developed by:
+ : Jason Chuang <[email protected]>
+ : Ashley Jin <[email protected]>
+
+and is distributed under the BSD-3 license.
+
+-----------
+ LICENSE
+-----------
+Copyright (c) 2013, Leland Stanford Junior University
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+ * Neither the name of the <organization> nor the
+ names of its contributors may be used to endorse or promote products
+ derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+----------------------
+ ADDITIONAL CREDITS
+----------------------
+Termite requires the use of the following libraries and tools.
+We thank their respective authors for developing and distributing the tools.
+
+Mallet: Machine learning for language toolkit
+ Project website: http:https://mallet.cs.umass.edu
+ Developed by Andrew McCallum, et al.
+ Distributed under a CPL license: lib/LICENSE-mallet
+
+STMT: Stanford topic modeling toolbox
+ Project website: http:https://nlp.stanford.edu/software/tmt
+ Developed by Daniel Ramage, et al.
+ Distributed under a GNU license: lib/LICENSE-stmt
+
+D3 javascript visualization library
+ Project website: http:https://d3js.org
+ Developed by Mike Bostock, et al.
+ Distributed under a BSD license: lib/LICENSE-d3
+
+Google closure javascript compiler
+ Project website: https://developers.google.com/closure/compiler/
+ Developed by Google engineers
+ Distributed under an Apache license: lib/LICENSE-closure-compiler
+
+html5slider
+ Project website: https://github.com/fryn/html5slider
+ Developed by Frank Yan
+ Distributed under an MIT license: http:https://opensource.org/licenses/MIT
+
+------------------
+ ONE-TIME SETUP
+------------------
+Run the setup script to fetch the following tools, libraries, and prepare all javascript
+files needed by Termite. This script only needs to be run once when Termite is first
+downloaded onto a new machine.
+ >> ./setup.sh
+
+Libraries fetched include:
+ : mallet
+ : closure-compiler.js
+ : d3.js
+ : html5slider.js
+
+The script creates a minified version of all javascript files.
+
+-----------------------
+ BUILD A TOPIC MODEL
+-----------------------
+Building a topic model in Termite requires running a single python script. The script reads
+in an input text corpus, and produces an output folder whose content can be inspected 
+using a web browser, described in the next section.
+
+Customize configuration file with the following information. A sample configuration
+file can be found in 'example.cfg'
+ : path to text corpus
+ : output directory for holding files generated by Termite
+ : number of topics to train
+ : number of terms to seriate
+ : topic model (either mallet or stmt)
+
+Process the text corpus, and build a topic model by running the execution script.
+ >> ./execute.py file.cfg
+
+The execution script calls in order: 
+ 1. tokenize.py Tokenize the text corpus
+ 2. train_mallet.py Train a topic model using MALLET
+ train_stmt.py or STMT.
+ 3. compute_saliency.py Compute term saliency
+ 4. compute_similarity.py Compute term similarity
+ 5. compute_seriation.py: Seriates terms
+ 6. prepare_data_for_client.py: Generates datafiles for client
+ 7. prepare_vis_for_client.py: Copies necessary scripts for client
+
+---------------------------- 
+ VIEW TOPIC MODEL OUTPUTS
+----------------------------
+You are now ready to visualize the topic model outputs! Termite's output can be viewed in
+a web browser. To view the files locally (on your own computer), you need to set up a local
+web server. Alternatively, you may copy the output file to a web server to publish the results.
+
+Termite outputs are stored in the 'public_html' subfolder within the output directory.
+
+To set up a local webserver:
+ 1. Change into output directory (specified in the configuration file)
+ >> cd <output_folder>/public_html
+ 2. Start a local server using python
+ >> ./web.sh
+ 4. Open http:https://localhost:8888 in a modern web browser (Chrome, Safari, Firefox, or Opera)
+ to view a visualization of the model outputs.
+
+-----------------------------
+ TOPIC MODEL VISUALIZATION
+-----------------------------
+The visualization should consist of a matrix view with a frequency bar view to the right.
+The top right contains user controls, two slider bars and one input text box. User
+controls are explained in more detail in the 'VISUALIZATION USE CASES' section.
+
+When the mouse is placed over terms, topics, circles, or bars in the matrix or frequency 
+bar view, the associated term, topic, bar, and circles will be highlighted in both views.
+
+---------------------------
+ VISUALIZATION USE CASES
+---------------------------
+The Termite visualization tool has a set of user controls in the top right section of the
+webpage. The user may specify
+ : the number of terms with highest affinity to show
+ : the number of terms with highest saliency to show
+ : specific terms to always show
+(Note: the highest affinity term set and highest saliency term set may contain overlapping words)