This repository houses Watson Assistant notebooks with a set of data operation and visualization functions.

To help improving your Watson Assistant after you have deployed it to production, we prepared the following two Jupyter notebooks. These notebooks include practical steps for measuring, analyzing, and actively improving your virtual assistant in a continuous manner. Check out IBM Watson Assistant Continuous Improvement Best Practices for more details.

The Measure notebook contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn’t, and to focus your improvement effort to one of the problem areas identified.

The Effectiveness notebook helps you understand relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort.

You can run the above notebooks locally or in Watson Studio:

• For running locally

  1. Install Jupyter Notebook, see Jupyter/IPython Notebook Quick Start Guide for more details.

  2. Download the Jupyter notebooks available in this repository's notebook directory.

     • Measure Notebook.ipynb
     • Effectiveness Notebook.ipynb

  3. Follow the instructions in each of the notebooks. Be sure to add your Watson Assistant credentials if necessary.

• For running in Watson Studio

  1. Create a Watson Studio account.

     Sign up in Watson Studio, or use an existing account. Lite plan is free to use.

  2. Create a new project and add a Cloud Object Storage (COS) account.

     For more information regarding COS plans, see Pricing.

  3. Copy Measure or Effectiveness notebook from Watson Studio community into your project.

  4. Follow the instructions in each notebook to add project tokens and Watson Assistant credentials if necessary.

This library is licensed under the Apache 2.0 license.