RecSys - Context-Aware.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<h1> Context-Aware Recommender Systems</h1>\n",
    "![Recommendation Systems Approaches](https://raw.githubusercontent.com/ziababar/recommender/master/images/Recommendation%20Systems.jpg)\n",
    "In content-based recommender systems, we use the context information (such as time, location etc.) while building recommendation engines. These recommender systems are next generation recommendations systems, which fall into the hyper-personalization category. Though content-based recommender systems are efficient, targeted at an individual level, and consider the user's personal preferences alone while building recommendation engines, people wanted recommendation engines to be more personalized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Context-Aware Recommender Steps](https://raw.githubusercontent.com/ziababar/recommender/master/images/context-aware-1.png)\n",
    "A typical content-based recommender system will perform the following steps:\n",
    "1. Retrieve user, item and activity data\n",
    "1. Define Context\n",
    "2. Create a context profile with respect to a user for item content.\n",
    "3. Generate recommendations for a context\n",
    "![Context-Aware Recommender Steps](https://raw.githubusercontent.com/ziababar/recommender/master/images/context-aware-2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Step 1 - Retrieve Data</h2>\n",
    "The first step would always be to gather the data and pull it into the programming environment.\n",
    "For our use case, we download the MovieLens dataset containing three sets of data,\n",
    "\n",
    "Movie data containing a certain movie's information, such as movieID, release date, URL, genre details, and so on\n",
    "User data containing the user information, such as userID, age, gender, occupation, ZIP code, and so on\n",
    "Ratings data containing userID, itemID, rating, timestamp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the libraries that are going to be used here\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import scipy\n",
    "import sklearn\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Column headers for the dataset\n",
    "data_cols = ['user id','movie id','rating','timestamp']\n",
    "item_cols = ['movie id','movie title','release date', 'video release date','IMDb URL','unknown','Action', 'Adventure','Animation','Childrens','Comedy','Crime', 'Documentary','Drama','Fantasy','Film-Noir','Horror', 'Musical','Mystery','Romance ','Sci-Fi','Thriller', 'War' ,'Western']\n",
    "user_cols = ['user id','age','gender','occupation', 'zip code']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of users\n",
    "df_u_user = pd.read_csv('/home/nbuser/library/dataset/u.user', header=None, sep='|', names=user_cols, encoding='latin-1')\n",
    "df_u_user = df_u_user.sort_values('user id', ascending=1)\n",
    "df_u_user.columns\n",
    "df_u_user.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of movie items\n",
    "df_u_item = pd.read_csv('/home/nbuser/library/dataset/u.item', header=None, sep='|', names=item_cols, encoding='latin-1')\n",
    "df_u_item = df_u_item.sort_values('movie id', ascending=1)\n",
    "df_u_item.columns\n",
    "df_u_item.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove the several columns from the dataframe as it's not required for this analysis\n",
    "df_u_item_final = df_u_item.drop(['release date', 'video release date','IMDb URL'], axis=1)\n",
    "df_u_item_final.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# User activity data\n",
    "df_u_data = pd.read_csv('/home/nbuser/library/dataset/u.data', header=None, sep='\\t', names=data_cols, encoding='latin-1')\n",
    "df_u_data = df_u_data.sort_values('user id', ascending=1)\n",
    "df_u_data.columns\n",
    "df_u_data.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_u_data.insert(4, 'hour', '0')\n",
    "df_u_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Step 2 - Define context</h2>\n",
    "The timestamp information available in the dataset is used as the context for calculating the preference value for movie genres for each user for each hour of the day. This context profile information is used for generating context aware recommendations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Loop through the entire dataset and convert the timestamp into the hour for each record\n",
    "for i in range(0, len(df_u_data.index)):\n",
    "    d = datetime.datetime.fromtimestamp(df_u_data.loc[i, ['timestamp']])\n",
    "    df_u_data.loc[i, ['hour']] = d.hour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# timestamp column is not longer required, drop it\n",
    "df_u_data_final = df_u_data.drop(['timestamp'], axis=1)\n",
    "df_u_data_final.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Merge the Movies and Ratings datasets using merge() function\n",
    "ratings = pd.merge(df_u_item, df_u_data_final, on='movie id')\n",
    "ratings.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Step 3 - Create a context profile for user </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since hour-of-day is being used as the context here, our recommendations will be made as per the time of the day. The set of recommendations for an active user will be different for each hour of the day."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pick a user for whom a context profile needs to be created. Sample user IDs are 10, 500, 900, 943\n",
    "UCP_pref = ratings.loc[ratings['user id'] == 943, ['hour', 'unknown','Action', 'Adventure','Animation','Childrens','Comedy','Crime', 'Documentary','Drama','Fantasy','Film-Noir','Horror', 'Musical','Mystery','Romance ','Sci-Fi','Thriller', 'War' ,'Western']]\n",
    "UCP_pref"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The records need to be aggregated so that we just get records by the hour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Use the groupby operator to sum all the ratings across the genres\n",
    "UCP_pref_sc = UCP_pref.groupby(['hour']).sum()\n",
    "UCP_pref_sc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Normalize the above dataset so that is actually scaled from 0 to 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "min_max_scaler = preprocessing.MinMaxScaler()\n",
    "np_scaled = min_max_scaler.fit_transform(UCP_pref_sc)\n",
    "df_normalized = pd.DataFrame(np_scaled)\n",
    "df_normalized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Step 4 - Generating context-aware recommendations</h2>\n",
    "Now that we have created the context profile for the active user, context-aware recommendations for the user can be generated.\n",
    "\n",
    "The following matrixes should be available in order for this to be done,\n",
    " - user context profile (UCP_Pref_SC)\n",
    " - user content recommendations (UCP_Pref_content)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For generating recommendations for a user at the ninth hour, an element wise multiplication of user content recommendations and the context row for the ninth hour of the day from the UCP_pref_SC object needs to be performed."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "python36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}