Part02_Numpy_Pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://static.wixstatic.com/media/64484b_829e39cd0bf4469aa86afb21b28c7e90~mv2.png\" width=\"250\" style=\"float: left;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2: Numpy and Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:1.5px solid gray\"> </hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you have seen in Part 1 of the course, Python already comes along with a variety of data structures. However, in this part we present two very important and standard packages that add to the basic structures in a valuable fashion. Namley, we give a brief introduction to `Numpy` and `Pandas`. \n",
    "\n",
    "The content of the notebook can be summarised as follows:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. [Preface on packages](#pre)<br/>\n",
    "2. [Numpy](#np)<br/>\n",
    "    2.1 [A first glimpse at Numpy](#np1)<br/>\n",
    "    2.2 [Subsetting](#np2)<br/>\n",
    "    2.3 [2D arrays](#np3)<br/>\n",
    "    2.4 [Basic statistics](#np4)<br/>\n",
    "    2.5 [Generating (pseudo) randoms](#np5)<br/>\n",
    "3. [Pandas](#sns)<br/>\n",
    "    3.1 [Creating a DataFrame](#pd1)<br/>\n",
    "    3.2 [Importing data into a DataFrame](#pd2)<br/>\n",
    "    3.3 [Analysing a DataFrame](#pd3)<br/>\n",
    "    3.4 [Sorting data](#pd4)<br/>\n",
    "    3.5 [Subsetting of a DataFrame](#pd5)<br/>\n",
    "    3.6 [Creating new columns](#pd6)<br/>\n",
    "    3.7 [Cumulative statistics](#pd7)<br/>\n",
    "    3.8 [Droping duplicates](#pd8)<br/>\n",
    "    3.9 [Counting](#pd9)<br/>\n",
    "    3.10 [Grouping](#pd10)<br/>\n",
    "    3.11 [Grouped summary statistics](#pd11)<br/>\n",
    "    3.12 [Pivot tables](#pd12)<br/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:1.5px solid gray\"> </hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1. Preface on packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thousands of packages are avaiable that are compatible with Python. The most important ones (there are many more of course) are:\n",
    "* *Numpy*\n",
    "* *Pandas*\n",
    "\n",
    "So how do you install/import packages? One advantage of the IDE PyCharm is, it will tell when you are importing an package that is not installed and directly offer you to install it. Do that! (There are other methods to install packages via the terminal but they are less convenient)\n",
    "\n",
    "`pip install` <-----\n",
    "\n",
    "![how to install][install_pack_image]\n",
    "![installed package][installed_pack_image]\n",
    "\n",
    "After you installed the packages you can import them. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:1.5px solid gray\"> </hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2. Numpy\n",
    "\n",
    "<a id='np'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This package provides the possibility to store data in multidimensional arrays. Those are a good alternative to the standard Python `list` structure. The latter has the drawback of relatively high memory costs and slow performance. Moreover, coming from a more mathematically perspective, the `ndarray` class introduce numeric operations, which can not be performed with lists. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A first glimpse at Numpy\n",
    "<a id='np1'></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[18.34 13.42 19.54 21.37 10.45 14.39 12.31]\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "#Temperatures\n",
    "monday = 18.34\n",
    "tuesday = 13.42\n",
    "wednesday = 19.54\n",
    "thursday = 21.37\n",
    "friday = 10.45\n",
    "saturday = 14.39\n",
    "sunday = 12.31\n",
    "week_temperatures = [monday,tuesday,wednesday,thursday,friday,saturday,sunday]\n",
    "\n",
    "week_temperatures_array = np.array(week_temperatures)\n",
    "print(week_temperatures_array)\n",
    "print(type(week_temperatures_array))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the command `import numpy as np` we imported all functions within the Numpy package. If you only want to use a certain function from a package, it is possible to only import this single method using the command `from x import y` as demonstrated below. However, these typ of imports are not completly optimal, because it is not clear that you used numpy during the code. Therefore general imports (first kind) are preferred."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[18.34 13.42 19.54 21.37 10.45 14.39 12.31]\n"
     ]
    }
   ],
   "source": [
    "from numpy import array\n",
    "\n",
    "#Temperatures\n",
    "monday  = 18.34\n",
    "tuesday = 13.42\n",
    "wednesday = 19.54\n",
    "thursday = 21.37\n",
    "friday = 10.45\n",
    "saturday = 14.39\n",
    "sunday = 12.31\n",
    "week_temperatures = [monday,tuesday,wednesday,thursday,friday,saturday,sunday]\n",
    "\n",
    "week_temperatures_array = array(week_temperatures) #<--- Numpy function is still used here\n",
    "print(week_temperatures_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Numpy` provides several built-in functions to manipulate or analyse an array object. Some examples are presented in the following. For more info you can check out the [Numpy Website](http:https://numpy.org/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------\n",
      "Mean computed manually:\n",
      "15.68857142857143\n",
      "----------------------------------------\n",
      "Mean computed with built-in function:\n",
      "15.68857142857143\n",
      "----------------------------------------\n",
      "Median computed with built-in function:\n",
      "14.39\n"
     ]
    }
   ],
   "source": [
    "#Temperatures\n",
    "monday = 18.34\n",
    "tuesday = 13.42\n",
    "wednesday = 19.54\n",
    "thursday = 21.37\n",
    "friday = 10.45\n",
    "saturday = 14.39\n",
    "sunday = 12.31\n",
    "week_temperatures = [monday,tuesday,wednesday,thursday,friday,saturday,sunday]\n",
    "week_temperatures_array = np.array(week_temperatures)\n",
    "\n",
    "#manual (with lists these kind of calculations can cause problems)\n",
    "mean_manual_week_temperatures = np.sum(week_temperatures_array)/len(week_temperatures_array)\n",
    "print(\"----------------------------------------\")\n",
    "print(\"Mean computed manually:\")\n",
    "print(mean_manual_week_temperatures)\n",
    "\n",
    "#with numpy function\n",
    "mean_np_week_temperatures = np.mean(week_temperatures_array)\n",
    "print(\"----------------------------------------\")\n",
    "print(\"Mean computed with built-in function:\")\n",
    "print(mean_np_week_temperatures)\n",
    "\n",
    "\n",
    "print(\"----------------------------------------\")\n",
    "print(\"Median computed with built-in function:\")\n",
    "print(np.median(week_temperatures_array)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subsetting \n",
    "<a id='np2'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For many applications it is important to understand how to access certain elements of an array - in other words how an element is callable. In `numpy` this is done using the square brackets `[]`. The arguments can be a single index, a range of indices but also logical statements as shown below. One should not at this point that Python uses a zero-based index system - i.e. the first element has index 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------------------------------------------------\n",
      "Second element of the temperature array:\n",
      "13.42\n",
      "-------------------------------------------------\n",
      "First three elements of the temperature array:\n",
      "[18.34 13.42 19.54]\n",
      "-------------------------------------------------\n",
      "Checking which temperatures are above 15:\n",
      "[ True False  True  True False False False]\n",
      "-------------------------------------------------\n",
      "Extracting all temperatures above 15:\n",
      "[18.34 19.54 21.37]\n"
     ]
    }
   ],
   "source": [
    "#Temperatures\n",
    "monday = 18.34\n",
    "tuesday = 13.42\n",
    "wednesday = 19.54\n",
    "thursday = 21.37\n",
    "friday = 10.45\n",
    "saturday = 14.39\n",
    "sunday = 12.31\n",
    "week_temperatures = [monday,tuesday,wednesday,thursday,friday,saturday,sunday]\n",
    "week_temperatures_array = np.array(week_temperatures)\n",
    "\n",
    "print(\"-------------------------------------------------\")\n",
    "print(\"Second element of the temperature array:\")\n",
    "print(week_temperatures_array[1]) #Extracting Elements\n",
    "print(\"-------------------------------------------------\")\n",
    "print(\"First three elements of the temperature array:\")\n",
    "print(week_temperatures_array[:3]) #Slicing\n",
    "print(\"-------------------------------------------------\")\n",
    "print(\"Checking which temperatures are above 15:\")\n",
    "print(week_temperatures_array > 15)\n",
    "print(\"-------------------------------------------------\")\n",
    "print(\"Extracting all temperatures above 15:\")\n",
    "## As subset condition (must be logical), all elements with temperature over 15.0\n",
    "print(week_temperatures_array[week_temperatures_array > 15.0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Numpy (try both ways)\n",
    "# Create a Numpy array on your own with AirQualityIndex Data you build on your own\n",
    "# Extract the last two elements\n",
    "# Extract all elements but the first one\n",
    "# Create a new array with AQI values over 100 and assign it to dangerous (variabel)\n",
    "# Create a new array with AQI values under 100, but not under 50 (use & and put both statements in parenthesis). Assign it to alright.\n",
    "# Create a new array with AQI values under 50. Assign it to nice\n",
    "# Print all variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2D Arrays \n",
    "<a id='np3'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you might know from other programming languagues arrays are most of the time multidimensional. So let's start small. 2 Dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 7)\n",
      "[11.  13.4 14.2 16.7 19.3 14.1 24.3]\n",
      "28.9\n",
      "28.9\n",
      "[[14.2 16.7]\n",
      " [28.9 22.3]]\n",
      "[28.9 22.3 17.3 17.8 19.2]\n",
      "[11.  13.4 14.2 16.7 19.3 14.1 24.3]\n"
     ]
    }
   ],
   "source": [
    "two_week_temparatures= np.array([[11.0, 13.4, 14.2, 16.7, 19.3, 14.1, 24.3],\n",
    "                                [23.4, 19.4, 28.9, 22.3, 17.3, 17.8, 19.2]])\n",
    "\n",
    "## shape function returns \"Dimensions\" of the array\n",
    "print(two_week_temparatures.shape)\n",
    "\n",
    "## Subsetting 2D Arrays\n",
    "print(two_week_temparatures[0])\n",
    "print(two_week_temparatures[1][2])\n",
    "##Alternative\n",
    "print(two_week_temparatures[1,2])\n",
    "\n",
    "# Both lists elements with index 2 and 3\n",
    "print(two_week_temparatures[:,2:4])\n",
    "\n",
    "#Second List Elements with index 2: till the end of the list\n",
    "print(two_week_temparatures[1,2:])\n",
    "\n",
    "# First List all elements \n",
    "print(two_week_temparatures[0,:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy the two_week_temperatures array and try out all kinds of slicing. Get familiar with the concept"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic statistics \n",
    "<a id='np4'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before analyzing a dataset it is important to get know your data. The best way to do that is to print out some summary statistics.\n",
    "We already used some `numpy` statistical functions earlier, here are some important methods:\n",
    "* `median()`, Median\n",
    "* `corrcoef()`, Correlation Coefficient\n",
    "* `std()`, Standard Deviation\n",
    "\n",
    "Do not forget to write the alias `np.` in front of these functions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------\n",
      "Temperature median over two weeks:\n",
      "18.5\n",
      "Temperature mean over two weeks:\n",
      "18.664285714285715\n",
      "--------------------------------------\n",
      "Temperature median of the first week:\n",
      "14.2\n",
      "Temperature mean of the first week:\n",
      "16.142857142857142\n",
      "--------------------------------------\n",
      "Standard deviation over two weeks:\n",
      "4.683497951448734\n",
      "Standard deviation of the first week:\n",
      "4.118549378068914\n",
      "Standard deviation of the second week:\n",
      "3.767313772600377\n"
     ]
    }
   ],
   "source": [
    "two_week_temparatures= np.array([[11.0, 13.4, 14.2, 16.7, 19.3, 14.1, 24.3],\n",
    "                                [23.4, 19.4, 28.9, 22.3, 17.3, 17.8, 19.2]])\n",
    "\n",
    "#over both weaks and all days\n",
    "print(\"--------------------------------------\")\n",
    "print(\"Temperature median over two weeks:\")\n",
    "print(np.median(two_week_temparatures))\n",
    "print(\"Temperature mean over two weeks:\")\n",
    "print(np.mean(two_week_temparatures))\n",
    "\n",
    "#Only from first \"week\" over all day\n",
    "print(\"--------------------------------------\")\n",
    "print(\"Temperature median of the first week:\")\n",
    "print(np.median(two_week_temparatures[0,:]))\n",
    "print(\"Temperature mean of the first week:\")\n",
    "print(np.mean(two_week_temparatures[0,:]))\n",
    "\n",
    "#Standard deviation\n",
    "print(\"--------------------------------------\")\n",
    "print(\"Standard deviation over two weeks:\")\n",
    "print(np.std(two_week_temparatures))\n",
    "print(\"Standard deviation of the first week:\")\n",
    "print(np.std(two_week_temparatures[0,:]))\n",
    "print(\"Standard deviation of the second week:\")\n",
    "print(np.std(two_week_temparatures[1,:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generating (pseudo) randoms\n",
    "<a id='np5'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the Numpy package you can also generate your own data to practice or more likely to model certain distributions. One way to generate your own data is to use `np.random`. It contains methods to draw samples from a huge variety of different distributions, e.g. Gaussian, Gamma, Weinbull, Poisson etc.. Normally the different functions only need the distribution parameters and the sample size as arguments. In case of the Normal distribution, a sample can be generated using `np.random.normal()` with the parameters $\\mu$ and $\\sigma$. In the following cell, we generate a standard Gaussian sample of size $10000$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "gaussian_sample = np.random.normal(0.0, 1.0, size = 10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To verify if the data is generated correctly, we can display the empirical histogram - this will be covered in more detail in the next part of the course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAjJ0lEQVR4nO3de5xdZX3v8c/XyEUFRGWqkgSCEi94KWqMWqqlChiLTWzFGq0WWtpICy89okdRPIhR2wCnKB5BRU29Ygpeo8YT8QheQRMEwQSRGNEkXghEQAQDge/5Yz2DKztrZvZMZs3eM/m+X6/9mr2etZ61f3vN3uu3n+dZF9kmIiKi0/16HUBERPSnJIiIiGiUBBEREY2SICIiolESRERENEqCiIiIRkkQU4yk2yU9apj575f0v3byNQ6XtHGY+ZZ08BDz/l7SV3fm9Sc7SQ+Q9EVJt0q6qNfx1A33vxvDumaV9d1/PNY3ESR9RdKxvY6jX0yaf9xkJukG4OHAPbXij9g+abxfy/ZeI8w/YbxfczRsfxL45EjLSfoIsNH2W1oPauIdQ/V5eJjtbd1WkjQL+Bmw22jq9TNJC4HXAk8Efk/1/j4KvM89OEnL9gsm+jX7WVoQE+evbe9Ve4x7chiJpGkT/Zr9qA9+0R4I/GSq7ORhbNtU0uuAc4CzgEdQJc0TgMOA3cc1wBgb23m0/ABuAI4YYt5xwHeAdwG3AOuBPyvlG4AbgWNry38EeD9wMfA74BvAgbX5Bg6uLfs+YAXVr7MjStk7assvAK4CbgN+Cswr5f8IXFteYz3wqlqdw6l+3Q/1fk31Rb++vKdzAdXe77fLc5X3fWN5/WuofkkuAu4G7gJuB75Yln88cGlZ5xpgfu01HwZ8saxnFfCOwdepxXRiielnpeycso1vA64Anl1b/nTgIuATZRtcAzwGeFOJdwNw1DDboDFW4G3lfd1d3tvxDXXnAqtLXL8Bzi7lvyjv4/byeBbwaODrwM3ATVSts307PnuvB64GbgX+G9izNv9/Ar8Cfgn8E9t/fo4GrixxbABOr9WbVZY9vsT1TWAa8L9LHOvL9jZw/4b3+GCqz+SLR/juDBfD4XR8Dql914bZjnuW/+vN5f+zCnh4mXcp8M/l+Zi3LbAf8KWy/i3At4D79XpfNOp9V68D2BUejJwgtlHtkKdR7dh+QbVT3QM4imoHtVdZ/iNl+jll/jnsuCOsJ4hbqX6R3a98MT5CSRDlC3QrcGSZPx14XJl3dPmCCPgL4A7gqWXeDl/Mjvfk8uXYFzgA2MwfE89x/DFBPJ9qx7xveZ3HA4+sxV5PZLsB64A3U/26fG7ZDo8t85eVxwOBQ6h2Jp3b5WLgocADStkrqBLL/YHXAb+ufcFPB/5QYrw/8DGq7o9TSyz/Qkk0De9/pFhPBz4xzPa7DHhleb4X8MzyfBYdO1zg4PL/2wMYoNpRv7vjs/d9YP/y3q8FTijz5lHtOJ8IPAi4gO0/P4cDTyqfjSeXZV/UEcvHSt0HUP0o+DEws7zWJZ3x1uKaR/W532Fex3LDxXA4wyeIobbjq6h+TDyQ6jv3NGCfMu9S/pggdmbb/gfVD7ndyuPZlB9Jk+mRLqaJ83lJt9Qe/1Kb9zPb/2X7HqpfITOBxba32v4q1S/O+sDhl21/0/ZWqh3WsyTNHOJ1v2D7O7bvtf2HjnnHA0ttX1zmb7L9YwDbX7b9U1e+AXyV6kPerSW2b7H9C6odxaENy9wN7A08jurLc63tXw2xvmdSfcmX2L7L9tepktDLStfZi4G32r7D9lqqfuxO/2F7i+07y3v8hO2bbW+z/Z9UO4LH1pb/lu2VrrqCLqLaSSyxfTdVMpolad/RxDrEe2vaLgdL2s/27bYvH2pB2+vK/2+r7c3A2VQJve49tn9pewvVjvHQUv53wH/Z/pHt31Mlrvq6L7V9TflsXA18qmHdp9v+fdmmf0e1A91QXus/hnmP+wE3udbNJum75btxp6TnjCKGoQy1He+m+mFwsO17bF9h+7bOyju5be8GHknVur/b9rdcMsdkkgQxcV5ke9/a44O1eb+pPR/ceXWW1QefNww+sX07VRN2/yFed8MQ5VAlop82zZD0AkmXS9oi6Rbgr6i+1N36de35HWwfPwBlx/leqtbSjZLOl7TPEOvbH9hg+95a2c+pWj0DVL/y6++16X1vVybp9ZKuLUcT3ULV7VF/j53/g5tKEh+cpul9jRBrN46n6s76saRVkl441IKSHi5pmaRNkm6j6jrp/D8N9b/Yn+23yc871v0MSZdI2izpVqoWQue66/WHXV+Hm4H96mMXtv/M9r5l3v1GEcNQhtqOHwdWAssk/VLSmZJ266y8k9v2LKpW5FclrZd0Spcx95UkiMnpvtaCpL2omre/HGLZ4X61bKDqRtqOpD2Az1D1Jz+8fGlXUHUDjSvb77H9NKpuocdQ9YnDjnH/Epgpqf6ZPQDYRNWFtQ2YUZvX1KK6b52Sng28gepX70PKe7yV8XmPw8U6ItvX234Z8CfAGcCnJT2I5v/lv5fyJ9neh6rbrNv38Cu2304HdMy/AFgOzLT9YKouk85112MaaX11lwFbqcbAhjNcDL+n6iYC7jsIY+C+wIbYjuUX/dtsH0I13vdC4B8aXnvM29b272y/zvajgPnAyZKe103dfpIEMTn9laQ/l7Q78HbgctvDtRSG8mHgHyU9T9L9JE2X9DiqfvM9KDteSS+gGgsZV5KeXn4h7kb1Zf8DMPir+zdA/XyO71H9QnuDpN0kHQ78NbCs/Kr/LHC6pAeW99D0ha/bmyqpbAbuL+k0YKjWy2gNGWs3lSW9QtJAaYHcUorvLbHey/bbZW+qAetbJU3njwm2GxcCx0k6RNIDgbd2zN8b2GL7D5LmAi/vYn2vljRD0kOAIX81276FasD+PEnHSNq7fAYPpRrT6CaGnwB7Sjq6fIbeQvW5BYbejpL+UtKTSkK5jao7qN7aq7/2mLatpBdKOliSqH543DPEa/S1JIiJ80VVJ7ENPj63E+u6gOrLvIVqgO0VY1mJ7e9TDY6/i+pDPHhE1O+AV1N94X9L9aVcvhPxDmUf4IPlNX5O1bVwVpn3YeCQ0if9edt3Ue1kX0B1RMl5wD8MjpkAJ1F1Ef2aqgvhU1S/UIeyEvi/VDuZn1Mlp7Ek2R10EetI5gFrJN1OdRDCQtt32r4DeCfwnbJdnkm1k30q1f/vy1SJsts4vwK8m+pInXXlb92/AYsl/Q44jerzMJwPUm3XHwI/GCkW22cCJ1O15H5THh8A3gh8d6QYbN9a5n+IqnX2e6B+AmfjdqQ6pPbTVMnhWqrP/ccbQhzztgVmA1+jSjCXAefZvmQU9fvC4KGHMUlM8RPIxo2kM4BH2M5ZsRFjlBZETAmSHifpyarMpRqg3JlWWsQur9dnlEaMl72pupX2p+qq+E/gCz2NKGKSSxdTREQ0ShdTREQ0mjJdTPvtt59nzZrV6zAiIiaVK6644ibbA03zpkyCmDVrFqtXr+51GBERk4qkIc94TxdTREQ0SoKIiIhGSRAREdEoCSIiIholQURERKMkiIiIaJQEERERjZIgIiKiURJEREQ0mjJnUkdMFrNO+fJ9z29YcnQPI4kYXhJExASoJ4WIySJdTBER0SgtiIgupWsodjWtJghJ86huFj4N+JDtJUMs92Kqm4g/3fbqUvYmqttG3gO82vbKNmONGG/pVorJrrUEIWkacC5wJLARWCVpue21HcvtDbwG+F6t7BBgIfAEqltIfk3SY2zf01a8EaOR1kTsCtpsQcwF1tleDyBpGbAAWNux3NuBM4D/WStbACyzvRX4maR1ZX2XtRhvRF9KMopeaXOQejqwoTa9sZTdR9JTgZm2O9viI9Yt9RdJWi1p9ebNm8cn6oiIAHp4FJOk+wFnA68b6zpsn297ju05AwONd8yLiIgxarOLaRMwszY9o5QN2ht4InCpJIBHAMslze+ibkREtKzNFsQqYLakgyTtTjXovHxwpu1bbe9ne5btWcDlwPxyFNNyYKGkPSQdBMwGvt9irBER0aG1FoTtbZJOAlZSHea61PYaSYuB1baXD1N3jaQLqQa0twEn5gimmAxyaGtMJa2eB2F7BbCio+y0IZY9vGP6ncA7WwsuIiKGlTOpI/pQWiLRD3ItpoiIaJQEERERjdLFFNEn0q0U/SYtiIiIaJQEERERjZIgIiKiURJEREQ0SoKIiIhGSRAREdEoCSIiIhrlPIiIDqM9HyHnL8RUlRZEREQ0SoKIiIhG6WKK6KF0T0U/SwsiIiIatZogJM2TdJ2kdZJOaZh/gqRrJF0l6duSDinlsyTdWcqvkvT+NuOMiIgdtdbFJGkacC5wJLARWCVpue21tcUusP3+svx84GxgXpn3U9uHthVfREQMr80WxFxgne31tu8ClgEL6gvYvq02+SDALcYTERGj0GaCmA5sqE1vLGXbkXSipJ8CZwKvrs06SNKVkr4h6dlNLyBpkaTVklZv3rx5PGOPiNjl9XyQ2va5th8NvBF4Syn+FXCA7acAJwMXSNqnoe75tufYnjMwMDBxQUdE7ALaPMx1EzCzNj2jlA1lGfA+ANtbga3l+RWlhfEYYHU7oUZMHfVDZ29YcnQPI4nJrs0WxCpgtqSDJO0OLASW1xeQNLs2eTRwfSkfKIPcSHoUMBtY32KsERHRobUWhO1tkk4CVgLTgKW210haDKy2vRw4SdIRwN3Ab4FjS/XnAIsl3Q3cC5xge0tbsUZExI5aPZPa9gpgRUfZabXnrxmi3meAz7QZW0REDC+X2oiYRDK+EBOp50cxRUREf0qCiIiIRkkQERHRKAkiIiIaZZA6YpLKgHW0LQkigty4J6JJupgiIqJREkRERDRKgoiIiEZJEBER0SgJIiIiGiVBREREoxzmGjGF5VyJ2BlpQURERKO0IGKXlBPjIkbWagtC0jxJ10laJ+mUhvknSLpG0lWSvi3pkNq8N5V610l6fptxRkTEjlpLEOWe0ucCLwAOAV5WTwDFBbafZPtQ4Ezg7FL3EKp7WD8BmAecN3iP6oiImBhtdjHNBdbZXg8gaRmwAFg7uIDt22rLPwhweb4AWGZ7K/AzSevK+i5rMd6ISStdZtGGNhPEdGBDbXoj8IzOhSSdCJwM7A48t1b38o660xvqLgIWARxwwAHjEnRERFR6fhST7XNtPxp4I/CWUdY93/Yc23MGBgbaCTAiYhfVZoLYBMysTc8oZUNZBrxojHUjImKctdnFtAqYLekgqp37QuDl9QUkzbZ9fZk8Ghh8vhy4QNLZwP7AbOD7LcYau4D000eMTmsJwvY2SScBK4FpwFLbayQtBlbbXg6cJOkI4G7gt8Cxpe4aSRdSDWhvA060fU9bsUZExI5aPVHO9gpgRUfZabXnrxmm7juBd7YXXUREDKfng9QREdGfcqmNiF1QLuIX3UiCiCktA9MRY5cupoiIaJQWRMQuIq2pGK20ICIiolESRERENEqCiIiIRkkQERHRKAkiIiIaJUFERESjJIiIiGiUBBEREY1yolxMOTkhLGJ8pAURERGNkiAiIqJRqwlC0jxJ10laJ+mUhvknS1or6WpJ/0/SgbV590i6qjyWtxlnRETsqLUxCEnTgHOBI4GNwCpJy22vrS12JTDH9h2S/hU4E3hpmXen7UPbii8iIobXZgtiLrDO9nrbdwHLgAX1BWxfYvuOMnk5MKPFeCIiYhS6ShCSPivpaEmjSSjTgQ216Y2lbCjHA1+pTe8pabWkyyW9aIi4FpVlVm/evHkUoUVExEi63eGfB7wcuF7SEkmPHc8gJL0CmAOcVSs+0Pac8rrvlvToznq2z7c9x/acgYGB8QwpImKX11WCsP01238PPBW4AfiapO9K+kdJuw1RbRMwszY9o5RtR9IRwKnAfNtba6+5qfxdD1wKPKWbWCMiYnx03WUk6WHAccA/Uw0un0OVMC4eosoqYLakgyTtDiwEtjsaSdJTgA9QJYcba+UPkbRHeb4fcBhQH9yOiIiWdXUUk6TPAY8FPg78te1flVn/LWl1Ux3b2ySdBKwEpgFLba+RtBhYbXs5VZfSXsBFkgB+YXs+8HjgA5LupUpiSzqOfoqIiJZ1e5jrB22vqBdI2sP21jJO0KjUWdFRdlrt+RFD1Psu8KQuY4uIiBZ028X0joayy8YzkIiI6C/DtiAkPYLq0NQHlPEClVn7AA9sObaImAD1ixvesOToHkYS/WakLqbnUw1MzwDOrpX/DnhzSzFFdCU7toh2DZsgbH8U+KikF9v+zATFFBERfWCkLqZX2P4EMEvSyZ3zbZ/dUC0iIqaAkbqYHlT+7tV2IBER0V9G6mL6QPn7tokJJyIi+kW3F+s7U9I+knYr923YXK6fFBERU1S3J8odZfsNkv6G6lpMfwt8E/hEW4FFjEbuQx0x/ro9UW4wkRwNXGT71pbiiYiIPtFtC+JLkn4M3An8q6QB4A/thRUREb3W7eW+TwH+jOr2oHcDv6fj7nARETG1jOae1I+jOh+iXudj4xxPRET0iW4v9/1x4NHAVcA9pdgkQURETFndtiDmAIfYdpvBRERE/+j2KKYfAY9oM5CIiOgv3SaI/YC1klZKWj74GKmSpHmSrpO0TtIpDfNPlrRW0tXlBLwDa/OOlXR9eRzb/VuKiIjx0G0X0+mjXbGkacC5wJHARmCVpOUdtw69kurIqDsk/StwJvBSSQ8F3krVtWXgilL3t6ONI6aWnBDXrs7tm8uo79q6Pcz1G1RnUO9Wnq8CfjBCtbnAOtvrbd8FLKPj0Fjbl9i+o0xeTnXfCajuQ3Gx7S0lKVwMzOsm1oiIGB/dXovpX4BPAx8oRdOBz49QbTqwoTa9sZQN5XjgK6OpK2mRpNWSVm/evHmEcCIiYjS6HYM4ETgMuA3A9vXAn4xXEOXCf3OAs0ZTz/b5tufYnjMwMDBe4UREBN0niK2lmwiAcrLcSIe8bgJm1qZnlLLtSDoCOBWYb3vraOpGRER7uk0Q35D0ZuABko4ELgK+OEKdVcBsSQdJ2h1YCGx35JOkp1B1W823fWNt1krgKEkPkfQQ4KhSFhERE6TbBHEKsBm4BngVsAJ4y3AVbG8DTqLasV8LXGh7jaTFkuaXxc6iulvdRZKuGjx01vYW4O1USWYVsLiURUTEBOnqMFfb90r6PPB5212PBtteQZVM6mWn1Z4fMUzdpcDSbl8rItpVPwQ2h7/uGoZtQahyuqSbgOuA68rd5E4brl5EREx+I3UxvZbq6KWn236o7YcCzwAOk/Ta1qOLiIieGSlBvBJ4me2fDRbYXg+8AviHNgOLiIjeGilB7Gb7ps7CMg6xWzshRUREPxgpQdw1xnkRETHJjXQU059Kuq2hXMCeLcQTERF9YtgEYXvaRAUSEf0nV8/dtXV7olxEROxikiAiIqJRtzcMiuiZdHNE9EZaEBER0SgJIiIiGiVBREREoySIiIholAQRERGNkiAiIqJRqwlC0jxJ10laJ+mUhvnPkfQDSdskHdMx755yl7n77jQXERETp7XzICRNA84FjgQ2AqskLbe9trbYL4DjgNc3rOJO24e2FV9ERAyvzRPl5gLryv0jkLQMWADclyBs31Dm3dtiHBERMQZtdjFNBzbUpjeWsm7tKWm1pMslvahpAUmLyjKrN2/u+lbZERHRhX4epD7Q9hzg5cC7JT26cwHb59ueY3vOwMDAxEcYETGFtZkgNgEza9MzSllXbG8qf9cDlwJPGc/gIiJieG2OQawCZks6iCoxLKRqDYxI0kOAO2xvlbQfcBhwZmuRRsSo1C+geMOSo3sYSbSptRaE7W3AScBK4FrgQttrJC2WNB9A0tMlbQReAnxA0ppS/fHAakk/BC4BlnQc/RQRES1r9XLftlcAKzrKTqs9X0XV9dRZ77vAk9qMLfpbLvEd0Xu5H0RE7JR0N01d/XwUU0RE9FASRERENEoXU/RUxhoi+ldaEBER0SgJIiIiGqWLKVoz1NEt6VaKmBySICJi3AyV/HP46+SULqaIiGiUBBEREY2SICIiolESRERENEqCiIiIRkkQERHRKAkiIiIaJUFERESjVhOEpHmSrpO0TtIpDfOfI+kHkrZJOqZj3rGSri+PY9uMMyIidtRagpA0DTgXeAFwCPAySYd0LPYL4Djggo66DwXeCjwDmAu8tdynOiIiJkibLYi5wDrb623fBSwDFtQXsH2D7auBezvqPh+42PYW278FLgbmtRhrRER0aPNaTNOBDbXpjVQtgrHWnd65kKRFwCKAAw44YGxRxoTIBfoiJp9JPUht+3zbc2zPGRgY6HU4ERFTSpsJYhMwszY9o5S1XTciIsZBmwliFTBb0kGSdgcWAsu7rLsSOErSQ8rg9FGlLCIiJkhrCcL2NuAkqh37tcCFttdIWixpPoCkp0vaCLwE+ICkNaXuFuDtVElmFbC4lEVExARp9YZBtlcAKzrKTqs9X0XVfdRUdymwtM34IiJiaJN6kDoiItqTW45GROuGuj959Le0ICIiolFaEBExodKamDzSgoiIiEZpQUREz6Q10d/SgoiIiEZpQcSYDPXLLxfli5g6kiAioi+ku6n/pIspIiIapQUROy3dShFTU1oQERHRKAkiIiIaJUFERESjJIiIiGiUBBEREY1aPYpJ0jzgHGAa8CHbSzrm7wF8DHgacDPwUts3SJpFdRe668qil9s+oc1YY2Q5WikmylCftZwfMbFaSxCSpgHnAkcCG4FVkpbbXltb7Hjgt7YPlrQQOAN4aZn3U9uHthVfREQMr80uprnAOtvrbd8FLAMWdCyzAPhoef5p4HmS1GJMERHRpTYTxHRgQ216YylrXMb2NuBW4GFl3kGSrpT0DUnPbnoBSYskrZa0evPmzeMbfUTELq5fB6l/BRxg+ynAycAFkvbpXMj2+bbn2J4zMDAw4UFGRExlbSaITcDM2vSMUta4jKT7Aw8Gbra91fbNALavAH4KPKbFWCMiokObRzGtAmZLOogqESwEXt6xzHLgWOAy4Bjg67YtaQDYYvseSY8CZgPrW4w1IiaBXPF1YrWWIGxvk3QSsJLqMNelttdIWgystr0c+DDwcUnrgC1USQTgOcBiSXcD9wIn2N7SVqyR+ztExI5aPQ/C9gpgRUfZabXnfwBe0lDvM8Bn2owthpakEBHQv4PUERHRY7kfRERMShmPaF9aEBER0SgtiF1Yxhpiqkhroh1pQURERKO0IHYhaTHEriCtifGTFkRERDRKgoiIiEZJEBER0ShjEBExZWU8YuckQUxB+VJE7Cjfi9FLgpjicuRSRIxVEsQUkUQQEeMtCWISS1KIGJuhvjvpetpeEsQkk6QQERMlCWISSFKImHgZ1G45QUiaB5xDdUe5D9le0jF/D+BjwNOAm4GX2r6hzHsTcDxwD/Bq2yvbjLUNo71L2676IYzoFzv7Y2yqJZXWEoSkacC5wJHARmCVpOW219YWOx74re2DJS0EzgBeKukQqtuPPgHYH/iapMfYvqeteMfLznzA0lKImBymWiIYSpstiLnAOtvrASQtAxYA9QSxADi9PP808F5JKuXLbG8FflbuWT0XuKytYLv5VZ+df8Suqa3vbzfrHSoBTUSSajNBTAc21KY3As8Yahnb2yTdCjyslF/eUXd65wtIWgQsKpO3S7puFPHtB9w00kI6YxRrHF9dxdcjiW1sEtvY9HVsOqM5tvHad3SznmGW6WbbHTjUjEk9SG37fOD8sdSVtNr2nHEOadz0c3yJbWwS29gktrHb2fjavFjfJmBmbXpGKWtcRtL9gQdTDVZ3UzciIlrUZoJYBcyWdJCk3akGnZd3LLMcOLY8Pwb4um2X8oWS9pB0EDAb+H6LsUZERIfWupjKmMJJwEqqw1yX2l4jaTGw2vZy4MPAx8sg9BaqJEJZ7kKqAe1twIktHME0pq6pCdTP8SW2sUlsY5PYxm6n4lP1gz0iImJ7uWFQREQ0SoKIiIhGSRCApNdJsqT9eh3LIElvl3S1pKskfVXS/r2OaZCksyT9uMT3OUn79jqmOkkvkbRG0r2Sen4IoqR5kq6TtE7SKb2Op07SUkk3SvpRr2PpJGmmpEskrS3/z9f0OqZBkvaU9H1JPyyxva3XMXWSNE3SlZK+NNZ17PIJQtJM4CjgF72OpcNZtp9s+1DgS8BpPY6n7mLgibafDPwEeFOP4+n0I+BvgW/2OpDaJWdeABwCvKxcSqZffASY1+sghrANeJ3tQ4BnAif20bbbCjzX9p8ChwLzJD2ztyHt4DXAtTuzgl0+QQDvAt4A9NVove3bapMPoo/is/1V29vK5OVU56n0DdvX2h7NWfVtuu+SM7bvAgYvOdMXbH+T6gjCvmP7V7Z/UJ7/jmpnt8MVFXrBldvL5G7l0TffUUkzgKOBD+3MenbpBCFpAbDJ9g97HUsTSe+UtAH4e/qrBVH3T8BXeh1EH2u65Exf7OQmE0mzgKcA3+txKPcpXThXATcCF9vum9iAd1P98L13Z1YyqS+10Q1JXwMe0TDrVODNVN1LPTFcbLa/YPtU4NRy6fOTgLf2S2xlmVOpugE+OVFxDeomvpgaJO0FfAb4Hx0t654q52YdWsbgPifpibZ7PpYj6YXAjbavkHT4zqxryicI20c0lUt6EnAQ8MPqArLMAH4gaa7tX/cytgafBFYwgQlipNgkHQe8EHiee3AyzSi2Xa/lsjE7QdJuVMnhk7Y/2+t4mti+RdIlVGM5PU8QwGHAfEl/BewJ7CPpE7ZfMdoV7bJdTLavsf0ntmfZnkXV9H/qRCWHkUiaXZtcAPy4V7F0KjeCegMw3/YdvY6nz3VzyZloUC79/2HgWttn9zqeOkkDg0fvSXoA1X1v+uI7avtNtmeU/dpCqksYjTo5wC6cICaBJZJ+JOlqqm6wvjnED3gvsDdwcTkM9/29DqhO0t9I2gg8C/iypJ7djbAM5g9ecuZa4ELba3oVTydJn6K6z8pjJW2UdHyvY6o5DHgl8NzyObuq/CruB48ELinfz1VUYxBjPpy0X+VSGxER0SgtiIiIaJQEERERjZIgIiKiURJEREQ0SoKIiIhGSRAxZUk6tVxpc/CquM9o+fUubfPqsZLmj3Q1WEmzJL28rRhi1zLlz6SOXZOkZ1Gd6f1U21vLpdx373FYO6Xcpnekk+xmAS8HLmg9oJjy0oKIqeqRwE22twLYvsn2LwEknSZpVTkR8fxyxu5gC+BdklZLulbS0yV9VtL1kt5RlplV7oXxybLMpyU9sPPFJR0l6TJJP5B0UbmeUOcyl0o6p7RufiRpbil/qKTPl5bP5ZKeXMqPk/Te8vwjkt4j6buS1ks6pqx2CfDsss7XSnpCuW/BVWV9szvjiBhKEkRMVV8FZkr6iaTzJP1Fbd57bT/d9hOBB1C1NAbdZXsO8H7gC8CJwBOB4yQ9rCzzWOA8248HbgP+rf7CpbXyFuAI208FVgMnDxHnA8s9P/4NWFrK3gZcWe638WbgY0PUfSTw5yX+JaXsFOBbtg+1/S7gBOCc8hpzqC4pE9GVJIiYksq1+p8GLAI2A/9dLjAI8JeSvifpGuC5wBNqVQe7cK4B1pR7EmwF1vPHi+5tsP2d8vwTVDvpumdS3RzoO+Vy0McCBw4R6qdKvN+kuqjavmV9Hy/lXwceJmmfhrqft32v7bXAw4dY/2XAmyW9ETjQ9p1DLBexg4xBxJRVLsd8KXBpSQbHSloGnAfMsb1B0ulUV7wctLX8vbf2fHB68PvSeX2azmlRXZvnZd2EOcL0cOrxqXHl9gWSvkd185gVkl5Vkk7EiNKCiClJ0mM7+tsPBX7OH5PBTWVc4JjOul04oAyCQzUg/O2O+ZcDh0k6uMTyIEmPGWJdLy3L/Dlwq+1bgW9R3SSKcj3/m0ZxH4TfUV1IkVL/UcB62++h6jJ7cpfriUgLIqasvYD/U7pstgHrgEXl2v0fpLpu/6+prsQ5WtdR3R95KbAWeF99pu3NpTvrU5L2KMVvobp/d6c/SLqS6paV/1TKTgeWliuF3kHVRdWtq4F7JP2Q6n7TewCvlHQ31fv991GsK3ZxuZprxCiouvXll8oA986u61Lg9bZX7+y6ItqQLqaIiGiUFkRERDRKCyIiIholQURERKMkiIiIaJQEERERjZIgIiKi0f8H7/UnbMk/b3QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.hist(gaussian_sample, bins = 100, density = True)\n",
    "plt.title('Empirical histogram of standard Gaussians')\n",
    "plt.xlabel('Sample points')\n",
    "plt.ylabel('Density')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clearly the random numbers are generated new everytime you rerun the regarding cell. Hence, it might be good to fix a certain \"random state\", so that the generated numbers stay the same. This can be done by defining a seed using the method `np.random.seed()`. So that in the same cell all random numbers stay the same when rerunning or are reproducable if you keep the used seed in mind."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.535895854646095"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(1000)\n",
    "#Generating a number from the uniform distribution between 0 and 10\n",
    "np.random.uniform(0,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another more general approach is to introduce a random generator using `np.random.RandomState()`, which takes a seed as an argument. Once this generator is defined, one can draw samples directly from this object (so you do not need to work in the same notebook cell as the seed is defined in)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "rs = np.random.RandomState(12345)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.20470766,  0.47894334, -0.51943872, -0.5557303 ,  1.96578057,\n",
       "        1.39340583,  0.09290788,  0.28174615,  0.76902257,  1.24643474])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rs.normal(0.0,1.0,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another application could be to empirically verify the strong law of large numbers (SLLN), which states that for an i.i.d sample $(X_1,...,X_N)$, the average converges almost surely to the true mean. In other words, \n",
    "$$\n",
    "\\bar{S}_N := \\frac{1}{N}\\sum_{i=1}^N X_i \\overset{a.s.}{\\rightarrow}  \\mathbb{E}[X_1]\n",
    "$$\n",
    "for $N\\rightarrow +\\infty$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = np.cumsum(np.random.normal(1.5,10,1000))\n",
    "for i in np.arange(len(s)):\n",
    "    s[i] = s[i]/(i+1)\n",
    "mean = 1.5*np.ones(len(s)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEWCAYAAABv+EDhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6TklEQVR4nO3deXwV1dnA8d+TjZAQthB2QpBFNgEFAasgKirWrbXaatW6tEWtVm1tfUvViq9W27pUfV0Qd4pbq7V1X0BRlB1E2fctrEkgkIXsz/vHzL25yc1yk9ybC5nn+/nkk9nunDN3knnmnDNzjqgqxhhjvCcm2hkwxhgTHRYAjDHGoywAGGOMR1kAMMYYj7IAYIwxHmUBwBhjPMoCQJSJyCoRmRDhNFRE+rnT00TkrgikEZH9hpDuDSKyV0TyRSQ1hO2vFpGvwpT2VBGZGY59NTL91iLyrogcFJF/1bA+qvmLlsC/d1O3uGhnoCUTkY+ARar6p2rLLwSeAXqq6pDmzJOqXt/UfYjI1cAvVPWUcO63EfmIBx4BxqrqtzWszwC2APGqWtbM2WsOFwNdgNQWenwmwqwEEFkvA1eIiFRbfiXwiv3TNlkXIBFYFe2MRElvYH0k/o5EJDbc+zzaiEiLv0G2ABBZ/wFSgXG+BSLSATgPmOHObxWRie70aBFZIiKH3GqNR9zlE0QkM3DHNXxuvojkishuEXlCRBJqypCIvCQi97nTnUTkPfdz+0VkrojEuOv+ICKbRCRPRFaLyA/d5YOAacBJbrVLbvX9uvO/FJGN7n7fEZHuAetURK4XkQ1u2k/WECR927YSkUdFZJf786i7bACwzt0sV0Q+q+HjXwaszxeRkwL2+5CIHBCRLSJyTsDydiLyvPs97hSR+0K9GIrIv0Rkj1sl86WIDHGX93GP0/fdPisi+wI+9w8RubWWfQ4SkTnu51eJyAXu8nuAPwE/cY/t543Nn7vuJRF5WkQ+EJEC4DQROUFEvnH/Bv4lIm9UO8fnichyN2/zRGRYHWnXes6lWlWViGS428e583Pc8zDPPdZ3RSRVRF5x/1cWi1PaC/R9EdksItki8qDvu3f3d62IrHHP/8ci0rtaPm8UkQ3ABnH8XUT2uWmtEJGh9X3XRw1VtZ8I/gDPAs8FzF8HLA+Y3wpMdKfnA1e6021wqjYAJgCZ1fYb+LmRwFicKr0MYA1wa8C2CvRzp18C7nOnH8C5mMe7P+MAcdddAnTHuUn4CVAAdHPXXQ18VS0/gfs9HcgGTgBaAf8HfFktP+8B7YF0IAuYVMv397/AAqAzkAbMA+5112W4+4qr5bNB6928lwK/BGKBG4BdAcf9Nk71XLKb5iLgulr2PxWYGTB/LZDiHvOj1c7zdmCkO70O2AwMClh3fA37jwc2An8EEtzvNQ84tqb0m5i/l4CDwMnuOW8LbANucfNxEVAScI6PB/YBY9zv8Sqcv8lWteSl1nNeQz6rnDdgjvs99AXaAauB9cBEnL/5GcCL1dL6HOjoprUep8oS4EJ3X4Pcz94JzKv22U/dz7YGzgaWuvkW93Pdon1dCdePlQAi72XgYhFJdOd/5i6rSSnQT0Q6qWq+qi4IJQFVXaqqC1S1TFW34lzATg3ho6VAN6C3qpaq6lz1/Reo/ktVd6lqhaq+AWwARoeSH+By4AVVXaaqxcAUnBJDRsA2f1HVXFXdjvPPOqKOff2vqu5T1SzgHpwqtKbYpqrPqmo5zrnoBnQRkS7A93GCZ4Gq7gP+Dlwayk5V9QVVzXOPeSowXETauau/AE4Vka7u/JvufB+ci21QGwZOUG+D812VqOpnOBfRyxpxzPXlD+C/qvq1qlbgnI844HH3b+PfOMHQZzLwjKouVNVyVX0ZKHbzXJtQz3lNXlTVTap6EPgQ2KSqs9Sp/voXTkAK9FdV3e+m9SiV39n1wAOqusb97P3AiMBSgLt+v6oexvkfSQEG4twkrFHV3Q3I9xHNAkCEqepXOHfDPxCRvjgX0Vdr2fznwABgrVusPS+UNERkgDhVOXtE5BDOH3WnED76IM7d0CducfkPAfv8WUDxPhcYGuI+wSk5bPPNqGo+kAP0CNhmT8B0Ic6Frt59udPda9k2VP60VbXQnWyDU6ceD+wOOO5ncEoCdRKRWBH5izjVZodw7oah8jv7AqckNx6namoOTpA+FZjrXnSr6w7sqLZuG1W/x5CEkD+AHdXS3um7IahhfW/gNt/35H5Xvaj73IR6zmuyN2D6cA3z1fcVmNfAv5newGMBed6Pc2ffo6bPukH3CeBJYJ+ITBeRtg3I9xHNAkDzmIFz538F8LGq7q1pI1XdoKqX4Vxw/gq8KSLJONUvSb7txKmTTgv46NPAWqC/qrbFqTKosU69Wnp5qnqbqh4DXAD8VkTOcO+GngVuwnnCpD2wMmCf9XUhuwvnH82X32SctpCd9eWpvn3hFOl3hfjZhnZ1uwPnLraTqrZ3f9pqaE9q/RSnemEiTjVFhrvc9519gVPFNsGd/gqnuuVUd74mu4BegfXXOMffmO+xvvxB1e9rN9DDV0/v6hUwvQP4c8D31F5Vk1T1tUbkrcrfN9C1tg0bIDCvgX8zO3Cq9ALz3VpV5wVsX+XvRlUfV9WRwGCcG7TfhyF/RwQLAM1jBs4/3i+pvfoHEblCRNLcO75cd3EFTh1mooicK86jj3fi1OP6pACHgHwRGYhTr10vtxGvn/tPfhAod9NLxvknyHK3uwanBOCzF+gptTQ0A68B14jICBFphVMiWehWTzXUa8CdIpImIp1wGj5DfbY9C+d4jgllY7do/wnwsIi0FZEYEekrIqFUp6XgBI8cnIvZ/dX2vQHnTvUK4AtVPYTzPf6I2gPAQpw75dtFJF6c90XOB14P5Xgakr8azMf5e7hJROLEeXQ5sArwWeB6ERnjNpQmu3+fKY3I23JgvIiku1VSUxqxj+p+LyIdRKQXTjvGG+7yacAUqWygbycil9S2ExE50T3GeJxAVYTzN9UiWABoBu6Fbx7OhfWdOjadBKwSkXzgMeBSVT3s1nv+CngO5+6vAAh8Kuh3OHd4eTj/mG8Qmv7ALCAf5x/+KVX9XFVXAw+7y/YCxwFfB3zuM5xHL/eISHYNxzsLuAt4C+dOsi8h1qPX4D5gCfAdsAJY5i6rl1u982fga7fIX1f9tM/PcBpcVwMHcOrqu4XwuRk4VQ073c/W1H7zBZCjqjsC5gXnmGrKfwnOBf8cnGrEp4CfqeraEPLTmPxVT/sinGrJXJzA9R5OEEFVl+Dc0DyB8z1txGlgbzBV/RTnb/Y7nAbX9xqzn2r+6+5rOfA+8Lyb1ts4pevX3aqwlTjfb23a4vxPHcD5/nJwqk5bBN+TD8YYUycRWQhMU9UXo50XEx5WAjDG1EhEThWRrm4V0FXAMOCjaOfLhE+Lf9PNGNNoxwL/xKm63Axc3JIegTRWBWSMMZ5lVUDGGONRR1UVUKdOnTQjIyPa2TDGmKPK0qVLs1U1rfryoyoAZGRksGTJkmhnwxhjjioisq2m5VYFZIwxHmUBwBhjPMoCgDHGeNRR1QZgjDm6lZaWkpmZSVFRUbSz0iIlJibSs2dP4uPjQ9reAoAxptlkZmaSkpJCRkYGUvMgcKaRVJWcnBwyMzPp06dPSJ+xKiBjTLMpKioiNTXVLv4RICKkpqY2qHRlAcAY06zs4h85Df1uPREAZq/Zy9NzNkU7G8YYc0TxRBvA5+v28cGKPdwwoW+0s2KMiaKcnBzOOOMMAPbs2UNsbCxpac4LsosWLSIhobYxjlomTwQAQbBO74wxqampLF++HICpU6fSpk0bfve73/nXl5WVERfnicsi4JEqIGOMqc3VV1/N9ddfz5gxY7j99tuZOnUqDz30kH/90KFD2bp1KwAzZ85k9OjRjBgxguuuu47y8vKg/WVkZDBlyhRGjBjBqFGjWLZsGWeffTZ9+/Zl2rRp/u0efPBBTjzxRIYNG8bdd9/tX/6DH/yAkSNHMmTIEKZPn+5f3qZNG+644w6GDx/O2LFj2bu3xqHFG8QTAUCk4aODG2OawYQJwT9PPeWsKyysef1LLznrs7OD1zVSZmYm8+bN45FHHql1mzVr1vDGG2/w9ddfs3z5cmJjY3nllVdq3DY9PZ3ly5czbtw4rr76at58800WLFjgv9B/8sknbNiwgUWLFrF8+XKWLl3Kl19+CcALL7zA0qVLWbJkCY8//jg5OTkAFBQUMHbsWL799lvGjx/Ps88+2+jj9fFEWUcAqwEyxtTmkksuITY2ts5tZs+ezdKlSznxxBMBOHz4MJ07d65x2wsuuACA4447jvz8fFJSUkhJSaFVq1bk5ubyySef8Mknn3D88ccDkJ+fz4YNGxg/fjyPP/44b7/9NgA7duxgw4YNpKamkpCQwHnnnQfAyJEj+fTTT5t83N4IAGJtAMYckebMqX1dUlLd6zt1qnt9AyQnJ/un4+LiqKio8M/7nqtXVa666ioeeOCBevfXqlUrAGJiYvzTvvmysjJUlSlTpnDddddV+dycOXOYNWsW8+fPJykpiQkTJvjTj4+P9z/mGRsbS1lZWSOPtpInqoCMMSZUGRkZLFu2DIBly5axZcsWAM444wzefPNN9u3bB8D+/fvZtq3GXpbrdfbZZ/PCCy+Qn58PwM6dO9m3bx8HDx6kQ4cOJCUlsXbtWhYsWBCGI6qdJ0oAYG0AxpjQ/OhHP2LGjBkMGTKEMWPGMGDAAAAGDx7Mfffdx1lnnUVFRQXx8fE8+eST9O7du8FpnHXWWaxZs4aTTjoJcBp4Z86cyaRJk5g2bRqDBg3i2GOPZezYsWE9tuqOqjGBR40apY0ZEOaed1fx5pJMVtxzdgRyZYwJ1Zo1axg0aFC0s9Gi1fQdi8hSVR1VfVtPVAEJYiUAY4ypJqoBQETai8ibIrJWRNaIyEmRSQdrBDbGmGqi3QbwGPCRql4sIglAUiQSsa6njDlyqKp1CBchDb3RjVoJQETaAeOB5wFUtURVcyOVnt3/GxN9iYmJ5OTkWIk8AnzjASQmJob8mWiWAPoAWcCLIjIcWArcoqoFgRuJyGRgMjhv1zWGUwXUtMwaY5quZ8+eZGZmkpWVFe2stEi+EcFCFc0AEAecAPxaVReKyGPAH4C7AjdS1enAdHCeAmpMQiKCWhnAmKiLj48PebQqE3nRbATOBDJVdaE7/yZOQAg76wrCGGOCRS0AqOoeYIeIHOsuOgNYHZHErL3JGGOCRPspoF8Dr7hPAG0GrolUQlYAMMaYqqIaAFR1ORD0dlq4CdYftDHGVOeNN4EFawQ2xphqvBEAsEZgY4ypzhsBwBqBjTEmiCcCAFgTgDHGVOeJACDYiGDGGFOdNwKAPQRkjDFBvBEAsEZgY4ypzhMBwFqBjTEmmDcCgDHGmCCeCAC++39rCDbGmEreCABuBLDrvzHGVPJGAHDLAHb9N8aYSt4IANYGbIwxQTwRAHysDcAYYyp5IgD4G4GjmgtjjDmyRHU8ABHZCuQB5UCZqkZkbABrBDbGmGDRHhEM4DRVzY5kAiK+RmCLAMYY4+OJKiBjjDHBoh0AFPhERJaKyOSaNhCRySKyRESWZGVlNS0xKwAYY4xftAPAKap6AnAOcKOIjK++gapOV9VRqjoqLS2tUYnYY6DGGBMsqgFAVXe6v/cBbwOjI5GO/0UwKwEYY4xf1AKAiCSLSIpvGjgLWBmZtJzf1ghsjDGVovkUUBfgbfcJnTjgVVX9KBIJWQ2QMcYEi1oAUNXNwPDmTbM5UzPGmCNbtBuBm0VlFZAxxhgfbwQAfyOwhQBjjPHxRgCwEoAxxgTxRAAwxhgTzFMBwGqAjDGmkicCgFgdkDHGBPFGAHB/24tgxhhTyRsBwMYDMMaYIN4IANHOgDHGHIE8EQB8rABgjDGVPBEA/COCWR2QMcb4eSQAOL/t8m+MMZW8EQDc31YAMMaYSvUGABEZICKzRWSlOz9MRO6MfNbCyIYEM8aYIKGUAJ4FpgClAKr6HXBpJDMVKfYegDHGVAolACSp6qJqy8rClQERiRWRb0TkvXDtMygN34Rd/40xxi+UAJAtIn1xL58icjGwO4x5uAVYE8b9BbFGYGOMCRZKALgReAYYKCI7gVuBG8KRuIj0BM4FngvH/mpNxwaFN8aYIPUOCekO3TjRHbg9RlXzwpj+o8DtQEptG4jIZGAyQHp6eqMSsTZgY4wJVm8AEJHfVpsHOAgsVdXljU1YRM4D9qnqUhGZUNt2qjodmA4watSoJt3DWyOwMcZUCqUKaBRwPdDD/bkOmAQ8KyK3NyHtk4ELRGQr8DpwuojMbML+amXvARhjTLBQAkBP4ARVvU1VbwNGAp2B8cDVjU1YVaeoak9VzcB5rPQzVb2isfurizUCG2NMsFACQGegOGC+FOiiqoerLT9i2aDwxhgTrN42AOAVYKGI/NedPx941W0UXh2OTKjqHGBOOPZVI2sENsaYIKE8BXSviHwEfM9ddL2qLnGnL49YziLACgDGGFMplBIAqrpYRLYBiQAikq6q2yOaszCyAoAxxgQLpTO4C0RkA7AF+ML9/WGkMxZOleMBRDkjxhhzBAmlEfheYCywXlX7ABOBBRHNVZjZoPDGGBMslABQqqo5QIyIxKjq5zjvBhw17E1gY4wJFkobQK6ItAG+BF4RkX1AQWSzFRlWBWSMMZVCKQFcCBQCvwE+AjbhPAp61LAXwYwxJlidJQARiQXeU9XTgArg5WbJVZjZi2DGGBOszhKAqpYDFSLSrpnyExFWAjDGmGChtAHkAytE5FMC6v5V9eaI5coYY0zEhRIA/u3+HPWsBsgYYyqF0hXEyyLSGkhX1XXNkKewE6l8E8AYY4wjlDeBzweW4zwBhIiMEJF3IpyvsLLxAIwxJlgoj4FOBUYDuQDuKGDHRCxHEdDQRuDNWfms3nWozm2KSsspLa9oWsaMMSaKQn0T+GC1ZUfVlU8a2B3c6Q9/wfcfn1vnNgPv+ohLps1vSraMMSaqQgkAq0Tkp0CsiPQXkf8D5jU1YRFJFJFFIvKtiKwSkXuaus/6hLsKaPmO3PDu0BhjmlEoAeDXwBCc0b9exRkQ/tYwpF0MnK6qw4ERwCQRGRuG/QaprAJqWASwF8eMMS1ZKI+BDlTVO4A7wpmwOlfXfHc23v2JyBW3IY3Aew4W+af3Hiqma7vEoG0sMBhjWoJQSgAPi8gaEblXRIaGM3ERiRWR5cA+4FNVXRjO/Vem4/wO5br9bWauf3rsA7Nr3Kao9KhqAjHGmBrVGwDcfoBOA7KAZ0RkhYjcGY7EVbVcVUcAPYHRNQUYEZksIktEZElWVlYjUwq9EfhwSXm92xSWlDUyH8YYc+QIpQSAqu5R1ceB63HeCfhTODOhqrnA58CkGtZNV9VRqjoqLS2taemEUMN0uDSUAFD/NsYYc6QL5UWwQSIyVURWAL4ngHo2NWERSROR9u50a+BMYG1T91tzWs7vUKqAikIIAAUBJQBrDzDGHK1CaQR+AXgdOFtVd4Ux7W7Ay26X0zHAP1X1vTDu368hbwFULwF8sGI3W7ILuPG0fv5l+w4V+6cPHi6lfVJCU7NojDHNLpS+gE6KRMKq+h1wfCT2XZ2vL6B5m7IZ2qPunq2rN/D+6pVlAKR3TOLvn67n2K4pfLhyj3/9NS8t5u1fnVzjvr5Yn8WwHu3okGwBwhhz5AmlCqi/iLwpIqtFZLPvpzkyFy6+EsD9H9Rfw1RUWk5CXPDX8uvXvmFzdkGViz/AN9tzAThYWMq2nMqRMvcXlHDVC4u49Y3ljc12kCn//o7HZm0I2/6MMd4WSiPwi8DTQBnO00AzgJmRzFQ0vPDVFnbmHqaotJykhFiuOTmDlFah1JBBWXkFYx+YzakPzvEv25zlvOIQ+F5BUxw8XMpri3bw91nrrQ8iY0xYhBIAWqvqbEBUdZuqTgXOjWy2wkvqaQTYX1DC/763miufX0hhSTmt42NJjI+lqKz2BuHv9U0FoH1SPA9+vM7fdnDaQ3P4039X8ths50593d68kB4trc832w/4pzfsza9jS2OMCU0oAaBYRGKADSJyk4j8EGgT4XyFVX0BwPckz+7cIvKLykhJjKNVXAyl5bU/4XNyv06M69+JuJgYXvx6q3/5luwCZszfxtwN2f5lk/+xpEn5B1iweb9/euXO6n3zGWNMw4USAG4BkoCbgZHAFcBVkcxUuNXXG2hZhXOhP1xazqGiUtomxtOudXydn2nXOp6UxDiy84spqadKJjAYNNS8jdkMuusjVu06SLd2ibRPiuerjY3fnzHG+ITyJvBiVc1X1UxVvUZVf6SqC5ojc2FTx/V/8db9PD1nk3/+QGEpbVvH0zax7gDQo0PrercJh6fmbOJwaTlzN2RTUlbBD0b04J1vdzHPgoAxpolCehO4Jbtk2nxemrfVP785K5+UxDhi6vhmrjqpN6f060RifGzI6fzqlaUNzlvmgcIqd/s5BSXcPulYAJYFtAkYY0xjhPaYy1Eupr5GgADFZRW0aRXHcbW8L5CRmsQ9FzpdFt06sT+t4mOYuz6bk/qm8vxXW6ps+7eLh7Fs2wFeX7yDD1bsoaSsosZHTKsrLa/gvvdWk51fUmX5Q5cMJykhjvZJ8ewO09NFxhjv8kQJoGHjgUGruFj6dU5h5s/HBK07uV8n/3T7pASmnDOID24Zx9hjUoO2Hdg1hbvPH+KfX7J1f9A24DRCb9yXz479hYAz0MzL87fx/ordtE2M45fj+jBpSFd+eHwPALq2TWRn7uEGHpUxxlRVawnAHfmr1sdgVPXmiOQoAhpSAgCIj3O2P6V/J9beO4mBd30EwNlDunDPBUNq/EznlFZBy9q1jqd1QmU10bUvL2btvecEbff2Nzv57T+/BeC2Mwfw8Kfr/esmDurCHecOrrL94O5t+feynazIPMhxPet+s9kYY2pTVwlgCbC0jp+jRkwDiwDxAQ0ArQKqbC4c0YO42Jq/sp4dWgct8z1JNOPa0aQmJ1BUWkFeUWnQdoFvF0//supL1oO7tw3a/pKRvYDKsQtWZB4My7sGxhhvqbUEoKovN2dGIkkaWgIIuMgHfrauR0M7Jicw9piO/OTEXvzmDeduPsV9Smj8gDQe+vFwrnlxMSt2HiRWhG37C/nxKOdC3iHJ2a51fCx5xU5Po+cP705uYQlnD+kalNaJGR0AuPM/KzmlXyfOf+IrxvXvxD9qqLIyxpja1NsILCJpwP8AgwH/+IiqenoE8xVWDS4BxNX8gboCgIjw+mSn37xvdxzkpXlbiQ1IeIh7J79+Tx5T310NwCUjeyIilJYryQmxfDf1bH741Nd8l3mQX47rw7Ce7WtMK7AUMuGhOQAs3Fxz+4IxxtQmlEbgV4A1QB/gHmArsDiCeQq7mAZGgIRaqnnaJ4X23P/UC4aw9S9Ve8to39rpETS/uHIsgdxCpzooK6+Y/l1SiI0RXvvlWB758fBan0LyeemaE6vMl5RX8Nzc2vvom7lgGw99vC6k/BtjvCGUAJCqqs8Dpar6hapeCxw1d//Q8BJAXC0f6NCEfv8T4mJIiI3h5fnb/MsyDxymtLyCrzZm06mNs+/kVnFcdELPequtJhzbmbm3n8baeycxcVAXAO57fw1XPLewxs7i7vzPSp74fCN7DxVxqIZ2CGOM94QSAHxXi90icq6IHA90jGCewq6ui2lN1TrxtTyrnxxi76C1SWoVS1Ze5WAy2QXFPPOF8xbygkZU4fTqmERifCyPXTqCn45JB+Crjdk8N3cLFz7xFR+s2B30mTH3z+YXLze9byJjzNEvlCvafSLSDrgNZ0jItsBvmpqwiPTC6Vq6C87jptNV9bGm7rcmdT0GOrBrCgu3VL34xtdSBdRUJWVV78xz8kvYsM/p2XNqLY+XhiK5VRz3//A4Xl24HYC/fuSMe/CrV5bRqU0CPdpXfUJp0Zb9qGqDG8eNMS1LKH0BvaeqB1V1paqepqojVfWdMKRdBtymqoOBscCNIjK4ns80Sl2XufIK5bge7bjrvMH+apjqbQAPXTKcBy46rsn58A0mf3I/56WxT1bt4bvMgwzv2Y6LRzZ5mGX+dF7w15edX8K3mU7voVd/L4NbzugPwLvfBZcOjDHeEsqIYC/7Bm935zuIyAtNTVhVd6vqMnc6D6ehuUdT91uT2koABwpKWLLtAO2T4vn5KX3whYq42KrbXzyyJ5eNTm9yPnyB5fmrTmREr/Z8snovW7ILSG0T/BJZY1x7Sh9mXDsagLvPH1wlqNzx/UFMvWAIk8cfA8DNr31jA9ob43GhVAENU9Vc34yqHnDbAcJGRDJwxgdeWMO6ycBkgPT0xl2Ea6vpuPrFRTUuj1QV0Fs3fA+AxPhY+nduw/IduWFPY/yANN696RSGdG9LTIzw0CXDq6xPbhXHpSf24vXFO/hifRYTju0c9jwYY44OoVzpYkSkg29GRDoSxk7kRKQN8BZwq6oeqr5eVaer6ihVHZWWltaoNAJLAIF3vWv35AFQUe1OuLbHQJvquJ7t/F03tA1ofPZVy4Qznboefb3nwiG0jo9lzrqssKZrjDm6hHIhfxiYLyL/wqkjuRj4czgSF5F4nIv/K6r673DssyaBXTuXV6i/iicuRigG/8hfvkAQSo+dTZXqtjc8ePEwhvdqH/H0ArWKi+XYrimscwPgByt2k94xiaH1vHtgjGlZ6g0AqjpDRJZQ+ez/Raq6uqkJi/MIyvPAGlV9pKn7q0tgCaAi4Gbf96Zuubuw1H1Kp6mPe4bip6PT6dG+NecP6x7xtGqS3jGJd77dRcYf3vcv+/Q34+nfJSUq+THGNL9ab3VFpK37uyOwB3jV/dnjLmuqk4ErgdNFZLn78/0w7DdIYG1IYHWPr0sF35CQvqEdkxNCH+ilsdonJXDhiB4Nfks5XEb27hC07P0a3hswpj4zF2xj5oJtlJZXkFtYUv8HzBGjrlvdV4HzcHr+DKwkF3f+mKYkrKpf0fCu+htFqrQBVC73lQAq3ADgCwRJzVACiLafjklHBB6fvZGXrz2R37yxnEdnbaBPp2QuHBGRh7FMC/SP+Vu567+rAOdt85TEOGb/9lQ6t/V3G+Zvd2vseydFpeUs2JzDuP5pZOUV88nqPYzpk8qxXZteWi0uK+fNpZkM6d6OEc1cFXskqKs30PPcappTVXV7M+Yp7AKrgMoDIkC8GwB8df6+qqCkBgz1eLSKj43hZydl8LOTMgC4+Yz+3PTqN9zy+nIuHNGD/OIy2nggEDbUvkNFZOeXkJbSClWlc9tEKiqUkvIKEuNjySsq9fcC21It3rqfZ77YzE9O7MUDH64lRmBI93as2HmQvKIyznlsLq3iYujbuQ0dkhJ459tdfK9vKn+7eBg92remrEL5n7e+Iye/hJ+OSWdMn460T0ogJ7+YPYeKGNLdaYvyvax4xXMLWbLtAOcP7853mblsy3EGTjpnaFfSOyZx+ZjepKcmoaocPFzKN9tz+eeSHSzbfoAZ146pEii2ZhdwqKiUT1fv5cnPN1apEv7pmHSuGNObgV1TmlQyLyuvQESqdAZ5pKrzP1xVVUTeB5r+FlQUBZ6GwCog38lPqlblk9Sq5QeA6s4b1p0Ne/N5bPYGfvzMfBZt2c/tk47lyrG9W/wFLVR3vL2CVxZWvRfqmJzA/oKq1R6Xje7FxSN7cnyvDlGr4ouUHfsLuf4fS8kpKGHWmr2IwEe3jPdfZL9Yn8VVLziPV+8KGLZ03qYcTvnr54zO6EjrhFi+WJ/l3x6gf+c2bM0p8D+Q4dM3LZlNWQUAvPvtLgAev+x4Zs7f5h9H45kvNzOwawqqsG5vXpXPn/3ol/ROTaJ9UgI927fmw5W7/f/3CXExnNw3lbat41m96xCvLtzOqwu3k5bSiivH9uZXE/pW6Xm3tLyCfy/LZGduEat3HaJ3ahJnDOrM3A3ZCHDdqX15Y/F2Hp+9kW7tErnnwiEM69n+iL6RkvpeBhKRl4EnVDXqPYCOGjVKlyxpeD8223MKGf/g5wB8+6ezaOf26jn07o/JLy7jzMFdePZnozjj4TlsyipgywPf92Q3CTn5xYy8b1aVZb84pQ931vCGsZeUVyh/fn8NL3ztjPk8cVBnlu84SHZ+cZ2fu/PcQZw2sDPHdEqu9e9pydb9vDRvK6cOSGNLdgG/GHcMHZOdJ8QKS8pISojjUFEpsSLsLyghLlbo1s7p2mPjvjzyiso4rke7Wgcqaoot2QV8vTGbURkd6NSmFcu35/KLGc7/32/PHEBZeQVjjkmtMkwqwOasfOasy+LMwV1YtesQE45N48nPNzLti020TYwnp6CEi47vQXpqEqt3HWL22n3+0ndcjPirYn16tG/Nh7eOY+aCbQzsmsLpA7ugqny0cg8rdx3kyc83+S/ycTHCgC5tuOHUfuw5VMTPX1pMWYVSXFZOx+QEzhvWndF9nCA0YUCa/7yUVyivL95ObmEpX67P8ncPk5GaRO/UZLblFLDVLXnUJy2lFbmFJZSWK13bJnLusG5kdErmvW93sXDLfi4c0Z2ubRM5qW8q/Tq3oWeHpCadp1CIyFJVHRW0PIQAsBboB2wDCnDbAFR1WCQyWpfGBoAd+wsZ9zcnAHxz15l0cP/BBtz5ISVlFVx6Yi/+8qNh5OQXsyW7gFEZR1Vfd2F17uNzWbWr6usYy+46039Ragnmbcrmp88u5LpTj2HKOYPILy7jqw3OS3GJ1ar/3vl2Fze/9o1/fs7vJpDRKRmA5+ZuZnC3toxIb09cTAwJcTFszsrnk9V7+dtHa/13msekJfPBzeOq7LuwpIyyCuWy6QuqfN/nDO3K+cO7c6CwhDveXllj/s8c3IW5G7IoKnUeWmjXOp63bjiJfp0bVydeWFLG3f9dxeKt++nXuQ3nD+/OwK5tOf+Jr4L6rwL49en9+O2ZAxp1k1RRoRSXVVQZKhWcev79BSV0D+i3SlWZuyGboT3a1fn3V16htVa3BPZ5FWr/V6rKq4u288wXm9m+v/KiP6JXe4b3bMdZQ7oSHxtDx+R4tmQX0jYxjjW7nUA2sncHrj+1L2v35PF/szcwe+2+etMb0as99144NKLDuzYlAPSuabmqbqtpeSQ1NgDszD3MyX/5DICld070d71ww8ylfLhyD9/efVadg714SU5+MY/N3sCM+VVP72OXjjjqG4f/u3wn97y7ukqVzcOXDGfqO6vIKy7je31TmXblSNq6VV7lFUrfP34AOHehj192fI1PT9Vk+pebuP+DtVWWvXjNiWSkJnPnf1awdnceOW4+Rmd0ZMXOgxwurXtYz9TkBP9nfEb0as/yHblcMTadqecP8ZcEyiuU+ZtyeH3xdrbvL2Roj3bcee4gkhKqVkcUlZb7x7yuLi5GuHVifw4VlbFgcw7ZecX856aT6ZySWOP2LdGBghLaJ8VTUFLeqKqc8grl09V7KC6roH/nFPp2TqaguJz4WOGztfvIPHCYmQu2UVRazr+ub3wQr0+DA4CItFXVQ7U98qmqzT4EVWMDwO6Dh9l2nNNHzgm9O/jf9H0tYywvDj+HT647Eb5fwxOoV1/t/GRnw8UXB6+/4Qb4yU9gxw648srg9bfdBuefD+vWwXXXBa+/806YOBGWL4dbbw1ef//98L3vwbx58Mc/Bq9/9FEYMQJmzYL77gte/8wzcOyx8O678PDDwev/8Q/o1QveeAOefrrKqnJV9r/8CidO+5aLV8zi4hWzGNAlpeqd2AcfQFISPPUU/POfwfufM8f5/dBD8N57Vde1bg0ffuhM33svzJ5ddX1qKrz1ljM9ZQrMn191fc+eMHOmM33rrc53GGjAAJg+3ZmePJmytetYsrXyTza73yBuGn0VAH9/9yHSC/dTWl5BbIzQv0sK7U8fz+Lrfs8l0+bzwZePMiihtOoja2ecAXfd5Uyfcw4cPlw1/fPOI/dXN9OudTzbho1mT0B9OMB7A8cx84RzGZGawFvv3kdsjFChsL+ghJz8YmaNOYfLn/4T61dtod+vrvF/74dLyyksKSfl1l9z+KKLaZe9h7Vn/sD/+GVaSiu6tk3k78PO57kOQzkmJ5P7P37Cn26/zm1IjI8l4e67SJh0Ns8+/hbHPTSVxPhYBnZtiwhs2JfPExOvoe+FZ/L7lP3wxz+iOHfG/gcqIvi3B8Cbb0KnTvDSS85PdUfR3x7r11ddP2KE8/0BXHEFRVu2sWrXIcpV6ZfWhlbjTyZrylTyi8sYevO1kJNDhSoxX3wRfJwhqi0ANOQx0MC//yY/BtqcausMrkKV2JjIv/V7NIoVIa1NInNvP43P/rAYVsD6vXkM6d6OlMQjt1GrNr4LpO8CnzK0G78c14fXF+3guB7t6FlQxr68YvblFbN29yH65Rdz06vLaNMqjmPSkpGDuQ1Os707gFBGajKJ8bFszXYaMwd3b8uYC4dw343nQmEhvO/8fcYIdGqTQKc2CRx7Wj9ISmB0n1QICLqt42NpHR8LcTEkuKXWvmnJbM52RpjLyismK6+YnemHuWRiT27p1Z2u6zqyO/cwOw8WsdHtfnzajKV8u0jotmUXJ7eOZ5B78QcY2r0t064YCd8b6Nx84Pzze7FdrDkkxscysFtb1u3JY/3ePJbN28rf3KFe/739AB2LCsnJLyYh82DYq4nqrQI6kjS2BLAvr4jRf3ai/Pwpp/sb0a55cRHZ+SW8++tTwprPluied1fx4tdb/fN//uFQWsXF8urCbTx1+Ui6tmueaoGKCuWZLzezM7eQ3511rP8iW13gs+el5RX8+tVv+HpTNsv/dFadj+fN35TDZc8u8M/f8f1B/HJ8eO519uUVkdamVcQupCVlFfzmn8tJio/l1jMHBI0D8dHK3Tw9ZxP9u6SwK/cwZRVKx6QEHrjoOH+7mImewyXl3P/BGnYfLGJc/05s2JfHKwu3owonZnTg/h8e1+g39RvdBuB++CLgFJw7/7mq+p9G5aKJGhsAsvOLGeU+3dKrY2u+/P1piAhXPr+QvKIy/nPjyeHOaov01tJMbvvXt0HLB3ZN4aNbx0ckzU9X72VzVj5ZecWc1DeVzVkF/PmDNQC0ioth1m9PpVfHJN7/bjez1+7lT+cNJiUxnqtfXMTcDdlV9jVpSFemXTmy3jRfXbidBz5YQ3xcDAv/eEbEeoc1pj7Z+cVUVChpKU27cWhMFZDvg0/hPAX0mrvoehE5U1VvbHRumllgFdCO/YfJLy4jJTGe8golPtaKtaH60cienDWkC+v25HHxtMp60bV78njw47X8+vT+ZB44TL/ObcKS3sqdB/nljMqA/9xXW/zTMQLFZRWM+9vnzPrteG58dRngvOizbHtu0L56tG8d8p38T8ekc9noXlblYaKuU5jGCqlNKJW5pwOD1C0quO8FrIporsKs+r+xr8xTVsfjY6ZmKYnxjMroyAMXHUd+URkTB3fhtIfm8OTnm3jyc2d84z//cCiXj6nx4TFn/furSU9NpleH1uQVlTGgS4r/RaLisnK2Zhdy139Wsjm7gBiBc47rxvo9ef7hM//6o+M4d1h3Js9YwrxNOUx85EsAenZo7b/492jfmsvHpnPO0G6kd0xq8Hm2i7/xglACwEYgHec9AIBe7rKjRvVGYF+tV3mFkuCBbh8iIXCEtK/+5zRO+evn/vnn526pNQA88ul6np27JWj50jsnBr2EBs7LVL8Y59y5b9yXT05+MWOOcYbUfPZnoxhy98cAPHX5CXRMTuDS6U79/Ts3nRy2kdaMaalCCQApwBoR8Q2fdSKwRETeAVDVCyKVuXCR6lW4bgCwEkB49OyQxF8uOo5B3dqyZNsB7n1vNZkHCv1vOKoq17y0uM4BaGq6+E8ef4z/4g/OI4yB1UvJreKYP+V0YkTo4nY+tuZ/J7Ez97Bd/I0JQSgB4E8Rz0WEBZUA3AhQXFpOQopdKMLhUrdE4HvD85S/fs5nt53KG0t28OJXW/1dbSfExvDGdWPZ7T4Xf/aQroy671MOFJZW2d/s206lb1r9bQm+J7p8WifEhq0NwpiWLpQBYb4A//gAcQHLm/1FsMaqfpPvqwIqLClvlr7/vaR/5zYkJ8RSUFLO6Q9XfXHl/OHd+b/LnOGkAweVnj/lDJ76fCObsgp46JLh7Mw9HNLF3xjTNKE8BTQZ+F+gCKggTOMBuPt+Aedls32qOrSp+6tNcAnAUVhSTuuEo++lpiOZiPDV/5zO8fd+6l82cVBn7jh3MH3cPnSqS4yP5bdnHeuftzt4Y5pHKFe/3wNDVTW73i0b7iXgCWBGBPbtV/2BDt+7D4UlZVYCiIAOyQl8dOs41u/N53BJGT8eZY9UGnMkCiUAbAJC6we1gVT1SxHJiMS+A1UvAVSo80bp4dLyoLEATHgM7NqWgV3bRjsbxpg6hBIApgDzRGQh4O8AXVVvjliuArhVUJMB0tPT69m6ZjU1AheVlaPqjeEfjTGmJqFc/Z4BPgNW4LQBNCtVnQ5MB6criMbsI6jyQaGg2Ol610oAxhivCiUAxKvqbyOekwgKagMA8oqcxw7b2nCHxhiPCqWXqw9FZLKIdBORjr6fiOcsjKo3QKpWdg/cPskCgDHGm0IpAVzm/p4SsCxcj4G+BkwAOolIJnC3qj7f1P3WR1Fy3RePautO2BhjWrpQXgTrE6nEVfWy+rcKvwrF/+ZpBysBGGM8qtYqIBG5PWD6kmrr7o9kpiLhNxMH+KdVlcKSMoCgMVKNMcYr6moDuDRgekq1dZMikJeI+l6/VP+0KpSWOw8U2XgAxhivqisASC3TNc0f8ar3B1Tmdk5moz0ZY7yqrquf1jJd0/wRL/BJIFWnK2iAOCsBGGM8qq4K8OEicgjnbr+1O4073zwjgIdR4NvAilLqKwHEWAnAGONNtQYAVW1Rr8gGVgFVKJSVKzECMTYgjDHGozxz+1ulBKBKaUUFcVb/b4zxMM9cAatWATklgHi7+zfGeJh3AkDAkTqPgVoJwBjjbZ65AlbtElopLVd7BNQY42meuQLGVH8MtLzCXgIzxniahwJA5bTivAdg7wAYY7zMQwGg8mJfoc57APYOgDHGyzxzBQyuArISgDHG2zwTAALbgJ2uICqIsxKAMcbDonoFFJFJIrJORDaKyB8im1bl9MHDpSzddoDu7VtHMkljjDmiRS0AiEgs8CRwDjAYuExEBkcqPQ3ovm7+pmwOFJbyi3ERG+vGGGOOeNEsAYwGNqrqZlUtAV4HLmyOhIvKnI7g0lJaNUdyxhhzRIpmAOgB7AiYz3SXVeEOSL9ERJZkZWWFJeHi0nLAegI1xnjbEX8FVNXpqjpKVUelpaWFZZ8lvq6g4+wpIGOMd0UzAOwEegXM93SXRURgI3BxmY0GZowx0bwCLgb6i0gfEUnAGYP4nUgllhBXeaglFgCMMSZ6AUBVy4CbgI+BNcA/VXVVpNLrnJLI5WPSgcoAkGABwBjjYVG9AqrqB6o6QFX7quqfI53emYO7AHCgsATAOoMzxniap26Bfd1BLN56ABGItQFhjDEe5qkAENgQHB8bg4gFAGOMd3krAFB5wY+1i78xxuO8FQCk5mljjPEibwWAaGfAGGOOIJ4KABYBjDGmkqcCQOCgMPYEkDHG6zwVAAIv+fYWsDHG6zx1FQx87DPGWoGNMR7nsQBQOR1nVUDGGI/zVgAImLY2AGOM13krAASWAKwfIGOMx3kqAGBvAhtjjJ+nAkBgrU96alL0MmKMMUcATwWAwKeAHrv0+CjmxBhjoi8qAUBELhGRVSJSISKjmi3dgOl2reObK1ljjDkiRasEsBK4CPiyORO1an9jjKkUF41EVXUN0Oz98Yt1BmSMMX4eawOIdg6MMebIEbESgIjMArrWsOoOVf1vA/YzGZgMkJ6e3sQ8NenjxhjTokQsAKjqxDDtZzowHWDUqFHalH1ZFZAxxlSyKiBjjPGoaD0G+kMRyQROAt4XkY+bJ93mSMUYY44O0XoK6G3g7eZO16qAjDGmklUBGWOMR3krAEQ7A8YYcwTxVgCwIoAxxvh5LABEOwfGGHPk8FQAsHGAjTGmkqcCQHJCbLSzYIwxRwxPBYCUROsC2hhjfDwVABLjPXW4xhhTJ09dEe0pIGOMqeSpAGCMMaZSVLqCiKZHfjycrm0To50NY4yJOs8FgItO6BntLBhjzBHBqoCMMcajLAAYY4xHWQAwxhiPsgBgjDEeFa0RwR4UkbUi8p2IvC0i7aORD2OM8bJolQA+BYaq6jBgPTAlSvkwxhjPikoAUNVPVLXMnV0A2LOZxhjTzI6ENoBrgQ9rWykik0VkiYgsycrKasZsGWNMyyaqGpkdi8wCutaw6g5V/a+7zR3AKOAiDSEjIpIFbGtkljoB2Y387NHKjtkb7Ji9oSnH3FtV06ovjFgAqI+IXA1cB5yhqoXNkN4SVR0V6XSOJHbM3mDH7A2ROOaodAUhIpOA24FTm+Pib4wxJli02gCeAFKAT0VkuYhMi1I+jDHGs6JSAlDVflFIdnoU0ow2O2ZvsGP2hrAfc9TaAIwxxkTXkfAYqDHGmCiwAGCMMR7liQAgIpNEZJ2IbBSRP0Q7P+EgIr1E5HMRWS0iq0TkFnd5RxH5VEQ2uL87uMtFRB53v4PvROSE6B5B44lIrIh8IyLvufN9RGShe2xviEiCu7yVO7/RXZ8R1Yw3koi0F5E33f6z1ojISS39PIvIb9y/65Ui8pqIJLa08ywiL4jIPhFZGbCswedVRK5yt98gIlc1JA8tPgCISCzwJHAOMBi4TEQGRzdXYVEG3Kaqg4GxwI3ucf0BmK2q/YHZ7jw4x9/f/ZkMPN38WQ6bW4A1AfN/Bf7uPlxwAPi5u/znwAF3+d/d7Y5GjwEfqepAYDjOsbfY8ywiPYCbgVGqOhSIBS6l5Z3nl4BJ1ZY16LyKSEfgbmAMMBq42xc0QqKqLfoHOAn4OGB+CjAl2vmKwHH+FzgTWAd0c5d1A9a5088AlwVs79/uaPrB6TdqNnA68B4gOG9HxlU/38DHwEnudJy7nUT7GBp4vO2ALdXz3ZLPM9AD2AF0dM/be8DZLfE8AxnAysaeV+Ay4JmA5VW2q++nxZcAqPxj8sl0l7UYbpH3eGAh0EVVd7ur9gBd3OmW8j08ivMSYYU7nwrkamXngoHH5T9md/1Bd/ujSR8gC3jRrfZ6TkSSacHnWVV3Ag8B24HdOOdtKS37PPs09Lw26Xx7IQC0aCLSBngLuFVVDwWuU+eWoMU85ysi5wH7VHVptPPSjOKAE4CnVfV4oIDKagGgRZ7nDsCFOMGvO5BMcFVJi9cc59ULAWAn0Ctgvqe77KgnIvE4F/9XVPXf7uK9ItLNXd8N2Ocubwnfw8nABSKyFXgdpxroMaC9iPheagw8Lv8xu+vbATnNmeEwyAQyVXWhO/8mTkBoyed5IrBFVbNUtRT4N865b8nn2aeh57VJ59sLAWAx0N99giABpzHpnSjnqclERIDngTWq+kjAqncA35MAV+G0DfiW/8x9mmAscDCgqHlUUNUpqtpTVTNwzuNnqno58DlwsbtZ9WP2fRcXu9sfVXfKqroH2CEix7qLzgBW04LPM07Vz1gRSXL/zn3H3GLPc4CGntePgbNEpINbcjrLXRaaaDeCNFNDy/dxRh7bhNMdddTzFIZjOgWnePgdsNz9+T5O3edsYAMwC+jobi84T0NtAlbgPGER9eNowvFPAN5zp48BFgEbgX8Brdzlie78Rnf9MdHOdyOPdQSwxD3X/wE6tPTzDNwDrAVWAv8AWrW08wy8htPGUYpT0vt5Y84rzpgqG92faxqSB+sKwhhjPMoLVUDGGGNqYAHAGGM8ygKAMcZ4lAUAY4zxKAsAxhjjURYAjGkCEVEReThg/nciMjWKWTImZBYAjGmaYuAiEekU7YwY01AWAIxpmjKcsVp/E+2MGNNQFgCMabongctFpF20M2JMQ1gAMKaJ1OmFdQbOICbGHDUsABgTHo/i9OWSHOV8GBMyCwDGhIGq7gf+SeUwhcYc8SwAGBM+DwP2NJA5alhvoMYY41FWAjDGGI+yAGCMMR5lAcAYYzzKAoAxxniUBQBjjPEoCwDGGONRFgCMMcaj/h/+dnS0tljULQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(s)\n",
    "plt.plot(mean,'--', color = 'r', label = 'True mean')\n",
    "plt.title('Visualisation of the law of large numbers')\n",
    "plt.xlabel('N')\n",
    "plt.ylabel('Empirical average')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"border:1.5px solid gray\"> </hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3. Pandas\n",
    "\n",
    "<a id='pd'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas is a library, created by Wes McKinney, to make data analysis with a focus on tabular data. It is built on top of `numpy`. The backbone of `pandas` is the class `DataFrame`, whose objects store data in a spreadsheet like way. Since this class has such a rich variety of functionalities, we will only cover the most important aspects in this course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a DataFrame \n",
    "<a id='pd1'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Normally you will not build your own data frame, but I still want to present a way how to do it.\n",
    "Let say we have the dictionary `parts_europe` from a previous exercise. You can create a Data Frame by calling `pd.DataFrame()` passing `parts_europe` as an argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "parts_europe = { \"country\":[\"Italy\", \"France\", \"Germany\", \"Poland\", \"United Kingdom\"],\n",
    "                 \"capital\":[\"Rome\", \"Paris\", \"Berlin\", \"Warsaw\", \"London\"],\n",
    "                 \"EU-Member\":[True,True, True, True, False]}\n",
    "df_parts_europe = pd.DataFrame(parts_europe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>country</th>\n",
       "      <th>capital</th>\n",
       "      <th>EU-Member</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Italy</td>\n",
       "      <td>Rome</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>France</td>\n",
       "      <td>Paris</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Germany</td>\n",
       "      <td>Berlin</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Poland</td>\n",
       "      <td>Warsaw</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>United Kingdom</td>\n",
       "      <td>London</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          country capital  EU-Member\n",
       "0           Italy    Rome       True\n",
       "1          France   Paris       True\n",
       "2         Germany  Berlin       True\n",
       "3          Poland  Warsaw       True\n",
       "4  United Kingdom  London      False"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_parts_europe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to manually change the index of the dataframe, which is just the numbers 0,1,2,... for now. It makes sense to define it in a natural way. Normally one row of a dataframe stores different features for a single instance, e.g. a certain timestamp, country, etc.. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_parts_europe.index= [\"IT\", \"FR\", \"DE\", \"PL\", \"UK\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>country</th>\n",
       "      <th>capital</th>\n",
       "      <th>EU-Member</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>IT</th>\n",
       "      <td>Italy</td>\n",
       "      <td>Rome</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FR</th>\n",
       "      <td>France</td>\n",
       "      <td>Paris</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DE</th>\n",
       "      <td>Germany</td>\n",
       "      <td>Berlin</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PL</th>\n",
       "      <td>Poland</td>\n",
       "      <td>Warsaw</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UK</th>\n",
       "      <td>United Kingdom</td>\n",
       "      <td>London</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           country capital  EU-Member\n",
       "IT           Italy    Rome       True\n",
       "FR          France   Paris       True\n",
       "DE         Germany  Berlin       True\n",
       "PL          Poland  Warsaw       True\n",
       "UK  United Kingdom  London      False"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_parts_europe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing data into a DataFrame\n",
    "<a id='pd2'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's get to the more important part: *Importing data*. By using Pandas `pd.read_csv()` you can read CSV (Comma Seperated Values) Files into python and assign it to a variable. The cool thing is, you do not even need to download it. You can just pass a link from a Github Repository as an Argument and it will read the data from there. As an example I will take the Airline Safety Dataset from FiveThirtyEight. You can find it [here](https://github.com/fivethirtyeight/data/tree/master/airline-safety). You should always remember passing the \"raw\" file to the `read_csv`function. You can get the link by clicking on the `.csv` file and then on the \"raw\" button. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "air_safety= pd.read_csv(\"https://raw.githubusercontent.com/fivethirtyeight/data/master/airline-safety/airline-safety.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>airline</th>\n",
       "      <th>avail_seat_km_per_week</th>\n",
       "      <th>incidents_85_99</th>\n",
       "      <th>fatal_accidents_85_99</th>\n",
       "      <th>fatalities_85_99</th>\n",
       "      <th>incidents_00_14</th>\n",
       "      <th>fatal_accidents_00_14</th>\n",
       "      <th>fatalities_00_14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Aer Lingus</td>\n",
       "      <td>320906734</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aeroflot*</td>\n",
       "      <td>1197672318</td>\n",
       "      <td>76</td>\n",
       "      <td>14</td>\n",
       "      <td>128</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Aerolineas Argentinas</td>\n",
       "      <td>385803648</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Aeromexico*</td>\n",
       "      <td>596871813</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>64</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Air Canada</td>\n",
       "      <td>1865253802</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 airline  avail_seat_km_per_week  incidents_85_99  \\\n",
       "0             Aer Lingus               320906734                2   \n",
       "1              Aeroflot*              1197672318               76   \n",
       "2  Aerolineas Argentinas               385803648                6   \n",
       "3            Aeromexico*               596871813                3   \n",
       "4             Air Canada              1865253802                2   \n",
       "\n",
       "   fatal_accidents_85_99  fatalities_85_99  incidents_00_14  \\\n",
       "0                      0                 0                0   \n",
       "1                     14               128                6   \n",
       "2                      0                 0                1   \n",
       "3                      1                64                5   \n",
       "4                      0                 0                2   \n",
       "\n",
       "   fatal_accidents_00_14  fatalities_00_14  \n",
       "0                      0                 0  \n",
       "1                      1                88  \n",
       "2                      0                 0  \n",
       "3                      0                 0  \n",
       "4                      0                 0  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "air_safety[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0               Aer Lingus\n",
       "1                Aeroflot*\n",
       "2    Aerolineas Argentinas\n",
       "3              Aeromexico*\n",
       "4               Air Canada\n",
       "Name: airline, dtype: object"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Column access\n",
    "air_safety[\"airline\"][:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>airline</th>\n",
       "      <th>fatalities_85_99</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Aer Lingus</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aeroflot*</td>\n",
       "      <td>128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Aerolineas Argentinas</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Aeromexico*</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Air Canada</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 airline  fatalities_85_99\n",
       "0             Aer Lingus                 0\n",
       "1              Aeroflot*               128\n",
       "2  Aerolineas Argentinas                 0\n",
       "3            Aeromexico*                64\n",
       "4             Air Canada                 0"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Two columns at once\n",
    "air_safety[[\"airline\", \"fatalities_85_99\"]][:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(air_safety)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to access elements is via the `loc` (label-based) and  `iloc` (index-based) functions. In this dataset we do not have labels, so we will use the `iloc` function. However, the `loc` works exactly the same, except you will have to pass the label strings as arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>airline</th>\n",
       "      <th>incidents_85_99</th>\n",
       "      <th>fatal_accidents_85_99</th>\n",
       "      <th>fatalities_85_99</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aeroflot*</td>\n",
       "      <td>76</td>\n",
       "      <td>14</td>\n",
       "      <td>128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Aerolineas Argentinas</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Aeromexico*</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Air Canada</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Air France</td>\n",
       "      <td>14</td>\n",
       "      <td>4</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 airline  incidents_85_99  fatal_accidents_85_99  \\\n",
       "1              Aeroflot*               76                     14   \n",
       "2  Aerolineas Argentinas                6                      0   \n",
       "3            Aeromexico*                3                      1   \n",
       "4             Air Canada                2                      0   \n",
       "5             Air France               14                      4   \n",
       "\n",
       "   fatalities_85_99  \n",
       "1               128  \n",
       "2                 0  \n",
       "3                64  \n",
       "4                 0  \n",
       "5                79  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "air_safety.iloc[[1,2,3,4,5], [0,2,3,4]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise\n",
    "Avengers dataset [here](https://github.com/fivethirtyeight/data/tree/master/avengers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Pandas with the correct alias\n",
    "# Create a dictionary called parts_germany contain minimum of 5 provinces (Bundesländer), their capitals as well as their population\n",
    "# Create a dataframe df_parts_germany from you dictionary\n",
    "# Print df_parts_germany\n",
    "# Change the indexes to short labels e.g. \"Bayern\" -->\"BY\"\n",
    "# Import the avengers dataset from Github (link above) remember to use the raw format link\n",
    "# Select the first three columns\n",
    "# Select rows 130, 131 and 132\n",
    "# Use the iloc function to subset all rows but only the name, appearances and years since joining column\n",
    "# Use the iloc function to subset the first 5 rows as well as the Name and Gender column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analysing a DataFrame\n",
    "<a id='pd3'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we read a `.csv` file (with Pandas) into Python, Pandas automatically creates a _DataFrame_ object. There is a rich variety of methods within the library to explore a data frame, e.g. to clean the data. Some of them are for example:\n",
    "* `my_df`, displays the whole dataframe \"my_df\"\n",
    "* `my_df.head()`, displays the first five rows\n",
    "* `my_df.tail()`, displays the last five rows\n",
    "* `my_df.info()`, displays a summary of the objects in the dataframe\n",
    "* `my_df.shape`, gives the dimensions of the data frame\n",
    "* `my_df.describe()`, displays a statistical summary of the dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>51.000000</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>5.100000e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>7225.784314</td>\n",
       "      <td>3504.882353</td>\n",
       "      <td>6.405637e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>15991.025083</td>\n",
       "      <td>7805.411811</td>\n",
       "      <td>7.327258e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>434.000000</td>\n",
       "      <td>75.000000</td>\n",
       "      <td>5.776010e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1446.500000</td>\n",
       "      <td>592.000000</td>\n",
       "      <td>1.777414e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3082.000000</td>\n",
       "      <td>1482.000000</td>\n",
       "      <td>4.461153e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>6781.500000</td>\n",
       "      <td>3196.000000</td>\n",
       "      <td>7.340946e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>109008.000000</td>\n",
       "      <td>52070.000000</td>\n",
       "      <td>3.946159e+07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         individuals  family_members     state_pop\n",
       "count      51.000000       51.000000  5.100000e+01\n",
       "mean     7225.784314     3504.882353  6.405637e+06\n",
       "std     15991.025083     7805.411811  7.327258e+06\n",
       "min       434.000000       75.000000  5.776010e+05\n",
       "25%      1446.500000      592.000000  1.777414e+06\n",
       "50%      3082.000000     1482.000000  4.461153e+06\n",
       "75%      6781.500000     3196.000000  7.340946e+06\n",
       "max    109008.000000    52070.000000  3.946159e+07"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "homeless_data= pd.read_csv(\"/Users/niklaswalter/Desktop/DSA/homelessness.csv\", index_col = 0)\n",
    "\n",
    "#homeless_data.head()\n",
    "#homeless_data.info()\n",
    "#homeless_data.shape\n",
    "homeless_data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sorting  data\n",
    "<a id='pd4'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First of all we will have a look on how to sort our data frame by using the `sort_values` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['region', 'state', 'individuals', 'family_members', 'state_pop'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "col = homeless_data.columns\n",
    "print(col)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Washington</td>\n",
       "      <td>16424</td>\n",
       "      <td>5880</td>\n",
       "      <td>7523869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>West South Central</td>\n",
       "      <td>Texas</td>\n",
       "      <td>19199</td>\n",
       "      <td>6111</td>\n",
       "      <td>28628666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>South Atlantic</td>\n",
       "      <td>Florida</td>\n",
       "      <td>21443</td>\n",
       "      <td>9587</td>\n",
       "      <td>21244317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Mid-Atlantic</td>\n",
       "      <td>New York</td>\n",
       "      <td>39827</td>\n",
       "      <td>52070</td>\n",
       "      <td>19530351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                region       state  individuals  family_members  state_pop\n",
       "47             Pacific  Washington        16424            5880    7523869\n",
       "43  West South Central       Texas        19199            6111   28628666\n",
       "9       South Atlantic     Florida        21443            9587   21244317\n",
       "32        Mid-Atlantic    New York        39827           52070   19530351\n",
       "4              Pacific  California       109008           20964   39461588"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "homeless_data_sort = homeless_data.sort_values(\"individuals\")\n",
    "homeless_data_sort.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Mid-Atlantic</td>\n",
       "      <td>New York</td>\n",
       "      <td>39827</td>\n",
       "      <td>52070</td>\n",
       "      <td>19530351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>New England</td>\n",
       "      <td>Massachusetts</td>\n",
       "      <td>6811</td>\n",
       "      <td>13257</td>\n",
       "      <td>6882635</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>South Atlantic</td>\n",
       "      <td>Florida</td>\n",
       "      <td>21443</td>\n",
       "      <td>9587</td>\n",
       "      <td>21244317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>West South Central</td>\n",
       "      <td>Texas</td>\n",
       "      <td>19199</td>\n",
       "      <td>6111</td>\n",
       "      <td>28628666</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                region          state  individuals  family_members  state_pop\n",
       "32        Mid-Atlantic       New York        39827           52070   19530351\n",
       "4              Pacific     California       109008           20964   39461588\n",
       "21         New England  Massachusetts         6811           13257    6882635\n",
       "9       South Atlantic        Florida        21443            9587   21244317\n",
       "43  West South Central          Texas        19199            6111   28628666"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "homelessness_fam = homeless_data.sort_values(\"family_members\", ascending= False)\n",
    "homelessness_fam.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>East North Central</td>\n",
       "      <td>Illinois</td>\n",
       "      <td>6752</td>\n",
       "      <td>3891</td>\n",
       "      <td>12723071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>East North Central</td>\n",
       "      <td>Ohio</td>\n",
       "      <td>6929</td>\n",
       "      <td>3320</td>\n",
       "      <td>11676341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>East North Central</td>\n",
       "      <td>Michigan</td>\n",
       "      <td>5209</td>\n",
       "      <td>3142</td>\n",
       "      <td>9984072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>East North Central</td>\n",
       "      <td>Wisconsin</td>\n",
       "      <td>2740</td>\n",
       "      <td>2167</td>\n",
       "      <td>5807406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>East North Central</td>\n",
       "      <td>Indiana</td>\n",
       "      <td>3776</td>\n",
       "      <td>1482</td>\n",
       "      <td>6695497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Tennessee</td>\n",
       "      <td>6139</td>\n",
       "      <td>1744</td>\n",
       "      <td>6771631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>2735</td>\n",
       "      <td>953</td>\n",
       "      <td>4461153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>2570</td>\n",
       "      <td>864</td>\n",
       "      <td>4887681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Mississippi</td>\n",
       "      <td>1024</td>\n",
       "      <td>328</td>\n",
       "      <td>2981020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Mid-Atlantic</td>\n",
       "      <td>New York</td>\n",
       "      <td>39827</td>\n",
       "      <td>52070</td>\n",
       "      <td>19530351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Mid-Atlantic</td>\n",
       "      <td>Pennsylvania</td>\n",
       "      <td>8163</td>\n",
       "      <td>5349</td>\n",
       "      <td>12800922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Mid-Atlantic</td>\n",
       "      <td>New Jersey</td>\n",
       "      <td>6048</td>\n",
       "      <td>3350</td>\n",
       "      <td>8886025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Colorado</td>\n",
       "      <td>7607</td>\n",
       "      <td>3250</td>\n",
       "      <td>5691287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Arizona</td>\n",
       "      <td>7259</td>\n",
       "      <td>2606</td>\n",
       "      <td>7158024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Utah</td>\n",
       "      <td>1904</td>\n",
       "      <td>972</td>\n",
       "      <td>3153550</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                region         state  individuals  family_members  state_pop\n",
       "13  East North Central      Illinois         6752            3891   12723071\n",
       "35  East North Central          Ohio         6929            3320   11676341\n",
       "22  East North Central      Michigan         5209            3142    9984072\n",
       "49  East North Central     Wisconsin         2740            2167    5807406\n",
       "14  East North Central       Indiana         3776            1482    6695497\n",
       "42  East South Central     Tennessee         6139            1744    6771631\n",
       "17  East South Central      Kentucky         2735             953    4461153\n",
       "0   East South Central       Alabama         2570             864    4887681\n",
       "24  East South Central   Mississippi         1024             328    2981020\n",
       "32        Mid-Atlantic      New York        39827           52070   19530351\n",
       "38        Mid-Atlantic  Pennsylvania         8163            5349   12800922\n",
       "30        Mid-Atlantic    New Jersey         6048            3350    8886025\n",
       "5             Mountain      Colorado         7607            3250    5691287\n",
       "2             Mountain       Arizona         7259            2606    7158024\n",
       "44            Mountain          Utah         1904             972    3153550"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "homelessness_reg_fam = homeless_data.sort_values([\"region\", \"family_members\"], ascending= [True, False])\n",
    "homelessness_reg_fam[:15]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the Walmart Sales data set\n",
    "# Explore the data frame with the fucntions you learned previously\n",
    "# Get familiar with the column names of the data frame\n",
    "# Sort the weekly sales column ascending and store the result to a variable to print out\n",
    "# Sort the fuel price column descending and store the result to a variable to print out\n",
    "# Sort weekly sales ascending, fuel price descending and unemployment ascending"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subsetting of a DataFrame\n",
    "<a id='pd5'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often we want to extract certain columns. This is possible by calling `my_df[\"COLUMN_I_LIKE\"]` or if you want multiple columns`my_df[[\"COLUMN_I_WANT\", ..., \"Last_Column\"]]`, which is basically just a list of column names. Remember if you want to extract multiple columns you will need extra `[]` for the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      2570\n",
       "1      1434\n",
       "2      7259\n",
       "3      2280\n",
       "4    109008\n",
       "Name: individuals, dtype: int64"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "individuals = homeless_data[\"individuals\"]\n",
    "#individuals = pd.DataFrame(individuals)\n",
    "individuals.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2570</td>\n",
       "      <td>864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1434</td>\n",
       "      <td>582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7259</td>\n",
       "      <td>2606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2280</td>\n",
       "      <td>432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   individuals  family_members\n",
       "0         2570             864\n",
       "1         1434             582\n",
       "2         7259            2606\n",
       "3         2280             432\n",
       "4       109008           20964"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "state_fam = homeless_data[[\"individuals\", \"family_members\"]]\n",
    "state_fam.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A more interesting way to filter columns out of a data frame is by creating `True`of `False` logical conditions. Pass that statement into square brackets.\n",
    "This is in my opinion one of the **most important** ways to subset data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>South Atlantic</td>\n",
       "      <td>Florida</td>\n",
       "      <td>21443</td>\n",
       "      <td>9587</td>\n",
       "      <td>21244317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Mid-Atlantic</td>\n",
       "      <td>New York</td>\n",
       "      <td>39827</td>\n",
       "      <td>52070</td>\n",
       "      <td>19530351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Oregon</td>\n",
       "      <td>11139</td>\n",
       "      <td>3337</td>\n",
       "      <td>4181886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>West South Central</td>\n",
       "      <td>Texas</td>\n",
       "      <td>19199</td>\n",
       "      <td>6111</td>\n",
       "      <td>28628666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Washington</td>\n",
       "      <td>16424</td>\n",
       "      <td>5880</td>\n",
       "      <td>7523869</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                region       state  individuals  family_members  state_pop\n",
       "4              Pacific  California       109008           20964   39461588\n",
       "9       South Atlantic     Florida        21443            9587   21244317\n",
       "32        Mid-Atlantic    New York        39827           52070   19530351\n",
       "37             Pacific      Oregon        11139            3337    4181886\n",
       "43  West South Central       Texas        19199            6111   28628666\n",
       "47             Pacific  Washington        16424            5880    7523869"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ind_gt_9k = homeless_data[homeless_data[\"individuals\"] > 9000]\n",
    "ind_gt_9k"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "fam_lt_1k_mnt = homeless_data[(homeless_data[\"family_members\"] < 1000) & (homeless_data[\"region\"] == \"Pacific\")]\n",
    "fam_lt_1k_or_mnt = homeless_data[(homeless_data[\"family_members\"] < 1000) | (homeless_data[\"region\"] == \"Pacific\")]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1434</td>\n",
       "      <td>582</td>\n",
       "      <td>735139</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    region   state  individuals  family_members  state_pop\n",
       "1  Pacific  Alaska         1434             582     735139"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fam_lt_1k_mnt.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>2570</td>\n",
       "      <td>864</td>\n",
       "      <td>4887681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1434</td>\n",
       "      <td>582</td>\n",
       "      <td>735139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>West South Central</td>\n",
       "      <td>Arkansas</td>\n",
       "      <td>2280</td>\n",
       "      <td>432</td>\n",
       "      <td>3009733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>South Atlantic</td>\n",
       "      <td>Delaware</td>\n",
       "      <td>708</td>\n",
       "      <td>374</td>\n",
       "      <td>965479</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               region       state  individuals  family_members  state_pop\n",
       "0  East South Central     Alabama         2570             864    4887681\n",
       "1             Pacific      Alaska         1434             582     735139\n",
       "3  West South Central    Arkansas         2280             432    3009733\n",
       "4             Pacific  California       109008           20964   39461588\n",
       "7      South Atlantic    Delaware          708             374     965479"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fam_lt_1k_or_mnt.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Arizona</td>\n",
       "      <td>7259</td>\n",
       "      <td>2606</td>\n",
       "      <td>7158024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Nevada</td>\n",
       "      <td>7058</td>\n",
       "      <td>486</td>\n",
       "      <td>3027341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Utah</td>\n",
       "      <td>1904</td>\n",
       "      <td>972</td>\n",
       "      <td>3153550</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      region       state  individuals  family_members  state_pop\n",
       "2   Mountain     Arizona         7259            2606    7158024\n",
       "4    Pacific  California       109008           20964   39461588\n",
       "28  Mountain      Nevada         7058             486    3027341\n",
       "44  Mountain        Utah         1904             972    3153550"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mojave_states = [\"California\", \"Arizona\", \"Nevada\", \"Utah\"]\n",
    "mojave_homeless = homeless_data[homeless_data[\"state\"].isin(mojave_states)]\n",
    "mojave_homeless"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Extract the weekly sales column, assign it to a variable and print it out\n",
    "# Extract the unemployment, type column, assign it to a variable and print it out\n",
    "# Extract the store, date column, assign it to a variable and print it out\n",
    "# Extract only stores of type \"B\", assign it to a variable and print it out\n",
    "# Extract only stores with weekly sales higher than 20.000, assign it to a variable and print it out\n",
    "# Extract only stores were the temperature was bigger than 25 degrees and unemployment higher than 7, assign it to a variable and print it out\n",
    "# Extract only stores were weekly sales are over 30.000 or unemployment is under 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating new columns\n",
    "<a id='pd6'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes it is also quite nice to add new columns e.g temperature also in °F or °C or price of a product in a different currency. Let's have a look how we can do this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "      <th>total</th>\n",
       "      <th>p_individuals</th>\n",
       "      <th>individuals_per_10k</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>2570</td>\n",
       "      <td>864</td>\n",
       "      <td>4887681</td>\n",
       "      <td>3434</td>\n",
       "      <td>0.748398</td>\n",
       "      <td>5.258117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1434</td>\n",
       "      <td>582</td>\n",
       "      <td>735139</td>\n",
       "      <td>2016</td>\n",
       "      <td>0.711310</td>\n",
       "      <td>19.506515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Arizona</td>\n",
       "      <td>7259</td>\n",
       "      <td>2606</td>\n",
       "      <td>7158024</td>\n",
       "      <td>9865</td>\n",
       "      <td>0.735834</td>\n",
       "      <td>10.141067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>West South Central</td>\n",
       "      <td>Arkansas</td>\n",
       "      <td>2280</td>\n",
       "      <td>432</td>\n",
       "      <td>3009733</td>\n",
       "      <td>2712</td>\n",
       "      <td>0.840708</td>\n",
       "      <td>7.575423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "      <td>129972</td>\n",
       "      <td>0.838704</td>\n",
       "      <td>27.623825</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               region       state  individuals  family_members  state_pop  \\\n",
       "0  East South Central     Alabama         2570             864    4887681   \n",
       "1             Pacific      Alaska         1434             582     735139   \n",
       "2            Mountain     Arizona         7259            2606    7158024   \n",
       "3  West South Central    Arkansas         2280             432    3009733   \n",
       "4             Pacific  California       109008           20964   39461588   \n",
       "\n",
       "    total  p_individuals  individuals_per_10k  \n",
       "0    3434       0.748398             5.258117  \n",
       "1    2016       0.711310            19.506515  \n",
       "2    9865       0.735834            10.141067  \n",
       "3    2712       0.840708             7.575423  \n",
       "4  129972       0.838704            27.623825  "
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Adding a new column\n",
    "homeless_data[\"total\"]= homeless_data[\"individuals\"] + homeless_data[\"family_members\"]\n",
    "# Adding a p_individuals col as proportion of individuals\n",
    "homeless_data[\"p_individuals\"]=  homeless_data[\"individuals\"]/homeless_data[\"total\"]\n",
    "homeless_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>individuals_per_10k</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>District of Columbia</td>\n",
       "      <td>53.738381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Hawaii</td>\n",
       "      <td>29.079406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>27.623825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>Oregon</td>\n",
       "      <td>26.636307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Nevada</td>\n",
       "      <td>23.314189</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   state  individuals_per_10k\n",
       "8   District of Columbia            53.738381\n",
       "11                Hawaii            29.079406\n",
       "4             California            27.623825\n",
       "37                Oregon            26.636307\n",
       "28                Nevada            23.314189"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Adding individuals_per_10k  as homeless individuals per 10k state pop\n",
    "homeless_data[\"individuals_per_10k\"] = 10000 * homeless_data[\"individuals\"] / homeless_data[\"state_pop\"]\n",
    "\n",
    "# Subsetting for rows with individuals_per_10k greater than 15\n",
    "high_homelessness = homeless_data[homeless_data[\"individuals_per_10k\"] > 15]\n",
    "\n",
    "# Sorting high_homelessness individuals_per_10k, descending\n",
    "high_homelessness_srt = high_homelessness.sort_values(\"individuals_per_10k\", ascending=False)\n",
    "\n",
    "# From high_homelessness_srt, select the state and indiv_per_10k cols\n",
    "result = high_homelessness_srt[[\"state\", \"individuals_per_10k\"]]\n",
    "\n",
    "result.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Create a new column that contains the fuel price per l in Euro (ca. *0.84), assign it to a variable\n",
    "# Subset for only holiday days in the stores and fuel price per l in Euro greater than 0.60, assign it to a variable\n",
    "# Sort the data frame ascending from the last task, assign it to a variable\n",
    "# Extract sales and fuel price per Euro, assign it to result\n",
    "# Print (head of) result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary statistics\n",
    "<a id='pd6'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Summary statistics summarize many numbers in one statistic. For example, mean, median, minimum, maximum, and standard deviation are summary statistics. \n",
    "Summary statistics help us understand the data better. Useful functions are e.g:\n",
    "\n",
    "* `mean()`\n",
    "* `median()`\n",
    "* `max()`\n",
    "* `min()`\n",
    "* `agg(self_defined_functions)\n",
    "* `agg(Numpy_functions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average of individuals:  7225.78431372549\n",
      "Average of individuals:  3082.0\n",
      "Maximum of individuals:  109008\n",
      "Minimum of individuals:  434\n"
     ]
    }
   ],
   "source": [
    "print(\"Average of individuals: \" , homeless_data[\"individuals\"].mean())\n",
    "print(\"Average of individuals: \" ,homeless_data[\"individuals\"].median())\n",
    "print(\"Maximum of individuals: \" ,homeless_data[\"individuals\"].max())\n",
    "print(\"Minimum of individuals: \" ,homeless_data[\"individuals\"].min())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------------------------------------------------------------------\n",
      "Interquantile Range of individuals:\n",
      "------------------------------------------------------------------\n",
      "5335.0\n",
      "------------------------------------------------------------------\n",
      "Interquantile Range of individuals, family members and state pop.:\n",
      "------------------------------------------------------------------\n",
      "individuals          5335.0\n",
      "family_members       2604.0\n",
      "state_pop         5563533.0\n",
      "dtype: float64\n",
      "------------------------------------------------------------------\n",
      "IQR and median of individuals, family members and state pop.:\n",
      "------------------------------------------------------------------\n",
      "        individuals  family_members  state_pop\n",
      "iqr          5335.0          2604.0  5563533.0\n",
      "median       3082.0          1482.0  4461153.0\n"
     ]
    }
   ],
   "source": [
    "# Defining a self-made IQR function\n",
    "def iqr(column):\n",
    "    return column.quantile(0.75) - column.quantile(0.25)\n",
    "\n",
    "print(\"------------------------------------------------------------------\")\n",
    "print(\"Interquantile Range of individuals:\")\n",
    "print(\"------------------------------------------------------------------\")\n",
    "print(homeless_data[\"individuals\"].agg(iqr))\n",
    "print(\"------------------------------------------------------------------\")\n",
    "print(\"Interquantile Range of individuals, family members and state pop.:\")\n",
    "print(\"------------------------------------------------------------------\")\n",
    "print(homeless_data[[\"individuals\", \"family_members\", \"state_pop\"]].agg(iqr))\n",
    "print(\"------------------------------------------------------------------\")\n",
    "print(\"IQR and median of individuals, family members and state pop.:\")\n",
    "print(\"------------------------------------------------------------------\")\n",
    "print(homeless_data[[\"individuals\", \"family_members\", \"state_pop\"]].agg([iqr, np.median]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Print the mean of weekly_sales\n",
    "# Print the median of weekly_sales\n",
    "# Print the maximum of date \n",
    "# Print the minimum the date\n",
    "# Def your own function to calculate the range (maximum - minimum) of the weekly sales column\n",
    "# Calculate the range of temperature_c, fuel_price_usd_per_l and unemployment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cumulative statistics\n",
    "<a id='pd7'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cumulative statistics can be helpful in order to track summary statistics over time. Some basic functions are: \n",
    "* `cummax()`\n",
    "* `cummin()`\n",
    "* `cumprod()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>individuals_cumsum</th>\n",
       "      <th>individuals_cummax</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>2570</td>\n",
       "      <td>2570</td>\n",
       "      <td>2570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>1434</td>\n",
       "      <td>4004</td>\n",
       "      <td>2570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>7259</td>\n",
       "      <td>11263</td>\n",
       "      <td>7259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>2280</td>\n",
       "      <td>13543</td>\n",
       "      <td>7259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>122551</td>\n",
       "      <td>109008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        state  individuals  individuals_cumsum  individuals_cummax\n",
       "0     Alabama         2570                2570                2570\n",
       "1      Alaska         1434                4004                2570\n",
       "2     Arizona         7259               11263                7259\n",
       "3    Arkansas         2280               13543                7259\n",
       "4  California       109008              122551              109008"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sort homeless_data by state\n",
    "homeless_by_state = homeless_data.sort_values(\"state\")\n",
    "\n",
    "# Cumulative sum of individuals\n",
    "homeless_by_state[\"individuals_cumsum\"] = homeless_by_state[\"individuals\"].cumsum()\n",
    "\n",
    "# Cumulative max of individuals\n",
    "homeless_by_state[\"individuals_cummax\"] = homeless_by_state[\"individuals\"].cummax()\n",
    "\n",
    "# Columns calculated\n",
    "homeless_by_state[[\"state\", \"individuals\", \"individuals_cumsum\",\"individuals_cummax\"]].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Sort sales by date\n",
    "# Get the cumulative sum of weekly_sales and add as cum_weekly_sales col\n",
    "# Get the cumulative max of weekly_sales, add as cum_max_sales col\n",
    "# Print the columns you calculated"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dropping duplicates\n",
    "<a id='pd8'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is an essential skill to remove duplicates get accurate counts because often, you don't want to count the same thing multiple times. \n",
    "Useful function in this case is `drop_duplicates(subset= ... )."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "      <th>individuals</th>\n",
       "      <th>family_members</th>\n",
       "      <th>state_pop</th>\n",
       "      <th>total</th>\n",
       "      <th>p_individuals</th>\n",
       "      <th>individuals_per_10k</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>East South Central</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>2570</td>\n",
       "      <td>864</td>\n",
       "      <td>4887681</td>\n",
       "      <td>3434</td>\n",
       "      <td>0.748398</td>\n",
       "      <td>5.258117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1434</td>\n",
       "      <td>582</td>\n",
       "      <td>735139</td>\n",
       "      <td>2016</td>\n",
       "      <td>0.711310</td>\n",
       "      <td>19.506515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Arizona</td>\n",
       "      <td>7259</td>\n",
       "      <td>2606</td>\n",
       "      <td>7158024</td>\n",
       "      <td>9865</td>\n",
       "      <td>0.735834</td>\n",
       "      <td>10.141067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>West South Central</td>\n",
       "      <td>Arkansas</td>\n",
       "      <td>2280</td>\n",
       "      <td>432</td>\n",
       "      <td>3009733</td>\n",
       "      <td>2712</td>\n",
       "      <td>0.840708</td>\n",
       "      <td>7.575423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pacific</td>\n",
       "      <td>California</td>\n",
       "      <td>109008</td>\n",
       "      <td>20964</td>\n",
       "      <td>39461588</td>\n",
       "      <td>129972</td>\n",
       "      <td>0.838704</td>\n",
       "      <td>27.623825</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               region       state  individuals  family_members  state_pop  \\\n",
       "0  East South Central     Alabama         2570             864    4887681   \n",
       "1             Pacific      Alaska         1434             582     735139   \n",
       "2            Mountain     Arizona         7259            2606    7158024   \n",
       "3  West South Central    Arkansas         2280             432    3009733   \n",
       "4             Pacific  California       109008           20964   39461588   \n",
       "\n",
       "    total  p_individuals  individuals_per_10k  \n",
       "0    3434       0.748398             5.258117  \n",
       "1    2016       0.711310            19.506515  \n",
       "2    9865       0.735834            10.141067  \n",
       "3    2712       0.840708             7.575423  \n",
       "4  129972       0.838704            27.623825  "
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "homeless_regions = homeless_data.drop_duplicates(subset = [\"region\", \"state_pop\"])\n",
    "homeless_regions.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>state</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Arizona</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Colorado</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Idaho</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Montana</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Nevada</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>New Mexico</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Utah</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>Mountain</td>\n",
       "      <td>Wyoming</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      region       state\n",
       "2   Mountain     Arizona\n",
       "5   Mountain    Colorado\n",
       "12  Mountain       Idaho\n",
       "26  Mountain     Montana\n",
       "28  Mountain      Nevada\n",
       "31  Mountain  New Mexico\n",
       "44  Mountain        Utah\n",
       "50  Mountain     Wyoming"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Subset the rows where region is Mountain and drop all duplicate states\n",
    "homeless_mnt_regions = homeless_data[homeless_data[\"region\"] == \"Mountain\"].drop_duplicates(subset = \"state\")\n",
    "homeless_mnt_regions[[\"region\", \"state\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Drop duplicate store/type combinations\n",
    "# Drop duplicate store/department combinations\n",
    "# Subset the rows where is_holiday is True and drop duplicate dates, assign it to a variable\n",
    "# Print date col your new column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counting\n",
    "<a id='pd9'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes we can get a nice overview of our dataset by counting e.g. the number of product types, regions or female, male individuals.\n",
    "Counting most of the time only makes sense when you have categorical data.\n",
    "So how can we count? Of course you could go through your data set, but let's be honest we are not living in the stone-age.\n",
    "We can use the following function:\n",
    "* `value_counts(sort = True/False, normalize = True/False)`\n",
    "If we set normalize to true the function will return. the proportion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------------------------------------------\n",
      "Total amount of the regions:\n",
      "------------------------------------------\n",
      "South Atlantic        9\n",
      "Mountain              8\n",
      "West North Central    7\n",
      "New England           6\n",
      "East North Central    5\n",
      "Pacific               5\n",
      "East South Central    4\n",
      "West South Central    4\n",
      "Mid-Atlantic          3\n",
      "Name: region, dtype: int64\n",
      "------------------------------------------\n",
      "Proportional amount of the regions:\n",
      "------------------------------------------\n",
      "South Atlantic        0.176471\n",
      "Mountain              0.156863\n",
      "West North Central    0.137255\n",
      "New England           0.117647\n",
      "East North Central    0.098039\n",
      "Pacific               0.098039\n",
      "East South Central    0.078431\n",
      "West South Central    0.078431\n",
      "Mid-Atlantic          0.058824\n",
      "Name: region, dtype: float64\n",
      "------------------------------------------\n",
      "Sorted total amount of the regions:\n",
      "------------------------------------------\n",
      "South Atlantic        9\n",
      "Mountain              8\n",
      "West North Central    7\n",
      "New England           6\n",
      "East North Central    5\n",
      "Pacific               5\n",
      "East South Central    4\n",
      "West South Central    4\n",
      "Mid-Atlantic          3\n",
      "Name: region, dtype: int64\n",
      "------------------------------------------\n",
      "Sorted proportional amount of the regions:\n",
      "------------------------------------------\n",
      "South Atlantic        0.176471\n",
      "Mountain              0.156863\n",
      "West North Central    0.137255\n",
      "New England           0.117647\n",
      "East North Central    0.098039\n",
      "Pacific               0.098039\n",
      "East South Central    0.078431\n",
      "West South Central    0.078431\n",
      "Mid-Atlantic          0.058824\n",
      "Name: region, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# Count the number of regions \n",
    "print(\"------------------------------------------\")\n",
    "print(\"Total amount of the regions:\")\n",
    "print(\"------------------------------------------\")\n",
    "region_counts = homeless_data[\"region\"].value_counts()\n",
    "print(region_counts)\n",
    "\n",
    "# Get the proportion regions \n",
    "print(\"------------------------------------------\")\n",
    "print(\"Proportional amount of the regions:\")\n",
    "print(\"------------------------------------------\")\n",
    "region_props = homeless_data[\"region\"].value_counts(normalize= True)\n",
    "print(region_props)\n",
    "\n",
    "# Count the number of each region and sort\n",
    "print(\"------------------------------------------\")\n",
    "print(\"Sorted total amount of the regions:\")\n",
    "print(\"------------------------------------------\")\n",
    "region_counts_sorted = homeless_data[\"region\"].value_counts(sort= True)\n",
    "print(region_counts_sorted)\n",
    "\n",
    "# Proportion of regions of and sort\n",
    "print(\"------------------------------------------\")\n",
    "print(\"Sorted proportional amount of the regions:\")\n",
    "print(\"------------------------------------------\")\n",
    "region_props_sorted = homeless_data[\"region\"].value_counts(sort=True, normalize=True)\n",
    "print(region_props_sorted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Count the number of stores of each store type in store_types\n",
    "# Count the proportion of stores of each store type in store_types.\n",
    "# Count the number of different departments in store_depts, sorting the counts in descending order.\n",
    "# Count the proportion of different departments in store_depts, sorting the proportions in descending order."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grouping\n",
    "<a id='pd10'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes we want to get summary statistics for each category of categorical variables. For example efficience for different types of shipping, advertising etc.\n",
    "We can either create groups manually like this: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.15523893, 0.77445649, 0.07030458])"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regions = [\"Mountain\", \"Pacific\", \"New England\"]\n",
    "# Subset for desired regions\n",
    "homeless_regions = homeless_data[homeless_data[\"region\"].isin(regions)]\n",
    "\n",
    "# Calculate total amount of homeless peope in these regions\n",
    "homeless_data_all =  homeless_regions[\"individuals\"].sum()\n",
    "\n",
    "# Subset for mountain region , calc total individuals\n",
    "mnt = homeless_data[homeless_data[\"region\"] == \"Mountain\"][\"individuals\"].sum()\n",
    "\n",
    "# Subset for pacific region, calc total individuals\n",
    "pac = homeless_data[homeless_data[\"region\"] == \"Pacific\"][\"individuals\"].sum()\n",
    "\n",
    "# Subset for New England region, calc total individuals\n",
    "new = homeless_data[homeless_data[\"region\"] == \"New England\"][\"individuals\"].sum()\n",
    "\n",
    "# Get proportion for each type\n",
    "homeless_prop_by_region = [mnt, pac, new] / homeless_data_all\n",
    "\n",
    "homeless_prop_by_region"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Calculate the total weekly_sales over the whole dataset.\n",
    "# Subset for type \"A\" stores, and calculate their total weekly sales.\n",
    "# Do the same for type \"B\" and type \"C\" stores.\n",
    "# Combine the A/B/C results into a list, and divide by sales_all to get the proportion of sales by type."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... or we can use the `groupby()` function which will save us a lot of time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "region\n",
       "East North Central    0.068942\n",
       "East South Central    0.033833\n",
       "Mid-Atlantic          0.146637\n",
       "Mountain              0.077313\n",
       "New England           0.035014\n",
       "Pacific               0.385699\n",
       "South Atlantic        0.141812\n",
       "West North Central    0.037912\n",
       "West South Central    0.072838\n",
       "Name: individuals, dtype: float64"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Group by region; calc total individuals\n",
    "homeless_data_by_region = homeless_data.groupby(\"region\")[\"individuals\"].sum()\n",
    "\n",
    "# Get proportion for each region\n",
    "homeless_data_by_region_prop = homeless_data_by_region / sum(homeless_data[\"individuals\"])\n",
    "homeless_data_by_region_prop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Group sales by \"type\", take the sum of \"weekly_sales\", and store as sales_by_type.\n",
    "# Calculate the proportion of sales at each store type by dividing by the sum of sales_by_type. Assign to sales_propn_by_type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grouped summary statistics\n",
    "<a id='pd11'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can also calculate multiple grouped summaries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>amin</th>\n",
       "      <th>amax</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>East North Central</th>\n",
       "      <td>2740</td>\n",
       "      <td>6929</td>\n",
       "      <td>5081.200000</td>\n",
       "      <td>5209.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>East South Central</th>\n",
       "      <td>1024</td>\n",
       "      <td>6139</td>\n",
       "      <td>3117.000000</td>\n",
       "      <td>2652.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mid-Atlantic</th>\n",
       "      <td>6048</td>\n",
       "      <td>39827</td>\n",
       "      <td>18012.666667</td>\n",
       "      <td>8163.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mountain</th>\n",
       "      <td>434</td>\n",
       "      <td>7607</td>\n",
       "      <td>3561.375000</td>\n",
       "      <td>1926.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New England</th>\n",
       "      <td>747</td>\n",
       "      <td>6811</td>\n",
       "      <td>2150.500000</td>\n",
       "      <td>1142.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pacific</th>\n",
       "      <td>1434</td>\n",
       "      <td>109008</td>\n",
       "      <td>28427.200000</td>\n",
       "      <td>11139.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South Atlantic</th>\n",
       "      <td>708</td>\n",
       "      <td>21443</td>\n",
       "      <td>5806.666667</td>\n",
       "      <td>3928.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West North Central</th>\n",
       "      <td>467</td>\n",
       "      <td>3993</td>\n",
       "      <td>1995.857143</td>\n",
       "      <td>1711.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West South Central</th>\n",
       "      <td>2280</td>\n",
       "      <td>19199</td>\n",
       "      <td>6710.500000</td>\n",
       "      <td>2681.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    amin    amax          mean   median\n",
       "region                                                 \n",
       "East North Central  2740    6929   5081.200000   5209.0\n",
       "East South Central  1024    6139   3117.000000   2652.5\n",
       "Mid-Atlantic        6048   39827  18012.666667   8163.0\n",
       "Mountain             434    7607   3561.375000   1926.5\n",
       "New England          747    6811   2150.500000   1142.5\n",
       "Pacific             1434  109008  28427.200000  11139.0\n",
       "South Atlantic       708   21443   5806.666667   3928.0\n",
       "West North Central   467    3993   1995.857143   1711.0\n",
       "West South Central  2280   19199   6710.500000   2681.5"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For each region, aggregate individuals: get min, max, mean, and median\n",
    "homeless_stats = homeless_data.groupby(\"region\")[\"individuals\"].agg([np.min, np.max, np.mean, np.median])\n",
    "homeless_stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">individuals</th>\n",
       "      <th colspan=\"4\" halign=\"left\">family_members</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>amin</th>\n",
       "      <th>amax</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "      <th>amin</th>\n",
       "      <th>amax</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>East North Central</th>\n",
       "      <td>2740</td>\n",
       "      <td>6929</td>\n",
       "      <td>5081.200000</td>\n",
       "      <td>5209.0</td>\n",
       "      <td>1482</td>\n",
       "      <td>3891</td>\n",
       "      <td>2800.400000</td>\n",
       "      <td>3142.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>East South Central</th>\n",
       "      <td>1024</td>\n",
       "      <td>6139</td>\n",
       "      <td>3117.000000</td>\n",
       "      <td>2652.5</td>\n",
       "      <td>328</td>\n",
       "      <td>1744</td>\n",
       "      <td>972.250000</td>\n",
       "      <td>908.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mid-Atlantic</th>\n",
       "      <td>6048</td>\n",
       "      <td>39827</td>\n",
       "      <td>18012.666667</td>\n",
       "      <td>8163.0</td>\n",
       "      <td>3350</td>\n",
       "      <td>52070</td>\n",
       "      <td>20256.333333</td>\n",
       "      <td>5349.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mountain</th>\n",
       "      <td>434</td>\n",
       "      <td>7607</td>\n",
       "      <td>3561.375000</td>\n",
       "      <td>1926.5</td>\n",
       "      <td>205</td>\n",
       "      <td>3250</td>\n",
       "      <td>1157.250000</td>\n",
       "      <td>658.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New England</th>\n",
       "      <td>747</td>\n",
       "      <td>6811</td>\n",
       "      <td>2150.500000</td>\n",
       "      <td>1142.5</td>\n",
       "      <td>354</td>\n",
       "      <td>13257</td>\n",
       "      <td>2916.500000</td>\n",
       "      <td>840.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pacific</th>\n",
       "      <td>1434</td>\n",
       "      <td>109008</td>\n",
       "      <td>28427.200000</td>\n",
       "      <td>11139.0</td>\n",
       "      <td>582</td>\n",
       "      <td>20964</td>\n",
       "      <td>6632.400000</td>\n",
       "      <td>3337.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South Atlantic</th>\n",
       "      <td>708</td>\n",
       "      <td>21443</td>\n",
       "      <td>5806.666667</td>\n",
       "      <td>3928.0</td>\n",
       "      <td>222</td>\n",
       "      <td>9587</td>\n",
       "      <td>2646.444444</td>\n",
       "      <td>2230.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West North Central</th>\n",
       "      <td>467</td>\n",
       "      <td>3993</td>\n",
       "      <td>1995.857143</td>\n",
       "      <td>1711.0</td>\n",
       "      <td>75</td>\n",
       "      <td>3250</td>\n",
       "      <td>1177.428571</td>\n",
       "      <td>773.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West South Central</th>\n",
       "      <td>2280</td>\n",
       "      <td>19199</td>\n",
       "      <td>6710.500000</td>\n",
       "      <td>2681.5</td>\n",
       "      <td>432</td>\n",
       "      <td>6111</td>\n",
       "      <td>2027.500000</td>\n",
       "      <td>783.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   individuals                                family_members  \\\n",
       "                          amin    amax          mean   median           amin   \n",
       "region                                                                         \n",
       "East North Central        2740    6929   5081.200000   5209.0           1482   \n",
       "East South Central        1024    6139   3117.000000   2652.5            328   \n",
       "Mid-Atlantic              6048   39827  18012.666667   8163.0           3350   \n",
       "Mountain                   434    7607   3561.375000   1926.5            205   \n",
       "New England                747    6811   2150.500000   1142.5            354   \n",
       "Pacific                   1434  109008  28427.200000  11139.0            582   \n",
       "South Atlantic             708   21443   5806.666667   3928.0            222   \n",
       "West North Central         467    3993   1995.857143   1711.0             75   \n",
       "West South Central        2280   19199   6710.500000   2681.5            432   \n",
       "\n",
       "                                                 \n",
       "                     amax          mean  median  \n",
       "region                                           \n",
       "East North Central   3891   2800.400000  3142.0  \n",
       "East South Central   1744    972.250000   908.5  \n",
       "Mid-Atlantic        52070  20256.333333  5349.0  \n",
       "Mountain             3250   1157.250000   658.5  \n",
       "New England         13257   2916.500000   840.5  \n",
       "Pacific             20964   6632.400000  3337.0  \n",
       "South Atlantic       9587   2646.444444  2230.0  \n",
       "West North Central   3250   1177.428571   773.0  \n",
       "West South Central   6111   2027.500000   783.5  "
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ind_fam_stats = homeless_data.groupby(\"region\")[[\"individuals\", \"family_members\"]].agg([np.min, np.max, np.mean, np.median])\n",
    "ind_fam_stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Make sure you import numpy with the alias np.\n",
    "# Get the min, max, mean, and median of weekly_sales for each store type using .groupby() and .agg(). Store this as sales_stats. Make sure to use numpy functions!\n",
    "# Get the min, max, mean, and median of unemployment and fuel_price_usd_per_l for each store type. Store this as unemp_fuel_stats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pivot tables\n",
    "<a id='pd12'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pivot tables are the standard way of aggregating data in Excel or other spreadsheet programms. In pandas, pivot tables are essentially just another way of performing grouped calculations. That is, the `.pivot_table()` method is just an alternative to `.groupby()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>individuals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>East North Central</th>\n",
       "      <td>5081.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>East South Central</th>\n",
       "      <td>3117.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mid-Atlantic</th>\n",
       "      <td>18012.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mountain</th>\n",
       "      <td>3561.375000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New England</th>\n",
       "      <td>2150.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pacific</th>\n",
       "      <td>28427.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South Atlantic</th>\n",
       "      <td>5806.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West North Central</th>\n",
       "      <td>1995.857143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West South Central</th>\n",
       "      <td>6710.500000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     individuals\n",
       "region                          \n",
       "East North Central   5081.200000\n",
       "East South Central   3117.000000\n",
       "Mid-Atlantic        18012.666667\n",
       "Mountain             3561.375000\n",
       "New England          2150.500000\n",
       "Pacific             28427.200000\n",
       "South Atlantic       5806.666667\n",
       "West North Central   1995.857143\n",
       "West South Central   6710.500000"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pivot for mean indivuals for each region\n",
    "mean_homeless_by_region = homeless_data.pivot_table(values= \"individuals\", index= \"region\")\n",
    "mean_homeless_by_region"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>individuals</th>\n",
       "      <th>False</th>\n",
       "      <th>True</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>East North Central</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5081.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>East South Central</th>\n",
       "      <td>NaN</td>\n",
       "      <td>3117.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mid-Atlantic</th>\n",
       "      <td>NaN</td>\n",
       "      <td>18012.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mountain</th>\n",
       "      <td>708.500000</td>\n",
       "      <td>4512.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New England</th>\n",
       "      <td>787.333333</td>\n",
       "      <td>3513.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pacific</th>\n",
       "      <td>NaN</td>\n",
       "      <td>28427.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South Atlantic</th>\n",
       "      <td>708.000000</td>\n",
       "      <td>6444.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West North Central</th>\n",
       "      <td>651.500000</td>\n",
       "      <td>2533.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West South Central</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6710.500000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "individuals              False         True \n",
       "region                                      \n",
       "East North Central         NaN   5081.200000\n",
       "East South Central         NaN   3117.000000\n",
       "Mid-Atlantic               NaN  18012.666667\n",
       "Mountain            708.500000   4512.333333\n",
       "New England         787.333333   3513.666667\n",
       "Pacific                    NaN  28427.200000\n",
       "South Atlantic      708.000000   6444.000000\n",
       "West North Central  651.500000   2533.600000\n",
       "West South Central         NaN   6710.500000"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pivot for mean_homeless_by_type_indover1k\n",
    "mean_homeless_by_type_indover1k = homeless_data.pivot_table(values = \"individuals\", index = \"region\", columns = homeless_data[\"individuals\"] > 1000)\n",
    "mean_homeless_by_type_indover1k"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the Walmart sales dataset\n",
    "# Make sure you import numpy with the alias np.\n",
    "# Get the mean weekly_sales by type using .pivot_table() and store as mean_sales_by_type.\n",
    "# Get the mean and median (using NumPy functions) of weekly_sales by type using .pivot_table() and store as mean_med_sales_by_type.\n",
    "# Get the mean of weekly_sales by type and is_holiday using .pivot_table() and store as mean_sales_by_type_holiday."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `.pivot_table()` method has several useful arguments, including `fill_value` (replaces missing values with a real value )and `margins` (gives the row and column totals of the pivot table contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "region                East North Central  East South Central  Mid-Atlantic  \\\n",
      "state                                                                        \n",
      "Alabama                                0                2570             0   \n",
      "Alaska                                 0                   0             0   \n",
      "Arizona                                0                   0             0   \n",
      "Arkansas                               0                   0             0   \n",
      "California                             0                   0             0   \n",
      "Colorado                               0                   0             0   \n",
      "Connecticut                            0                   0             0   \n",
      "Delaware                               0                   0             0   \n",
      "District of Columbia                   0                   0             0   \n",
      "Florida                                0                   0             0   \n",
      "Georgia                                0                   0             0   \n",
      "Hawaii                                 0                   0             0   \n",
      "Idaho                                  0                   0             0   \n",
      "Illinois                            6752                   0             0   \n",
      "Indiana                             3776                   0             0   \n",
      "Iowa                                   0                   0             0   \n",
      "Kansas                                 0                   0             0   \n",
      "Kentucky                               0                2735             0   \n",
      "Louisiana                              0                   0             0   \n",
      "Maine                                  0                   0             0   \n",
      "Maryland                               0                   0             0   \n",
      "Massachusetts                          0                   0             0   \n",
      "Michigan                            5209                   0             0   \n",
      "Minnesota                              0                   0             0   \n",
      "Mississippi                            0                1024             0   \n",
      "Missouri                               0                   0             0   \n",
      "Montana                                0                   0             0   \n",
      "Nebraska                               0                   0             0   \n",
      "Nevada                                 0                   0             0   \n",
      "New Hampshire                          0                   0             0   \n",
      "New Jersey                             0                   0          6048   \n",
      "New Mexico                             0                   0             0   \n",
      "New York                               0                   0         39827   \n",
      "North Carolina                         0                   0             0   \n",
      "North Dakota                           0                   0             0   \n",
      "Ohio                                6929                   0             0   \n",
      "Oklahoma                               0                   0             0   \n",
      "Oregon                                 0                   0             0   \n",
      "Pennsylvania                           0                   0          8163   \n",
      "Rhode Island                           0                   0             0   \n",
      "South Carolina                         0                   0             0   \n",
      "South Dakota                           0                   0             0   \n",
      "Tennessee                              0                6139             0   \n",
      "Texas                                  0                   0             0   \n",
      "Utah                                   0                   0             0   \n",
      "Vermont                                0                   0             0   \n",
      "Virginia                               0                   0             0   \n",
      "Washington                             0                   0             0   \n",
      "West Virginia                          0                   0             0   \n",
      "Wisconsin                           2740                   0             0   \n",
      "Wyoming                                0                   0             0   \n",
      "\n",
      "region                Mountain  New England  Pacific  South Atlantic  \\\n",
      "state                                                                  \n",
      "Alabama                      0            0        0               0   \n",
      "Alaska                       0            0     1434               0   \n",
      "Arizona                   7259            0        0               0   \n",
      "Arkansas                     0            0        0               0   \n",
      "California                   0            0   109008               0   \n",
      "Colorado                  7607            0        0               0   \n",
      "Connecticut                  0         2280        0               0   \n",
      "Delaware                     0            0        0             708   \n",
      "District of Columbia         0            0        0            3770   \n",
      "Florida                      0            0        0           21443   \n",
      "Georgia                      0            0        0            6943   \n",
      "Hawaii                       0            0     4131               0   \n",
      "Idaho                     1297            0        0               0   \n",
      "Illinois                     0            0        0               0   \n",
      "Indiana                      0            0        0               0   \n",
      "Iowa                         0            0        0               0   \n",
      "Kansas                       0            0        0               0   \n",
      "Kentucky                     0            0        0               0   \n",
      "Louisiana                    0            0        0               0   \n",
      "Maine                        0         1450        0               0   \n",
      "Maryland                     0            0        0            4914   \n",
      "Massachusetts                0         6811        0               0   \n",
      "Michigan                     0            0        0               0   \n",
      "Minnesota                    0            0        0               0   \n",
      "Mississippi                  0            0        0               0   \n",
      "Missouri                     0            0        0               0   \n",
      "Montana                    983            0        0               0   \n",
      "Nebraska                     0            0        0               0   \n",
      "Nevada                    7058            0        0               0   \n",
      "New Hampshire                0          835        0               0   \n",
      "New Jersey                   0            0        0               0   \n",
      "New Mexico                1949            0        0               0   \n",
      "New York                     0            0        0               0   \n",
      "North Carolina               0            0        0            6451   \n",
      "North Dakota                 0            0        0               0   \n",
      "Ohio                         0            0        0               0   \n",
      "Oklahoma                     0            0        0               0   \n",
      "Oregon                       0            0    11139               0   \n",
      "Pennsylvania                 0            0        0               0   \n",
      "Rhode Island                 0          747        0               0   \n",
      "South Carolina               0            0        0            3082   \n",
      "South Dakota                 0            0        0               0   \n",
      "Tennessee                    0            0        0               0   \n",
      "Texas                        0            0        0               0   \n",
      "Utah                      1904            0        0               0   \n",
      "Vermont                      0          780        0               0   \n",
      "Virginia                     0            0        0            3928   \n",
      "Washington                   0            0    16424               0   \n",
      "West Virginia                0            0        0            1021   \n",
      "Wisconsin                    0            0        0               0   \n",
      "Wyoming                    434            0        0               0   \n",
      "\n",
      "region                West North Central  West South Central  \n",
      "state                                                         \n",
      "Alabama                                0                   0  \n",
      "Alaska                                 0                   0  \n",
      "Arizona                                0                   0  \n",
      "Arkansas                               0                2280  \n",
      "California                             0                   0  \n",
      "Colorado                               0                   0  \n",
      "Connecticut                            0                   0  \n",
      "Delaware                               0                   0  \n",
      "District of Columbia                   0                   0  \n",
      "Florida                                0                   0  \n",
      "Georgia                                0                   0  \n",
      "Hawaii                                 0                   0  \n",
      "Idaho                                  0                   0  \n",
      "Illinois                               0                   0  \n",
      "Indiana                                0                   0  \n",
      "Iowa                                1711                   0  \n",
      "Kansas                              1443                   0  \n",
      "Kentucky                               0                   0  \n",
      "Louisiana                              0                2540  \n",
      "Maine                                  0                   0  \n",
      "Maryland                               0                   0  \n",
      "Massachusetts                          0                   0  \n",
      "Michigan                               0                   0  \n",
      "Minnesota                           3993                   0  \n",
      "Mississippi                            0                   0  \n",
      "Missouri                            3776                   0  \n",
      "Montana                                0                   0  \n",
      "Nebraska                            1745                   0  \n",
      "Nevada                                 0                   0  \n",
      "New Hampshire                          0                   0  \n",
      "New Jersey                             0                   0  \n",
      "New Mexico                             0                   0  \n",
      "New York                               0                   0  \n",
      "North Carolina                         0                   0  \n",
      "North Dakota                         467                   0  \n",
      "Ohio                                   0                   0  \n",
      "Oklahoma                               0                2823  \n",
      "Oregon                                 0                   0  \n",
      "Pennsylvania                           0                   0  \n",
      "Rhode Island                           0                   0  \n",
      "South Carolina                         0                   0  \n",
      "South Dakota                         836                   0  \n",
      "Tennessee                              0                   0  \n",
      "Texas                                  0               19199  \n",
      "Utah                                   0                   0  \n",
      "Vermont                                0                   0  \n",
      "Virginia                               0                   0  \n",
      "Washington                             0                   0  \n",
      "West Virginia                          0                   0  \n",
      "Wisconsin                              0                   0  \n",
      "Wyoming                                0                   0  \n",
      "region                East North Central  East South Central  Mid-Atlantic  \\\n",
      "state                                                                        \n",
      "Alabama                              0.0                2570      0.000000   \n",
      "Alaska                               0.0                   0      0.000000   \n",
      "Arizona                              0.0                   0      0.000000   \n",
      "Arkansas                             0.0                   0      0.000000   \n",
      "California                           0.0                   0      0.000000   \n",
      "Colorado                             0.0                   0      0.000000   \n",
      "Connecticut                          0.0                   0      0.000000   \n",
      "Delaware                             0.0                   0      0.000000   \n",
      "District of Columbia                 0.0                   0      0.000000   \n",
      "Florida                              0.0                   0      0.000000   \n",
      "Georgia                              0.0                   0      0.000000   \n",
      "Hawaii                               0.0                   0      0.000000   \n",
      "Idaho                                0.0                   0      0.000000   \n",
      "Illinois                          6752.0                   0      0.000000   \n",
      "Indiana                           3776.0                   0      0.000000   \n",
      "Iowa                                 0.0                   0      0.000000   \n",
      "Kansas                               0.0                   0      0.000000   \n",
      "Kentucky                             0.0                2735      0.000000   \n",
      "Louisiana                            0.0                   0      0.000000   \n",
      "Maine                                0.0                   0      0.000000   \n",
      "Maryland                             0.0                   0      0.000000   \n",
      "Massachusetts                        0.0                   0      0.000000   \n",
      "Michigan                          5209.0                   0      0.000000   \n",
      "Minnesota                            0.0                   0      0.000000   \n",
      "Mississippi                          0.0                1024      0.000000   \n",
      "Missouri                             0.0                   0      0.000000   \n",
      "Montana                              0.0                   0      0.000000   \n",
      "Nebraska                             0.0                   0      0.000000   \n",
      "Nevada                               0.0                   0      0.000000   \n",
      "New Hampshire                        0.0                   0      0.000000   \n",
      "New Jersey                           0.0                   0   6048.000000   \n",
      "New Mexico                           0.0                   0      0.000000   \n",
      "New York                             0.0                   0  39827.000000   \n",
      "North Carolina                       0.0                   0      0.000000   \n",
      "North Dakota                         0.0                   0      0.000000   \n",
      "Ohio                              6929.0                   0      0.000000   \n",
      "Oklahoma                             0.0                   0      0.000000   \n",
      "Oregon                               0.0                   0      0.000000   \n",
      "Pennsylvania                         0.0                   0   8163.000000   \n",
      "Rhode Island                         0.0                   0      0.000000   \n",
      "South Carolina                       0.0                   0      0.000000   \n",
      "South Dakota                         0.0                   0      0.000000   \n",
      "Tennessee                            0.0                6139      0.000000   \n",
      "Texas                                0.0                   0      0.000000   \n",
      "Utah                                 0.0                   0      0.000000   \n",
      "Vermont                              0.0                   0      0.000000   \n",
      "Virginia                             0.0                   0      0.000000   \n",
      "Washington                           0.0                   0      0.000000   \n",
      "West Virginia                        0.0                   0      0.000000   \n",
      "Wisconsin                         2740.0                   0      0.000000   \n",
      "Wyoming                              0.0                   0      0.000000   \n",
      "All                               5081.2                3117  18012.666667   \n",
      "\n",
      "region                Mountain  New England   Pacific  South Atlantic  \\\n",
      "state                                                                   \n",
      "Alabama                  0.000          0.0       0.0        0.000000   \n",
      "Alaska                   0.000          0.0    1434.0        0.000000   \n",
      "Arizona               7259.000          0.0       0.0        0.000000   \n",
      "Arkansas                 0.000          0.0       0.0        0.000000   \n",
      "California               0.000          0.0  109008.0        0.000000   \n",
      "Colorado              7607.000          0.0       0.0        0.000000   \n",
      "Connecticut              0.000       2280.0       0.0        0.000000   \n",
      "Delaware                 0.000          0.0       0.0      708.000000   \n",
      "District of Columbia     0.000          0.0       0.0     3770.000000   \n",
      "Florida                  0.000          0.0       0.0    21443.000000   \n",
      "Georgia                  0.000          0.0       0.0     6943.000000   \n",
      "Hawaii                   0.000          0.0    4131.0        0.000000   \n",
      "Idaho                 1297.000          0.0       0.0        0.000000   \n",
      "Illinois                 0.000          0.0       0.0        0.000000   \n",
      "Indiana                  0.000          0.0       0.0        0.000000   \n",
      "Iowa                     0.000          0.0       0.0        0.000000   \n",
      "Kansas                   0.000          0.0       0.0        0.000000   \n",
      "Kentucky                 0.000          0.0       0.0        0.000000   \n",
      "Louisiana                0.000          0.0       0.0        0.000000   \n",
      "Maine                    0.000       1450.0       0.0        0.000000   \n",
      "Maryland                 0.000          0.0       0.0     4914.000000   \n",
      "Massachusetts            0.000       6811.0       0.0        0.000000   \n",
      "Michigan                 0.000          0.0       0.0        0.000000   \n",
      "Minnesota                0.000          0.0       0.0        0.000000   \n",
      "Mississippi              0.000          0.0       0.0        0.000000   \n",
      "Missouri                 0.000          0.0       0.0        0.000000   \n",
      "Montana                983.000          0.0       0.0        0.000000   \n",
      "Nebraska                 0.000          0.0       0.0        0.000000   \n",
      "Nevada                7058.000          0.0       0.0        0.000000   \n",
      "New Hampshire            0.000        835.0       0.0        0.000000   \n",
      "New Jersey               0.000          0.0       0.0        0.000000   \n",
      "New Mexico            1949.000          0.0       0.0        0.000000   \n",
      "New York                 0.000          0.0       0.0        0.000000   \n",
      "North Carolina           0.000          0.0       0.0     6451.000000   \n",
      "North Dakota             0.000          0.0       0.0        0.000000   \n",
      "Ohio                     0.000          0.0       0.0        0.000000   \n",
      "Oklahoma                 0.000          0.0       0.0        0.000000   \n",
      "Oregon                   0.000          0.0   11139.0        0.000000   \n",
      "Pennsylvania             0.000          0.0       0.0        0.000000   \n",
      "Rhode Island             0.000        747.0       0.0        0.000000   \n",
      "South Carolina           0.000          0.0       0.0     3082.000000   \n",
      "South Dakota             0.000          0.0       0.0        0.000000   \n",
      "Tennessee                0.000          0.0       0.0        0.000000   \n",
      "Texas                    0.000          0.0       0.0        0.000000   \n",
      "Utah                  1904.000          0.0       0.0        0.000000   \n",
      "Vermont                  0.000        780.0       0.0        0.000000   \n",
      "Virginia                 0.000          0.0       0.0     3928.000000   \n",
      "Washington               0.000          0.0   16424.0        0.000000   \n",
      "West Virginia            0.000          0.0       0.0     1021.000000   \n",
      "Wisconsin                0.000          0.0       0.0        0.000000   \n",
      "Wyoming                434.000          0.0       0.0        0.000000   \n",
      "All                   3561.375       2150.5   28427.2     5806.666667   \n",
      "\n",
      "region                West North Central  West South Central            All  \n",
      "state                                                                        \n",
      "Alabama                         0.000000                 0.0    2570.000000  \n",
      "Alaska                          0.000000                 0.0    1434.000000  \n",
      "Arizona                         0.000000                 0.0    7259.000000  \n",
      "Arkansas                        0.000000              2280.0    2280.000000  \n",
      "California                      0.000000                 0.0  109008.000000  \n",
      "Colorado                        0.000000                 0.0    7607.000000  \n",
      "Connecticut                     0.000000                 0.0    2280.000000  \n",
      "Delaware                        0.000000                 0.0     708.000000  \n",
      "District of Columbia            0.000000                 0.0    3770.000000  \n",
      "Florida                         0.000000                 0.0   21443.000000  \n",
      "Georgia                         0.000000                 0.0    6943.000000  \n",
      "Hawaii                          0.000000                 0.0    4131.000000  \n",
      "Idaho                           0.000000                 0.0    1297.000000  \n",
      "Illinois                        0.000000                 0.0    6752.000000  \n",
      "Indiana                         0.000000                 0.0    3776.000000  \n",
      "Iowa                         1711.000000                 0.0    1711.000000  \n",
      "Kansas                       1443.000000                 0.0    1443.000000  \n",
      "Kentucky                        0.000000                 0.0    2735.000000  \n",
      "Louisiana                       0.000000              2540.0    2540.000000  \n",
      "Maine                           0.000000                 0.0    1450.000000  \n",
      "Maryland                        0.000000                 0.0    4914.000000  \n",
      "Massachusetts                   0.000000                 0.0    6811.000000  \n",
      "Michigan                        0.000000                 0.0    5209.000000  \n",
      "Minnesota                    3993.000000                 0.0    3993.000000  \n",
      "Mississippi                     0.000000                 0.0    1024.000000  \n",
      "Missouri                     3776.000000                 0.0    3776.000000  \n",
      "Montana                         0.000000                 0.0     983.000000  \n",
      "Nebraska                     1745.000000                 0.0    1745.000000  \n",
      "Nevada                          0.000000                 0.0    7058.000000  \n",
      "New Hampshire                   0.000000                 0.0     835.000000  \n",
      "New Jersey                      0.000000                 0.0    6048.000000  \n",
      "New Mexico                      0.000000                 0.0    1949.000000  \n",
      "New York                        0.000000                 0.0   39827.000000  \n",
      "North Carolina                  0.000000                 0.0    6451.000000  \n",
      "North Dakota                  467.000000                 0.0     467.000000  \n",
      "Ohio                            0.000000                 0.0    6929.000000  \n",
      "Oklahoma                        0.000000              2823.0    2823.000000  \n",
      "Oregon                          0.000000                 0.0   11139.000000  \n",
      "Pennsylvania                    0.000000                 0.0    8163.000000  \n",
      "Rhode Island                    0.000000                 0.0     747.000000  \n",
      "South Carolina                  0.000000                 0.0    3082.000000  \n",
      "South Dakota                  836.000000                 0.0     836.000000  \n",
      "Tennessee                       0.000000                 0.0    6139.000000  \n",
      "Texas                           0.000000             19199.0   19199.000000  \n",
      "Utah                            0.000000                 0.0    1904.000000  \n",
      "Vermont                         0.000000                 0.0     780.000000  \n",
      "Virginia                        0.000000                 0.0    3928.000000  \n",
      "Washington                      0.000000                 0.0   16424.000000  \n",
      "West Virginia                   0.000000                 0.0    1021.000000  \n",
      "Wisconsin                       0.000000                 0.0    2740.000000  \n",
      "Wyoming                         0.000000                 0.0     434.000000  \n",
      "All                          1995.857143              6710.5    7225.784314  \n"
     ]
    }
   ],
   "source": [
    "print(homeless_data.pivot_table(values=\"individuals\", index=\"state\", columns=\"region\", fill_value=0, margins= False))\n",
    "print(homeless_data.pivot_table(values=\"individuals\", index=\"state\", columns=\"region\", fill_value=0, margins= True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create Pivot table, Print the mean weekly_sales by department and type, filling in any missing values with 0\n",
    "#Create Pivot table, Print the mean weekly_sales by department and type, filling in any missing values with 0 and summing all rows and columns."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}