notebooks/plugy_guide.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plugy guide\n",
    "`plugy` is a Python module for the analysis of plug microfluidics spectrometric reads. Plug-based microfluidics is a three phase microfluidics system with fluorinated oil as continuous phase carrying large (~500 nl volume) aqueous droplets (plugs), separated by similar size mineral oil plugs. At the spectrometric data acquisition the plugs pass through a transparent tube targeted by multiple laser beams, while photomultiplers (PMTs) detect the fluorescence. `plugy` is able to read and process the recordings of the PMT data. It detects the plugs and quantifies their fluorescence across multiple channels. If the plug sequence comes from a drug combinational screening experiment, where blue plugs (so called barcode) separate each of the samples (drug combinations) from each other, `plugy` is able to identify these samples, organize the data by sample and perform various statiscics and visualizations.\n",
    "## 1. Installation\n",
    "`plugy` is available in the git repositories at https://git.embl.de/grp-merten/plugy and https://github.com/saezlab/plugy. Once you have Python 3 on your computer, install it by `pip`:\n",
    "```bash\n",
    "pip3 install git+https://git.embl.de/grp-merten/plugy.git@dev\n",
    "```\n",
    "## 2. Basic usage\n",
    "If you analyse a drug combination screen you need three input files:\n",
    "* The PMT recordings\n",
    "* The valve (chip inlet) layout: which drug was connected to which inlet\n",
    "* The sample sequence: a list of the samples generated during the experiment\n",
    "\n",
    "For processing any other kind of PMT recording, to identify plugs and samples, the first one is enough, no layout and sequence needed.\n",
    "\n",
    "Normally we need to import the these two submodules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from plugy import exp\n",
    "from plugy.data import config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we create a configuration object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'Exp53_1.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_one neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_one_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, using the configuration, we create an experiment object. By default the experiment object runs all the analysis and visualization upon creation. You will see later how to change this behaviour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:21:49 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 15:21:49 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:21:49 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:21:49 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp53/sequence_9_drugs_and_one neg_Ctr.csv\n",
      "16.11.20 15:21:49 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:21:49 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:21:49 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 15:21:51 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 15:21:52 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 15:21:52 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 15:21:52 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00,  7.54it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:21:55 - plugy.data.plug - INFO - Found 2 cycles, 2 with the expected number of samples. Sample count deviations: 0=0, 1=0. Best barcode detection parameters: times=1.\n",
      "16.11.20 15:21:55 - plugy.data.plug - INFO - Labelling samples with compound names\n",
      "16.11.20 15:21:55 - plugy.data.exp - INFO - Calculating statistics\n",
      "16.11.20 15:22:12 - plugy.data.exp - INFO - Plotted PMT data to /home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/results/qc/pmt_overview.png\n",
      "16.11.20 15:22:39 - plugy.data.exp - WARNING - Contamination above threshold (0.1398159072488064 > 0.03)\n",
      "16.11.20 15:24:14 - plugy.data.exp - CRITICAL - Quality control failed due to the following reasons: Contamination above threshold (0.1398159072488064 > 0.03). See also the QC plots for more information. In case you still want to continue, you can set the `ignore_qc_result` config parameter to True.\n",
      "16.11.20 15:24:14 - plugy.data.exp - INFO - Running drug combination analysis\n",
      "16.11.20 15:24:17 - plugy.data.exp - INFO - Saving violin plots to /home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/results/drug_comb_z_violins.png\n",
      "16.11.20 15:24:27 - plugy.data.exp - INFO - Saving violin plots to /home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/results/drug_comb_z_violins_by-cycle.png\n",
      "16.11.20 15:24:33 - plugy.data.exp - INFO - Saving heatmap(s) to /home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/results/drug_comb_z_heatmap.png\n",
      "16.11.20 15:24:35 - plugy.data.exp - INFO - Saving heatmap(s) to /home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/results/drug_comb_z_heatmap_by-cycle.png\n"
     ]
    }
   ],
   "source": [
    "plug_exp = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. How to access the data inside the experiment object\n",
    "The experiment object contains data frames (`pandas.DataFrame` objects, read more about `pandas` here: https://pandas.pydata.org/pandas-docs/stable/user_guide/10min.html) with the raw PTM recordings, the plugs and the samples. The raw data is a very long data frame as it contains 300 rows per second if you used a 300 Hz acquisition rate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>green</th>\n",
       "      <th>orange</th>\n",
       "      <th>uv</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2054.813000</td>\n",
       "      <td>0.015259</td>\n",
       "      <td>0.012818</td>\n",
       "      <td>0.201117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2054.816333</td>\n",
       "      <td>0.016785</td>\n",
       "      <td>0.013733</td>\n",
       "      <td>0.191961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2054.819667</td>\n",
       "      <td>0.017090</td>\n",
       "      <td>0.011292</td>\n",
       "      <td>0.189825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2054.823000</td>\n",
       "      <td>0.017396</td>\n",
       "      <td>0.011597</td>\n",
       "      <td>0.196539</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2054.826333</td>\n",
       "      <td>0.017090</td>\n",
       "      <td>0.012818</td>\n",
       "      <td>0.201422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1943095</th>\n",
       "      <td>8531.796333</td>\n",
       "      <td>0.000610</td>\n",
       "      <td>0.007019</td>\n",
       "      <td>0.005188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1943096</th>\n",
       "      <td>8531.799667</td>\n",
       "      <td>0.000305</td>\n",
       "      <td>0.011292</td>\n",
       "      <td>0.006714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1943097</th>\n",
       "      <td>8531.803000</td>\n",
       "      <td>0.000305</td>\n",
       "      <td>0.010681</td>\n",
       "      <td>0.006714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1943098</th>\n",
       "      <td>8531.806333</td>\n",
       "      <td>0.000305</td>\n",
       "      <td>0.007324</td>\n",
       "      <td>0.006104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1943099</th>\n",
       "      <td>8531.809667</td>\n",
       "      <td>0.000610</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.007630</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1943100 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                time     green    orange        uv\n",
       "0        2054.813000  0.015259  0.012818  0.201117\n",
       "1        2054.816333  0.016785  0.013733  0.191961\n",
       "2        2054.819667  0.017090  0.011292  0.189825\n",
       "3        2054.823000  0.017396  0.011597  0.196539\n",
       "4        2054.826333  0.017090  0.012818  0.201422\n",
       "...              ...       ...       ...       ...\n",
       "1943095  8531.796333  0.000610  0.007019  0.005188\n",
       "1943096  8531.799667  0.000305  0.011292  0.006714\n",
       "1943097  8531.803000  0.000305  0.010681  0.006714\n",
       "1943098  8531.806333  0.000305  0.007324  0.006104\n",
       "1943099  8531.809667  0.000610  0.013123  0.007630\n",
       "\n",
       "[1943100 rows x 4 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plug_exp.pmt_data.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see the plugs. In this experiment we detected 2,134 plugs. For each plug `plugy` calculated its median fluorescence for each channel. It also identified whether the plug is part of the barcoding, and assigned a sample number and a cycle number. Each plug has a start and end time, the difference of these gives the length of the plugs which, assuming a steady flow rate, is proportional to the volume. Typically the length of a plug is around 0.5-2.5 seconds (approx. 100-600 nl at a 800 ul/h flow rate)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>end_time</th>\n",
       "      <th>barcode_peak_median</th>\n",
       "      <th>control_peak_median</th>\n",
       "      <th>readout_peak_median</th>\n",
       "      <th>readout_per_control</th>\n",
       "      <th>barcode</th>\n",
       "      <th>cycle_nr</th>\n",
       "      <th>sample_nr</th>\n",
       "      <th>discard</th>\n",
       "      <th>name</th>\n",
       "      <th>compound_a</th>\n",
       "      <th>compound_b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2056.223000</td>\n",
       "      <td>2056.986333</td>\n",
       "      <td>0.123905</td>\n",
       "      <td>0.012818</td>\n",
       "      <td>0.011902</td>\n",
       "      <td>0.928538</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2058.256333</td>\n",
       "      <td>2059.329667</td>\n",
       "      <td>0.178685</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.015259</td>\n",
       "      <td>1.162768</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2060.149667</td>\n",
       "      <td>2061.353000</td>\n",
       "      <td>0.249336</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.020142</td>\n",
       "      <td>1.534862</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2062.153000</td>\n",
       "      <td>2063.156333</td>\n",
       "      <td>0.205390</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.017090</td>\n",
       "      <td>1.302294</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2065.563000</td>\n",
       "      <td>2066.316333</td>\n",
       "      <td>0.082705</td>\n",
       "      <td>0.012818</td>\n",
       "      <td>0.008850</td>\n",
       "      <td>0.690435</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2129</th>\n",
       "      <td>8353.953000</td>\n",
       "      <td>8355.193000</td>\n",
       "      <td>0.670644</td>\n",
       "      <td>0.013581</td>\n",
       "      <td>0.029603</td>\n",
       "      <td>2.179817</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2130</th>\n",
       "      <td>8356.086333</td>\n",
       "      <td>8357.049667</td>\n",
       "      <td>0.636311</td>\n",
       "      <td>0.014039</td>\n",
       "      <td>0.028993</td>\n",
       "      <td>2.065176</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2131</th>\n",
       "      <td>8357.953000</td>\n",
       "      <td>8358.909667</td>\n",
       "      <td>0.633869</td>\n",
       "      <td>0.013733</td>\n",
       "      <td>0.028993</td>\n",
       "      <td>2.111192</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2132</th>\n",
       "      <td>8359.236333</td>\n",
       "      <td>8359.969667</td>\n",
       "      <td>0.568712</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.026246</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2133</th>\n",
       "      <td>8361.406333</td>\n",
       "      <td>8362.469667</td>\n",
       "      <td>0.668661</td>\n",
       "      <td>0.013733</td>\n",
       "      <td>0.030213</td>\n",
       "      <td>2.200029</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2134 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       start_time     end_time  barcode_peak_median  control_peak_median  \\\n",
       "0     2056.223000  2056.986333             0.123905             0.012818   \n",
       "1     2058.256333  2059.329667             0.178685             0.013123   \n",
       "2     2060.149667  2061.353000             0.249336             0.013123   \n",
       "3     2062.153000  2063.156333             0.205390             0.013123   \n",
       "4     2065.563000  2066.316333             0.082705             0.012818   \n",
       "...           ...          ...                  ...                  ...   \n",
       "2129  8353.953000  8355.193000             0.670644             0.013581   \n",
       "2130  8356.086333  8357.049667             0.636311             0.014039   \n",
       "2131  8357.953000  8358.909667             0.633869             0.013733   \n",
       "2132  8359.236333  8359.969667             0.568712             0.013123   \n",
       "2133  8361.406333  8362.469667             0.668661             0.013733   \n",
       "\n",
       "      readout_peak_median  readout_per_control  barcode  cycle_nr  sample_nr  \\\n",
       "0                0.011902             0.928538     True         0          0   \n",
       "1                0.015259             1.162768     True         0          0   \n",
       "2                0.020142             1.534862     True         0          0   \n",
       "3                0.017090             1.302294     True         0          0   \n",
       "4                0.008850             0.690435     True         0          0   \n",
       "...                   ...                  ...      ...       ...        ...   \n",
       "2129             0.029603             2.179817     True         1         73   \n",
       "2130             0.028993             2.065176     True         1         73   \n",
       "2131             0.028993             2.111192     True         1         73   \n",
       "2132             0.026246             2.000000     True         1         73   \n",
       "2133             0.030213             2.200029     True         1         73   \n",
       "\n",
       "      discard name compound_a compound_b  \n",
       "0        True  NaN        NaN        NaN  \n",
       "1        True  NaN        NaN        NaN  \n",
       "2        True  NaN        NaN        NaN  \n",
       "3        True  NaN        NaN        NaN  \n",
       "4        True  NaN        NaN        NaN  \n",
       "...       ...  ...        ...        ...  \n",
       "2129     True  NaN        NaN        NaN  \n",
       "2130     True  NaN        NaN        NaN  \n",
       "2131     True  NaN        NaN        NaN  \n",
       "2132     True  NaN        NaN        NaN  \n",
       "2133     True  NaN        NaN        NaN  \n",
       "\n",
       "[2134 rows x 13 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plug_exp.plug_data.plug_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, another data frame is available with the samples. This we create from a copy of the previous data frame, but here we don't have the barcode plugs and or plugs discarded for any reason. Instead, if sequence and layout data are available we have the sample name, the name of the compounds, and z-scores calculated for each plug."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>end_time</th>\n",
       "      <th>barcode_peak_median</th>\n",
       "      <th>control_peak_median</th>\n",
       "      <th>readout_peak_median</th>\n",
       "      <th>readout_peak_z_score</th>\n",
       "      <th>readout_per_control_z_score</th>\n",
       "      <th>readout_per_control</th>\n",
       "      <th>cycle_nr</th>\n",
       "      <th>sample_nr</th>\n",
       "      <th>name</th>\n",
       "      <th>compound_a</th>\n",
       "      <th>compound_b</th>\n",
       "      <th>readout_media_norm</th>\n",
       "      <th>readout_media_norm_z_score</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>2103.499667</td>\n",
       "      <td>2105.433000</td>\n",
       "      <td>0.036012</td>\n",
       "      <td>0.291910</td>\n",
       "      <td>0.427564</td>\n",
       "      <td>-1.145513</td>\n",
       "      <td>-0.608381</td>\n",
       "      <td>1.464714</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.793676</td>\n",
       "      <td>-0.932854</td>\n",
       "      <td>1.933333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>2106.199667</td>\n",
       "      <td>2107.429667</td>\n",
       "      <td>0.035096</td>\n",
       "      <td>0.307016</td>\n",
       "      <td>0.520035</td>\n",
       "      <td>-0.847285</td>\n",
       "      <td>-0.211035</td>\n",
       "      <td>1.693837</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.917933</td>\n",
       "      <td>-0.581419</td>\n",
       "      <td>1.230000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>2108.039667</td>\n",
       "      <td>2109.103000</td>\n",
       "      <td>0.035401</td>\n",
       "      <td>0.275277</td>\n",
       "      <td>0.381176</td>\n",
       "      <td>-1.295118</td>\n",
       "      <td>-0.747143</td>\n",
       "      <td>1.384700</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.750462</td>\n",
       "      <td>-1.055076</td>\n",
       "      <td>1.063333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>2109.996333</td>\n",
       "      <td>2111.099667</td>\n",
       "      <td>0.035096</td>\n",
       "      <td>0.281991</td>\n",
       "      <td>0.405896</td>\n",
       "      <td>-1.215394</td>\n",
       "      <td>-0.652293</td>\n",
       "      <td>1.439393</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.780168</td>\n",
       "      <td>-0.971059</td>\n",
       "      <td>1.103333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2111.836333</td>\n",
       "      <td>2112.813000</td>\n",
       "      <td>0.036012</td>\n",
       "      <td>0.268868</td>\n",
       "      <td>0.361644</td>\n",
       "      <td>-1.358111</td>\n",
       "      <td>-0.815884</td>\n",
       "      <td>1.345062</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.729095</td>\n",
       "      <td>-1.115507</td>\n",
       "      <td>0.976667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2109</th>\n",
       "      <td>8296.136333</td>\n",
       "      <td>8297.583000</td>\n",
       "      <td>0.036927</td>\n",
       "      <td>0.805383</td>\n",
       "      <td>1.370433</td>\n",
       "      <td>1.895332</td>\n",
       "      <td>-0.197586</td>\n",
       "      <td>1.701592</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.245276</td>\n",
       "      <td>0.344398</td>\n",
       "      <td>1.446667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2110</th>\n",
       "      <td>8298.359667</td>\n",
       "      <td>8299.023000</td>\n",
       "      <td>0.038148</td>\n",
       "      <td>0.818812</td>\n",
       "      <td>1.083407</td>\n",
       "      <td>0.969644</td>\n",
       "      <td>-0.853891</td>\n",
       "      <td>1.323145</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.968439</td>\n",
       "      <td>-0.438576</td>\n",
       "      <td>0.663333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2111</th>\n",
       "      <td>8299.643000</td>\n",
       "      <td>8301.349667</td>\n",
       "      <td>0.038759</td>\n",
       "      <td>0.621052</td>\n",
       "      <td>1.322062</td>\n",
       "      <td>1.739329</td>\n",
       "      <td>0.543187</td>\n",
       "      <td>2.128746</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.558189</td>\n",
       "      <td>1.229406</td>\n",
       "      <td>1.706667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2112</th>\n",
       "      <td>8302.333000</td>\n",
       "      <td>8304.159667</td>\n",
       "      <td>0.037538</td>\n",
       "      <td>0.885647</td>\n",
       "      <td>1.698660</td>\n",
       "      <td>2.953893</td>\n",
       "      <td>0.177689</td>\n",
       "      <td>1.917988</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.404133</td>\n",
       "      <td>0.793692</td>\n",
       "      <td>1.826667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2113</th>\n",
       "      <td>8304.506333</td>\n",
       "      <td>8305.083000</td>\n",
       "      <td>0.037538</td>\n",
       "      <td>0.955535</td>\n",
       "      <td>2.389294</td>\n",
       "      <td>5.181254</td>\n",
       "      <td>1.187847</td>\n",
       "      <td>2.500478</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.830791</td>\n",
       "      <td>2.000404</td>\n",
       "      <td>0.576667</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1144 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       start_time     end_time  barcode_peak_median  control_peak_median  \\\n",
       "21    2103.499667  2105.433000             0.036012             0.291910   \n",
       "22    2106.199667  2107.429667             0.035096             0.307016   \n",
       "23    2108.039667  2109.103000             0.035401             0.275277   \n",
       "24    2109.996333  2111.099667             0.035096             0.281991   \n",
       "25    2111.836333  2112.813000             0.036012             0.268868   \n",
       "...           ...          ...                  ...                  ...   \n",
       "2109  8296.136333  8297.583000             0.036927             0.805383   \n",
       "2110  8298.359667  8299.023000             0.038148             0.818812   \n",
       "2111  8299.643000  8301.349667             0.038759             0.621052   \n",
       "2112  8302.333000  8304.159667             0.037538             0.885647   \n",
       "2113  8304.506333  8305.083000             0.037538             0.955535   \n",
       "\n",
       "      readout_peak_median  readout_peak_z_score  readout_per_control_z_score  \\\n",
       "21               0.427564             -1.145513                    -0.608381   \n",
       "22               0.520035             -0.847285                    -0.211035   \n",
       "23               0.381176             -1.295118                    -0.747143   \n",
       "24               0.405896             -1.215394                    -0.652293   \n",
       "25               0.361644             -1.358111                    -0.815884   \n",
       "...                   ...                   ...                          ...   \n",
       "2109             1.370433              1.895332                    -0.197586   \n",
       "2110             1.083407              0.969644                    -0.853891   \n",
       "2111             1.322062              1.739329                     0.543187   \n",
       "2112             1.698660              2.953893                     0.177689   \n",
       "2113             2.389294              5.181254                     1.187847   \n",
       "\n",
       "      readout_per_control  cycle_nr  sample_nr          name compound_a  \\\n",
       "21               1.464714         0          0  Cell Control         FS   \n",
       "22               1.693837         0          0  Cell Control         FS   \n",
       "23               1.384700         0          0  Cell Control         FS   \n",
       "24               1.439393         0          0  Cell Control         FS   \n",
       "25               1.345062         0          0  Cell Control         FS   \n",
       "...                   ...       ...        ...           ...        ...   \n",
       "2109             1.701592         1         73  Cell Control         FS   \n",
       "2110             1.323145         1         73  Cell Control         FS   \n",
       "2111             2.128746         1         73  Cell Control         FS   \n",
       "2112             1.917988         1         73  Cell Control         FS   \n",
       "2113             2.500478         1         73  Cell Control         FS   \n",
       "\n",
       "     compound_b  readout_media_norm  readout_media_norm_z_score    length  \n",
       "21           FS            0.793676                   -0.932854  1.933333  \n",
       "22           FS            0.917933                   -0.581419  1.230000  \n",
       "23           FS            0.750462                   -1.055076  1.063333  \n",
       "24           FS            0.780168                   -0.971059  1.103333  \n",
       "25           FS            0.729095                   -1.115507  0.976667  \n",
       "...         ...                 ...                         ...       ...  \n",
       "2109         FS            1.245276                    0.344398  1.446667  \n",
       "2110         FS            0.968439                   -0.438576  0.663333  \n",
       "2111         FS            1.558189                    1.229406  1.706667  \n",
       "2112         FS            1.404133                    0.793692  1.826667  \n",
       "2113         FS            1.830791                    2.000404  0.576667  \n",
       "\n",
       "[1144 rows x 16 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plug_exp.plug_data.sample_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With these data frames you can do whatever is possible with `pandas`, for example you can easily export the data to CSV files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "plug_exp.plug_data.sample_df.to_csv('exp53_samples.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. How to run only certain parts of the workflow\n",
    "Often you don't want to run everything upon the creation of the experiment object. Either your data doesn't contain samples or drug combinations and you just want to detect plugs. Or you are getting an error and you want to inspect the object so you need access to the instance. In all cases when you don't want to run everything, you need to pass the `run = False` parameter to the config. Then the object will be created without running anything."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:48:03 - plugy.data.exp - INFO - Initializing PlugExperiment\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'Exp53_1.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_one neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_one_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False, # <---- see here\n",
    ")\n",
    "\n",
    "plug_exp_norun = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After creating such an object, you can manually call the methods step by step. For example, to read the PMT data and detect the plugs you can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:51:18 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:51:18 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:51:18 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp53/sequence_9_drugs_and_one neg_Ctr.csv\n",
      "16.11.20 15:51:18 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:51:18 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:51:18 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 15:51:19 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 15:51:19 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 15:51:19 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 15:51:20 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00,  9.73it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:51:22 - plugy.data.plug - INFO - Found 2 cycles, 2 with the expected number of samples. Sample count deviations: 0=0, 1=0. Best barcode detection parameters: times=1.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "plug_exp_norun.setup()\n",
    "plug_exp_norun.load()\n",
    "plug_exp_norun.detect_plugs()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we have the data frame with the plugs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>end_time</th>\n",
       "      <th>barcode_peak_median</th>\n",
       "      <th>control_peak_median</th>\n",
       "      <th>readout_peak_median</th>\n",
       "      <th>readout_per_control</th>\n",
       "      <th>barcode</th>\n",
       "      <th>cycle_nr</th>\n",
       "      <th>sample_nr</th>\n",
       "      <th>discard</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2056.223000</td>\n",
       "      <td>2056.986333</td>\n",
       "      <td>0.123905</td>\n",
       "      <td>0.012818</td>\n",
       "      <td>0.011902</td>\n",
       "      <td>0.928538</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2058.256333</td>\n",
       "      <td>2059.329667</td>\n",
       "      <td>0.178685</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.015259</td>\n",
       "      <td>1.162768</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2060.149667</td>\n",
       "      <td>2061.353000</td>\n",
       "      <td>0.249336</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.020142</td>\n",
       "      <td>1.534862</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2062.153000</td>\n",
       "      <td>2063.156333</td>\n",
       "      <td>0.205390</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.017090</td>\n",
       "      <td>1.302294</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2065.563000</td>\n",
       "      <td>2066.316333</td>\n",
       "      <td>0.082705</td>\n",
       "      <td>0.012818</td>\n",
       "      <td>0.008850</td>\n",
       "      <td>0.690435</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2129</th>\n",
       "      <td>8353.953000</td>\n",
       "      <td>8355.193000</td>\n",
       "      <td>0.670644</td>\n",
       "      <td>0.013581</td>\n",
       "      <td>0.029603</td>\n",
       "      <td>2.179817</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2130</th>\n",
       "      <td>8356.086333</td>\n",
       "      <td>8357.049667</td>\n",
       "      <td>0.636311</td>\n",
       "      <td>0.014039</td>\n",
       "      <td>0.028993</td>\n",
       "      <td>2.065176</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2131</th>\n",
       "      <td>8357.953000</td>\n",
       "      <td>8358.909667</td>\n",
       "      <td>0.633869</td>\n",
       "      <td>0.013733</td>\n",
       "      <td>0.028993</td>\n",
       "      <td>2.111192</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2132</th>\n",
       "      <td>8359.236333</td>\n",
       "      <td>8359.969667</td>\n",
       "      <td>0.568712</td>\n",
       "      <td>0.013123</td>\n",
       "      <td>0.026246</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2133</th>\n",
       "      <td>8361.406333</td>\n",
       "      <td>8362.469667</td>\n",
       "      <td>0.668661</td>\n",
       "      <td>0.013733</td>\n",
       "      <td>0.030213</td>\n",
       "      <td>2.200029</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2134 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       start_time     end_time  barcode_peak_median  control_peak_median  \\\n",
       "0     2056.223000  2056.986333             0.123905             0.012818   \n",
       "1     2058.256333  2059.329667             0.178685             0.013123   \n",
       "2     2060.149667  2061.353000             0.249336             0.013123   \n",
       "3     2062.153000  2063.156333             0.205390             0.013123   \n",
       "4     2065.563000  2066.316333             0.082705             0.012818   \n",
       "...           ...          ...                  ...                  ...   \n",
       "2129  8353.953000  8355.193000             0.670644             0.013581   \n",
       "2130  8356.086333  8357.049667             0.636311             0.014039   \n",
       "2131  8357.953000  8358.909667             0.633869             0.013733   \n",
       "2132  8359.236333  8359.969667             0.568712             0.013123   \n",
       "2133  8361.406333  8362.469667             0.668661             0.013733   \n",
       "\n",
       "      readout_peak_median  readout_per_control  barcode  cycle_nr  sample_nr  \\\n",
       "0                0.011902             0.928538     True         0          0   \n",
       "1                0.015259             1.162768     True         0          0   \n",
       "2                0.020142             1.534862     True         0          0   \n",
       "3                0.017090             1.302294     True         0          0   \n",
       "4                0.008850             0.690435     True         0          0   \n",
       "...                   ...                  ...      ...       ...        ...   \n",
       "2129             0.029603             2.179817     True         1         73   \n",
       "2130             0.028993             2.065176     True         1         73   \n",
       "2131             0.028993             2.111192     True         1         73   \n",
       "2132             0.026246             2.000000     True         1         73   \n",
       "2133             0.030213             2.200029     True         1         73   \n",
       "\n",
       "      discard  \n",
       "0        True  \n",
       "1        True  \n",
       "2        True  \n",
       "3        True  \n",
       "4        True  \n",
       "...       ...  \n",
       "2129     True  \n",
       "2130     True  \n",
       "2131     True  \n",
       "2132     True  \n",
       "2133     True  \n",
       "\n",
       "[2134 rows x 10 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plug_exp_norun.plug_data.plug_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1. Only read the PMT recordings\n",
    "If you pass the `run = False` and `init = True` parameters the experiment object will be created with the PMT data read but without detecting plugs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:53:32 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 15:53:32 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:53:32 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:53:32 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp53/sequence_9_drugs_and_one neg_Ctr.csv\n",
      "16.11.20 15:53:32 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:53:32 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:53:32 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 15:53:33 - plugy.data.pmt - INFO - Correcting acquisition time\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'Exp53_1.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_one neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_one_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False,\n",
    "    init = True, # <---- see here\n",
    ")\n",
    "\n",
    "plug_exp_raw = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2. Only detect the plugs\n",
    "For this pass the `run = False` and `plugs = True` parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:55:20 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 15:55:20 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:55:20 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:55:20 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp53/sequence_9_drugs_and_one neg_Ctr.csv\n",
      "16.11.20 15:55:20 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:55:20 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:55:20 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 15:55:21 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 15:55:21 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 15:55:21 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 15:55:22 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00,  9.66it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:55:24 - plugy.data.plug - INFO - Found 2 cycles, 2 with the expected number of samples. Sample count deviations: 0=0, 1=0. Best barcode detection parameters: times=1.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'Exp53_1.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_one neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_one_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False,\n",
    "    plugs = True, # <---- see here\n",
    ")\n",
    "\n",
    "plug_exp_plugs = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3. Identify the samples without creating the figures\n",
    "Sometimes you just want the data frame with the samples without generating all the figures. For this pass the `run = False` and `samples = True` parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:59:49 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 15:59:49 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:59:49 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Channel_9_drugs_and_one_neg_Ctr.csv\n",
      "16.11.20 15:59:49 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp53/sequence_9_drugs_and_one neg_Ctr.csv\n",
      "16.11.20 15:59:49 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:59:49 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp53/Exp53_1.txt\n",
      "16.11.20 15:59:49 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 15:59:50 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 15:59:50 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 15:59:50 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 15:59:51 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00,  8.51it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:59:53 - plugy.data.plug - INFO - Found 2 cycles, 2 with the expected number of samples. Sample count deviations: 0=0, 1=0. Best barcode detection parameters: times=1.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 15:59:54 - plugy.data.plug - INFO - Labelling samples with compound names\n",
      "16.11.20 15:59:54 - plugy.data.exp - INFO - Calculating statistics\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'Exp53_1.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_one neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_one_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False,\n",
    "    samples = True, # <---- see here\n",
    ")\n",
    "\n",
    "plug_exp_norun = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Adjusting sample and cycle detection\n",
    "Due to various technical issues of the experiments it might happen that in some of the cycles the number of identified samples doesn't match the expected, hence the identity of the samples can't be determined. In most of the cases you can fix this error by increasing the maximum allowed plug length or the ratio of the barcoding channel.\n",
    "### 5.1. Adjusting the maximum plug length\n",
    "You should do this for example if you have fused plugs. Use the `peak_max_width` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 16:30:40 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 16:30:40 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:30:40 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:30:40 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp54/sequence_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:30:40 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 16:30:40 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 16:30:40 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 16:30:40 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 16:30:40 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 16:30:40 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 16:30:41 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00, 20.70it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 16:30:41 - plugy.data.plug - INFO - Found 1 cycles, 0 with the expected number of samples. Sample count deviations: 0=-62. Best barcode detection parameters: times=1.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'cycle_III_with adaptor_Not good/Exp54_cycle_III.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False,\n",
    "    plugs = True,\n",
    ")\n",
    "\n",
    "plug_exp = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you see above: \"Found 1 cycles, 0 with the expected number of samples\". Notice that we passed the parameters `run = False` and `plugs = True`. If we go further than plug detection we get an error. Let's try with a `peak_max_width` increased to 7 seconds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 16:33:03 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 16:33:03 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:33:03 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:33:03 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp54/sequence_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:33:03 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 16:33:03 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 16:33:03 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 16:33:04 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 16:33:04 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 16:33:04 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 16:33:04 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00, 15.02it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 16:33:06 - plugy.data.plug - INFO - Found 1 cycles, 1 with the expected number of samples. Sample count deviations: 0=0. Best barcode detection parameters: times=1.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'cycle_III_with adaptor_Not good/Exp54_cycle_III.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 7,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False,\n",
    "    plugs = True,\n",
    ")\n",
    "\n",
    "plug_exp = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This way we could detect the long, fused plugs, so the number of samples match.\n",
    "### 5.2. Barcode ratio\n",
    "Sometimes it's not always true that the barcoding blue dye is the highest in the barcode plugs, or it is highest also in some sample plugs. You can adjust this ratio manually, or you can scan a range to find the ratio which results the highest number of detected cycles. For this set the `barcoding_method` to \"adaptive\" and use the `barcoding_param` option. Here we scan a range from 0.3 to 1.6:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 16:37:05 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 16:37:05 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:37:05 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:37:05 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp54/sequence_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 16:37:05 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 16:37:05 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 16:37:05 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 16:37:06 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 16:37:06 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 16:37:06 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 16:37:07 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [adaptive_method=simple, block_size=7, higher_threshold_factor=1, ratio=1.6, thresholding_method=local]: 100%|██████████| 21/21 [00:01<00:00, 16.56it/s] \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 16:37:09 - plugy.data.plug - INFO - Found 1 cycles, 0 with the expected number of samples. Sample count deviations: 0=26. Best barcode detection parameters: adaptive_method=simple, block_size=7, higher_threshold_factor=1, ratio=0.3, thresholding_method=local.\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'cycle_III_with adaptor_Not good/Exp54_cycle_III.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 7,\n",
    "    barcoding_method = 'adaptive',\n",
    "    barcoding_param = {'ratio': (0.3, 1.6)},\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    run = False,\n",
    "    plugs = True,\n",
    ")\n",
    "\n",
    "plug_exp = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example the ratio doesn't really matter, so finally the value 0.3 was selected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3. Further parameters\n",
    "If the two methods above doesn't work, first inspect the plot of the raw data (pmt_overview.png) and try to find out what is the issue. Sometimes you see between certain samples there are only one barcode plug, or more than 12, or between the cycles there are less than 12 barcode plugs. Or in certain samples there are less than 3 plugs. To address these issues you can adjust the `min_between_samples_barcodes`, the `min_end_cycle_barcodes` or the `min_plugs_in_sample` parameters. There are many further parameters to fine tune the sample detection but these are out of the scope of this tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Merging multiple experiments\n",
    "Sometimes for technical reasons you can record the data in multiple sessions, but for the analysis and visualization you want to concatenate these sequences. For this, create a config and experiment object for all recordings except the first one, using the `run = False` and the `samples = True` parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:13:09 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 17:13:09 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:13:09 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:13:09 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp54/sequence_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:13:09 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_II_wo adaptor/Exp54_cycle_II_1.txt\n",
      "16.11.20 17:13:09 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_II_wo adaptor/Exp54_cycle_II_1.txt\n",
      "16.11.20 17:13:09 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 17:13:10 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 17:13:10 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 17:13:10 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 17:13:11 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00, 13.65it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:13:12 - plugy.data.plug - INFO - Found 1 cycles, 1 with the expected number of samples. Sample count deviations: 0=0. Best barcode detection parameters: times=1.\n",
      "16.11.20 17:13:12 - plugy.data.plug - INFO - Labelling samples with compound names\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:13:12 - plugy.data.exp - INFO - Calculating statistics\n",
      "16.11.20 17:13:12 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 17:13:12 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:13:12 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:13:12 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp54/sequence_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:13:12 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 17:13:12 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle_III_with adaptor_Not good/Exp54_cycle_III.txt\n",
      "16.11.20 17:13:12 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 17:13:13 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 17:13:14 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 17:13:14 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 17:13:14 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00, 15.51it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:13:15 - plugy.data.plug - INFO - Found 1 cycles, 1 with the expected number of samples. Sample count deviations: 0=0. Best barcode detection parameters: times=1.\n",
      "16.11.20 17:13:15 - plugy.data.plug - INFO - Labelling samples with compound names\n",
      "16.11.20 17:13:15 - plugy.data.exp - INFO - Calculating statistics\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "plugy_config_ii = config.PlugyConfig(\n",
    "    pmt_file = 'cycle_II_wo adaptor/Exp54_cycle_II_1.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    # we don't want to run the entire thing\n",
    "    run = False,\n",
    "    # but want to run until sample detection\n",
    "    samples = True,\n",
    ")\n",
    "\n",
    "plugy_config_iii = config.PlugyConfig(\n",
    "    pmt_file = 'cycle_III_with adaptor_Not good/Exp54_cycle_III.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 7,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    # we don't want to run the entire thing\n",
    "    run = False,\n",
    "    # but want to run until sample detection\n",
    "    samples = True,\n",
    ")\n",
    "\n",
    "plug_exp_ii = exp.PlugExperiment(plugy_config_ii)\n",
    "plug_exp_iii = exp.PlugExperiment(plugy_config_iii)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, create a config and an experiment for the first recording, and provide all the subsequent experiment objects in the `append` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:14:37 - plugy.data.exp - INFO - Initializing PlugExperiment\n",
      "16.11.20 17:14:37 - plugy.data.bd - INFO - Creating ChannelMap object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:14:37 - plugy.data.bd - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/Channel_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:14:37 - plugy.data.bd - INFO - Reading PlugSequence from /home/denes/Dokumentumok/microfluidics/vida/data/exp54/sequence_9_drugs_and_pos_neg_Ctr.csv\n",
      "16.11.20 17:14:37 - plugy.data.pmt - INFO - Creating PmtData object from file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle I wo adaptor/exp54_cycle_I.txt\n",
      "16.11.20 17:14:37 - plugy.data.pmt - INFO - Reading file /home/denes/Dokumentumok/microfluidics/vida/data/exp54/cycle I wo adaptor/exp54_cycle_I.txt\n",
      "16.11.20 17:14:37 - plugy.data.pmt - INFO - Detected uncompressed txt file\n",
      "16.11.20 17:14:38 - plugy.data.pmt - INFO - Correcting acquisition time\n",
      "16.11.20 17:14:39 - plugy.data.plug - INFO - Creating PlugData object\n",
      "16.11.20 17:14:39 - plugy.data.plug - INFO - Finding plugs\n",
      "16.11.20 17:14:39 - plugy.data.pmt - INFO - Merging plugs with centers closer than 0.20 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adjusting barcode detection [times=1]: 100%|██████████| 1/1 [00:00<00:00, 13.27it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:14:40 - plugy.data.plug - INFO - Found 1 cycles, 1 with the expected number of samples. Sample count deviations: 0=0. Best barcode detection parameters: times=1.\n",
      "16.11.20 17:14:40 - plugy.data.plug - INFO - Labelling samples with compound names\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:14:41 - plugy.data.plug - INFO - Found 2 cycles, 2 with the expected number of samples. Sample count deviations: 0=0, 1=0. Best barcode detection parameters: times=1.\n",
      "16.11.20 17:14:41 - plugy.data.plug - INFO - Found 3 cycles, 3 with the expected number of samples. Sample count deviations: 0=0, 1=0, 2=0. Best barcode detection parameters: times=1.\n",
      "16.11.20 17:14:41 - plugy.data.exp - INFO - Calculating statistics\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/plugy/exp.py:252: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations. \n",
      "  pmt_overview_fig.tight_layout()\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.11.20 17:14:52 - plugy.data.exp - INFO - Plotted PMT data to /home/denes/Dokumentumok/microfluidics/vida/data/exp53/results/qc/pmt_overview.png\n",
      "16.11.20 17:15:21 - plugy.data.exp - WARNING - Contamination above threshold (0.08873243148938015 > 0.03)\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/plugy/exp.py\", line 524, in plot_sample_cycles\n",
      "    self.plug_data.plot_sample_cycles()\n",
      "  File \"/home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/plugy/data/plug.py\", line 1189, in plot_sample_cycles\n",
      "    sample_cycle_ax[idx_y][idx_x] = self.plot_sample(\n",
      "  File \"/home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/plugy/data/plug.py\", line 1223, in plot_sample\n",
      "    axes = self.plot_plug_pmt_data(axes = axes, cut = (peak_data.start_time.min() - offset, peak_data.end_time.max() + offset))\n",
      "  File \"/home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/plugy/data/plug.py\", line 629, in plot_plug_pmt_data\n",
      "    axes = self.pmt_data.plot_pmt_data(axes, cut = cut)\n",
      "  File \"/home/denes/Dokumentumok/microfluidics/plugy_new/notebooks/plugy/data/pmt.py\", line 299, in plot_pmt_data\n",
      "    axes.set_xticks(range(int(round(df.time.min())), int(round(df.time.max())), major_tick_freq), minor = False)\n",
      "ValueError: cannot convert float NaN to integer\n",
      "16.11.20 17:15:34 - plugy.data.exp - ERROR - Failed to plot sample cycles\n",
      "16.11.20 17:15:34 - plugy.data.exp - CRITICAL - Quality control failed due to the following reasons: Contamination above threshold (0.08873243148938015 > 0.03). See also the QC plots for more information. In case you still want to continue, you can set the `ignore_qc_result` config parameter to True.\n",
      "16.11.20 17:15:34 - plugy.data.exp - INFO - Running drug combination analysis\n",
      "16.11.20 17:15:37 - plugy.data.exp - INFO - Saving violin plots to /home/denes/Dokumentumok/microfluidics/vida/data/exp53/results/drug_comb_z_violins.png\n",
      "16.11.20 17:15:47 - plugy.data.exp - INFO - Saving violin plots to /home/denes/Dokumentumok/microfluidics/vida/data/exp53/results/drug_comb_z_violins_by-cycle.png\n",
      "16.11.20 17:15:55 - plugy.data.exp - INFO - Saving heatmap(s) to /home/denes/Dokumentumok/microfluidics/vida/data/exp53/results/drug_comb_z_heatmap.png\n",
      "16.11.20 17:15:59 - plugy.data.exp - INFO - Saving heatmap(s) to /home/denes/Dokumentumok/microfluidics/vida/data/exp53/results/drug_comb_z_heatmap_by-cycle.png\n"
     ]
    }
   ],
   "source": [
    "plugy_config = config.PlugyConfig(\n",
    "    pmt_file = 'cycle I wo adaptor/exp54_cycle_I.txt',\n",
    "    seq_file = 'sequence_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    channel_file = 'Channel_9_drugs_and_pos_neg_Ctr.csv',\n",
    "    auto_detect_cycles = True,\n",
    "    peak_max_width = 2.5,\n",
    "    ignore_qc_result = True,\n",
    "    normalize_using_control = True,\n",
    "    readout_analysis_column = 'readout_per_control_z_score',\n",
    "    figure_export_file_type = 'png',\n",
    "    # we want to merge these 2,\n",
    "    # appending them to the end of the first one:\n",
    "    append = (plug_exp_ii, plug_exp_iii),\n",
    ")\n",
    "\n",
    "\n",
    "plug_exp = exp.PlugExperiment(plugy_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you see, we ended up with 3 cycles from the three acquisition sessions. All the data has been combined together in the data frames of the last object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>level_0</th>\n",
       "      <th>index</th>\n",
       "      <th>start_time</th>\n",
       "      <th>end_time</th>\n",
       "      <th>barcode_peak_median</th>\n",
       "      <th>control_peak_median</th>\n",
       "      <th>readout_peak_median</th>\n",
       "      <th>readout_peak_z_score</th>\n",
       "      <th>readout_per_control_z_score</th>\n",
       "      <th>readout_per_control</th>\n",
       "      <th>cycle_nr</th>\n",
       "      <th>sample_nr</th>\n",
       "      <th>name</th>\n",
       "      <th>compound_a</th>\n",
       "      <th>compound_b</th>\n",
       "      <th>readout_media_norm</th>\n",
       "      <th>readout_media_norm_z_score</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1481.650667</td>\n",
       "      <td>1484.040667</td>\n",
       "      <td>0.010987</td>\n",
       "      <td>0.239265</td>\n",
       "      <td>0.280160</td>\n",
       "      <td>-0.795588</td>\n",
       "      <td>-0.286266</td>\n",
       "      <td>1.170919</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.111713</td>\n",
       "      <td>-0.834247</td>\n",
       "      <td>2.390000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>24.0</td>\n",
       "      <td>1484.994000</td>\n",
       "      <td>1487.007333</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.258797</td>\n",
       "      <td>0.270394</td>\n",
       "      <td>-0.901775</td>\n",
       "      <td>-0.675440</td>\n",
       "      <td>1.044811</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.992203</td>\n",
       "      <td>-1.179383</td>\n",
       "      <td>2.013333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1491.394000</td>\n",
       "      <td>1492.187333</td>\n",
       "      <td>0.009461</td>\n",
       "      <td>0.264290</td>\n",
       "      <td>0.157170</td>\n",
       "      <td>-2.132872</td>\n",
       "      <td>-2.064537</td>\n",
       "      <td>0.594688</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.564986</td>\n",
       "      <td>-2.413158</td>\n",
       "      <td>0.793333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1493.367333</td>\n",
       "      <td>1495.810667</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.260628</td>\n",
       "      <td>0.307321</td>\n",
       "      <td>-0.500263</td>\n",
       "      <td>-0.260848</td>\n",
       "      <td>1.179156</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.120412</td>\n",
       "      <td>-0.809126</td>\n",
       "      <td>2.443333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>27.0</td>\n",
       "      <td>1496.714000</td>\n",
       "      <td>1498.007333</td>\n",
       "      <td>0.009156</td>\n",
       "      <td>0.278939</td>\n",
       "      <td>0.293436</td>\n",
       "      <td>-0.651242</td>\n",
       "      <td>-0.653347</td>\n",
       "      <td>1.051970</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.999786</td>\n",
       "      <td>-1.157484</td>\n",
       "      <td>1.293333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1909</th>\n",
       "      <td>1174</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16475.934333</td>\n",
       "      <td>16478.211000</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.319529</td>\n",
       "      <td>0.263985</td>\n",
       "      <td>-0.877151</td>\n",
       "      <td>-1.468434</td>\n",
       "      <td>0.826169</td>\n",
       "      <td>2</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.026962</td>\n",
       "      <td>-1.122568</td>\n",
       "      <td>2.276667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1910</th>\n",
       "      <td>1175</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16478.787667</td>\n",
       "      <td>16481.061000</td>\n",
       "      <td>0.010071</td>\n",
       "      <td>0.311289</td>\n",
       "      <td>0.274972</td>\n",
       "      <td>-0.795263</td>\n",
       "      <td>-1.356541</td>\n",
       "      <td>0.883333</td>\n",
       "      <td>2</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.098463</td>\n",
       "      <td>-0.950920</td>\n",
       "      <td>2.273333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1911</th>\n",
       "      <td>1176</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16481.871000</td>\n",
       "      <td>16485.797667</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.299081</td>\n",
       "      <td>0.236518</td>\n",
       "      <td>-1.081868</td>\n",
       "      <td>-1.537635</td>\n",
       "      <td>0.790816</td>\n",
       "      <td>2</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>0.983842</td>\n",
       "      <td>-1.226086</td>\n",
       "      <td>3.926667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1912</th>\n",
       "      <td>1177</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16487.287667</td>\n",
       "      <td>16491.307667</td>\n",
       "      <td>0.010071</td>\n",
       "      <td>0.313425</td>\n",
       "      <td>0.271310</td>\n",
       "      <td>-0.822556</td>\n",
       "      <td>-1.391195</td>\n",
       "      <td>0.865630</td>\n",
       "      <td>2</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.077742</td>\n",
       "      <td>-1.000663</td>\n",
       "      <td>4.020000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1913</th>\n",
       "      <td>1178</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16492.024333</td>\n",
       "      <td>16493.337667</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.317087</td>\n",
       "      <td>0.267342</td>\n",
       "      <td>-0.852131</td>\n",
       "      <td>-1.435257</td>\n",
       "      <td>0.843119</td>\n",
       "      <td>2</td>\n",
       "      <td>73</td>\n",
       "      <td>Cell Control</td>\n",
       "      <td>FS</td>\n",
       "      <td>FS</td>\n",
       "      <td>1.050419</td>\n",
       "      <td>-1.066256</td>\n",
       "      <td>1.313333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1914 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      level_0  index    start_time      end_time  barcode_peak_median  \\\n",
       "0           0   23.0   1481.650667   1484.040667             0.010987   \n",
       "1           1   24.0   1484.994000   1487.007333             0.009766   \n",
       "2           2   25.0   1491.394000   1492.187333             0.009461   \n",
       "3           3   26.0   1493.367333   1495.810667             0.009766   \n",
       "4           4   27.0   1496.714000   1498.007333             0.009156   \n",
       "...       ...    ...           ...           ...                  ...   \n",
       "1909     1174    NaN  16475.934333  16478.211000             0.009766   \n",
       "1910     1175    NaN  16478.787667  16481.061000             0.010071   \n",
       "1911     1176    NaN  16481.871000  16485.797667             0.009766   \n",
       "1912     1177    NaN  16487.287667  16491.307667             0.010071   \n",
       "1913     1178    NaN  16492.024333  16493.337667             0.009766   \n",
       "\n",
       "      control_peak_median  readout_peak_median  readout_peak_z_score  \\\n",
       "0                0.239265             0.280160             -0.795588   \n",
       "1                0.258797             0.270394             -0.901775   \n",
       "2                0.264290             0.157170             -2.132872   \n",
       "3                0.260628             0.307321             -0.500263   \n",
       "4                0.278939             0.293436             -0.651242   \n",
       "...                   ...                  ...                   ...   \n",
       "1909             0.319529             0.263985             -0.877151   \n",
       "1910             0.311289             0.274972             -0.795263   \n",
       "1911             0.299081             0.236518             -1.081868   \n",
       "1912             0.313425             0.271310             -0.822556   \n",
       "1913             0.317087             0.267342             -0.852131   \n",
       "\n",
       "      readout_per_control_z_score  readout_per_control  cycle_nr  sample_nr  \\\n",
       "0                       -0.286266             1.170919         0          0   \n",
       "1                       -0.675440             1.044811         0          0   \n",
       "2                       -2.064537             0.594688         0          0   \n",
       "3                       -0.260848             1.179156         0          0   \n",
       "4                       -0.653347             1.051970         0          0   \n",
       "...                           ...                  ...       ...        ...   \n",
       "1909                    -1.468434             0.826169         2         73   \n",
       "1910                    -1.356541             0.883333         2         73   \n",
       "1911                    -1.537635             0.790816         2         73   \n",
       "1912                    -1.391195             0.865630         2         73   \n",
       "1913                    -1.435257             0.843119         2         73   \n",
       "\n",
       "              name compound_a compound_b  readout_media_norm  \\\n",
       "0     Cell Control         FS         FS            1.111713   \n",
       "1     Cell Control         FS         FS            0.992203   \n",
       "2     Cell Control         FS         FS            0.564986   \n",
       "3     Cell Control         FS         FS            1.120412   \n",
       "4     Cell Control         FS         FS            0.999786   \n",
       "...            ...        ...        ...                 ...   \n",
       "1909  Cell Control         FS         FS            1.026962   \n",
       "1910  Cell Control         FS         FS            1.098463   \n",
       "1911  Cell Control         FS         FS            0.983842   \n",
       "1912  Cell Control         FS         FS            1.077742   \n",
       "1913  Cell Control         FS         FS            1.050419   \n",
       "\n",
       "      readout_media_norm_z_score    length  \n",
       "0                      -0.834247  2.390000  \n",
       "1                      -1.179383  2.013333  \n",
       "2                      -2.413158  0.793333  \n",
       "3                      -0.809126  2.443333  \n",
       "4                      -1.157484  1.293333  \n",
       "...                          ...       ...  \n",
       "1909                   -1.122568  2.276667  \n",
       "1910                   -0.950920  2.273333  \n",
       "1911                   -1.226086  3.926667  \n",
       "1912                   -1.000663  4.020000  \n",
       "1913                   -1.066256  1.313333  \n",
       "\n",
       "[1914 rows x 18 columns]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plug_exp.plug_data.sample_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Saving the results of subsequent runs into separate directories\n",
    "By default `plugy` saves the figures into the `results` directory. You can provide a different directory by the `results_base_dir` option. At each subsequent run in the same working directory, the figures will be overwritten. If you want to keep the old figures and create new directories, you can use the option `result_subdirs = True`. If you want to add a timestamp to these directory names, add the option `timestamp_result_subdirs = True`. Before version 0.5.0 this was the default behaviour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}