docs
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
parent directory.. | ||||
<!-- Generated by pkgdown: do not edit by hand --> <!DOCTYPE html> <html> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>C:/Users/robin/Documents/GitHub/contextual/README.md • contextual</title> <!-- jquery --> <script src="https://code.jquery.com/jquery-3.1.0.min.js" integrity="sha384-nrOSfDHtoPMzJHjVTdCopGqIqeYETSXhZDFyniQ8ZHcVy08QesyHcnOUpMpqnmWq" crossorigin="anonymous"></script> <!-- Bootstrap --> <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous"> <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script> <!-- Font Awesome icons --> <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-T8Gy5hrqNKT+hzMclPo118YTQO6cYprQmhrYwIiQ/3axmI1hQomh7Ud2hPOy8SP1" crossorigin="anonymous"> <!-- clipboard.js --> <script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/1.7.1/clipboard.min.js" integrity="sha384-cV+rhyOuRHc9Ub/91rihWcGmMmCXDeksTtCihMupQHSsi8GIIRDG0ThDc3HGQFJ3" crossorigin="anonymous"></script> <!-- pkgdown --> <link href="pkgdown.css" rel="stylesheet"> <script src="jquery.sticky-kit.min.js"></script> <script src="pkgdown.js"></script> <meta property="og:title" content="C:/Users/robin/Documents/GitHub/contextual/README.md" /> <!-- mathjax --> <script src='https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script> <!--[if lt IE 9]> <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script> <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> <![endif]--> </head> <body> <div class="container template-title-body"> <header> <div class="navbar navbar-inverse navbar-fixed-top" role="navigation"> <div class="container"> <div class="navbar-header"> <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="index.html">contextual</a> </div> <div id="navbar" class="navbar-collapse collapse"> <ul class="nav navbar-nav"> <li> <a href="index.html"> <span class="fa fa-home fa-lg"></span> </a> </li> <li> <a href="articles/contextual.html">Get Started</a> </li> <li> <a href="reference/index.html">Reference</a> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"> Articles <span class="caret"></span> </a> <ul class="dropdown-menu" role="menu"> <li> <a href="articles/glossary.html">Glossary</a> </li> <li> <a href="articles/literature.html">literature</a> </li> <li> <a href="articles/posts.html">Posts</a> </li> </ul> </li> </ul> <ul class="nav navbar-nav navbar-right"> <li> <a href="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/Nth-iteration-labs/contextual"> <span class="fa fa-github fa-lg"></span> </a> </li> </ul> </div><!--/.nav-collapse --> </div><!--/.container --> </div><!--/.navbar --> </header> <div class="row"> <div class="contents col-md-12"> <div class="page-header"> <h1>C:/Users/robin/Documents/GitHub/contextual/README.md</h1> </div> <div id="contextual-multi-armed-bandits-in-r" class="section level1"> <h1 class="hasAnchor"> <a href="#contextual-multi-armed-bandits-in-r" class="anchor"></a>Contextual: Multi-Armed Bandits in R</h1> <p><a href="https://tidyverse.org/lifecycle/#experimental"><img src="https://img.shields.io/badge/lifecycle-experimental-orange.svg" alt="lifecycle"></a> <a href="https://ci.appveyor.com/project/Nth-iteration-labs/contextual"><img src="https://ci.appveyor.com/api/projects/status/github/Nth-iteration-labs/contextual?branch=master&svg=true" alt="AppVeyor Build Status"></a> <a href="https://travis-ci.org/Nth-iteration-labs/contextual"><img src="https://travis-ci.org/Nth-iteration-labs/contextual.svg?branch=master" alt="Build Status"></a> <a href="https://codecov.io/gh/Nth-iteration-labs/contextual"><img src="https://codecov.io/gh/Nth-iteration-labs/contextual/branch/master/graph/badge.svg" alt="codecov"></a> <a href="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/Nth-iteration-labs/contextual"><img src="https://img.shields.io/badge/devel%20version-0.0.0.9012-blue.svg?style=flat" alt="develVersion"></a> <a href="https://www.gnu.org/licenses/gpl-3.0"><img src="https://img.shields.io/badge/License-GPL%20v3-blue.svg" alt="License: GPL v3"></a> <a href="https://zenodo.org/badge/latestdoi/114037654"><img src="https://zenodo.org/badge/114037654.svg" alt="DOI"></a> <a href="https://cran.r-project.org/package=contextual"><img src="http:https://www.r-pkg.org/badges/version/contextual" alt="CRAN_Status_Badge"></a></p> <div id="overview" class="section level2"> <h2 class="hasAnchor"> <a href="#overview" class="anchor"></a>Overview</h2> <p>Contextual is an R package to facilitate the simulation and analysis of Contextual Multi-Armed Bandit (CMAB) policies and algorithms.</p> </div> <div id="documentation" class="section level2"> <h2 class="hasAnchor"> <a href="#documentation" class="anchor"></a>Documentation</h2> <ul> <li><a href="https://nth-iteration-labs.github.io/contextual/reference/index.html">Online reference</a></li> <li><a href="https://pavlov.tech/category/contextual/">Blog at Pavlov</a></li> </ul> </div> <div id="installation" class="section level2"> <h2 class="hasAnchor"> <a href="#installation" class="anchor"></a>Installation</h2> <div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r"><span class="co"># Install development version from GitHub:</span> <span class="co"># install.packages("devtools")</span> devtools<span class="op">::</span><span class="kw"><a href="http:https://www.rdocumentation.org/packages/devtools/topics/install_github">install_github</a></span>(<span class="st">"Nth-iteration-labs/contextual"</span>)</code></pre></div> <p>If you encounter a clear bug, please file a minimal reproducible example on <a href="https://github.com/Nth-iteration-labs/contextual/issues">GitHub</a>.</p> </div> <div id="usage" class="section level2"> <h2 class="hasAnchor"> <a href="#usage" class="anchor"></a>Usage</h2> <p>Running a simulation to compare basic Multi-Armed Bandit policies:</p> <div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r"><span class="kw">library</span>(contextual) horizon <-<span class="st"> </span>100L simulations <-<span class="st"> </span>1000L weight_per_arm <-<span class="st"> </span><span class="kw">c</span>(<span class="fl">0.9</span>, <span class="fl">0.1</span>, <span class="fl">0.1</span>) bandit <-<span class="st"> </span>SyntheticBandit<span class="op">$</span><span class="kw">new</span>(<span class="dt">data =</span> weight_per_arm) agents <-<span class="st"> </span><span class="kw">list</span>( Agent<span class="op">$</span><span class="kw">new</span>(EpsilonGreedyPolicy<span class="op">$</span><span class="kw">new</span>(<span class="fl">0.1</span>, <span class="st">"\U190-greedy"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(RandomPolicy<span class="op">$</span><span class="kw">new</span>(<span class="st">"Random"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(OraclePolicy<span class="op">$</span><span class="kw">new</span>(<span class="st">"Oracle"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(ThompsonSamplingPolicy<span class="op">$</span><span class="kw">new</span>(<span class="fl">1.0</span>, <span class="fl">1.0</span>, <span class="st">"TS"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(Exp3Policy<span class="op">$</span><span class="kw">new</span>(<span class="fl">0.1</span>, <span class="st">"Exp3"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(LinUCBPolicy<span class="op">$</span><span class="kw">new</span>(<span class="fl">1.0</span>, <span class="st">"LinUCB"</span>), bandit) ) simulation <-<span class="st"> </span>Simulator<span class="op">$</span><span class="kw">new</span>(agents, horizon, simulations) history <-<span class="st"> </span>simulation<span class="op">$</span><span class="kw">run</span>() <span class="kw">plot</span>(history, <span class="dt">type =</span> <span class="st">"grid"</span>)</code></pre></div> <p><img src="https://raw.githubusercontent.com/Nth-iteration-labs/contextual/master/tools/readme/basic_multi_plot.png" width="100%"></p> <p>Running a simulation to compare a contextual (LinUCB) and some non-contextual Multi-Armed Bandit policies:</p> <div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r"><span class="kw">library</span>(contextual) horizon <-<span class="st"> </span>100L simulations <-<span class="st"> </span>5000L <span class="co">#k1 #k2 #k3 -> k armed bandit</span> weights <-<span class="st"> </span><span class="kw">matrix</span>( <span class="kw">c</span>( <span class="fl">0.9</span>, <span class="fl">0.3</span>, <span class="fl">0.2</span>, <span class="co">#d1</span> <span class="fl">0.5</span>, <span class="fl">0.6</span>, <span class="fl">0.2</span>, <span class="co">#d2</span> <span class="fl">0.2</span>, <span class="fl">0.1</span>, <span class="fl">0.5</span>), <span class="dt">nrow =</span> <span class="dv">3</span>, <span class="dt">ncol =</span> <span class="dv">3</span>) <span class="co">#d3 -> d features in context</span> bandit <-<span class="st"> </span>SyntheticBandit<span class="op">$</span><span class="kw">new</span>(<span class="dt">data =</span> weights ) agents <-<span class="st"> </span><span class="kw">list</span>( Agent<span class="op">$</span><span class="kw">new</span>(EpsilonGreedyPolicy<span class="op">$</span><span class="kw">new</span>(<span class="fl">0.1</span>, <span class="st">"\U190-greedy"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(LinUCBPolicy<span class="op">$</span><span class="kw">new</span>(<span class="fl">1.0</span>, <span class="st">"LinUCB"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(OraclePolicy<span class="op">$</span><span class="kw">new</span>(<span class="st">"Oracle"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(RandomPolicy<span class="op">$</span><span class="kw">new</span>(<span class="st">"Random"</span>), bandit), Agent<span class="op">$</span><span class="kw">new</span>(<span class="kw"><a href="reference/ThompsonSamplingPolicy.html">ThompsonSamplingPolicy</a></span>(<span class="dv">1</span>, <span class="dv">1</span>, <span class="st">"TSampling"</span>), bandit) ) simulation <-<span class="st"> </span>Simulator<span class="op">$</span><span class="kw">new</span>(agents, horizon, simulations) history <-<span class="st"> </span>simulation<span class="op">$</span><span class="kw">run</span>() <span class="kw">plot</span>(history, <span class="dt">type =</span> <span class="st">"grid"</span>)</code></pre></div> <p><img src="https://raw.githubusercontent.com/Nth-iteration-labs/contextual/master/tools/readme/basic_contextual_plot.png" width="100%"></p> </div> </div> </div> </div> <footer> <div class="copyright"> <p>Developed by Robin van Emden.</p> </div> <div class="pkgdown"> <p>Site built with <a href="http:https://pkgdown.r-lib.org/">pkgdown</a>.</p> </div> </footer> </div> </body> </html>
