index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="https://www.w3.org/2005/Atom">
  <channel>
    <title>Objective Funk on Objective Funk</title>
    <link>https://nsaphra.github.io/</link>
    <description>Recent content in Objective Funk on Objective Funk</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-us</language>
    <copyright>&amp;copy; 2018</copyright>
    <lastBuildDate>Wed, 20 Apr 2016 00:00:00 -0700</lastBuildDate>
    <atom:link href="/" rel="self" type="application/rss+xml" />
    
    <item>
      <title>Sources of Variance in Pretraining and Finetuning</title>
      <link>https://nsaphra.github.io/talk/ucirvine/</link>
      <pubDate>Mon, 20 Jun 2022 13:00:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/talk/ucirvine/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Interpretability Creationism</title>
      <link>https://nsaphra.github.io/post/creationism/</link>
      <pubDate>Tue, 07 Jun 2022 00:00:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/post/creationism/</guid>
      <description>

&lt;p&gt;For centuries, Europeans agreed that the presence of a cuckoo egg was a great honor to a nesting bird, as it granted an opportunity to exhibit Christian hospitality. The devout bird enthusiastically fed her holy guest, even more so than she would her own (evicted) chicks &lt;a href=&#34;https://app.thestorygraph.com/books/37ed3b62-8a3a-448b-9e37-cd5e5f51c640&#34; target=&#34;_blank&#34;&gt;(Davies, 2015)&lt;/a&gt;. In 1859, Charles Darwin’s studies of another occasional brood parasite, finches, called into question any rosy, cooperative view of bird behavior &lt;a href=&#34;https://app.thestorygraph.com/books/44185106-8198-42ef-bacf-8a9bf691e654&#34; target=&#34;_blank&#34;&gt;(Darwin, 1859)&lt;/a&gt;. Without considering the evolution of the cuckoo’s role, it would have been difficult to recognize the nesting bird not as a gracious host to the cuckoo chick, but as an unfortunate dupe. The historical process is essential to understanding its biological consequences; as evolutionary biologist Theodosius Dobzhansky put it, &lt;a href=&#34;https://en.wikipedia.org/wiki/Nothing_in_Biology_Makes_Sense_Except_in_the_Light_of_Evolution#cite_note-Dobz_Nothing-1&#34; target=&#34;_blank&#34;&gt;Nothing in Biology Makes Sense Except in the Light of Evolution&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://upload.wikimedia.org/wikipedia/commons/5/5c/Reed_warbler_cuckoo.jpg&#34; alt=&#34;By Per Harald Olsen - Own work, CC BY-SA 3.0&#34; width=&#34;200&#34;/&gt;&lt;/p&gt;

&lt;p&gt;Certainly SGD is not literally biological evolution, but post-hoc analysis in machine learning &lt;a href=&#34;https://twitter.com/ch402/status/1533164918886703104&#34; target=&#34;_blank&#34;&gt;has a lot in common&lt;/a&gt; with scientific approaches in biology, and likewise often requires an understanding of the origin of model behavior. Therefore, the following holds whether looking at parasitic brooding behavior or at the inner representations of a neural network: if we do not consider how a system develops, it is difficult to distinguish a pleasing story from a useful analysis.&lt;/p&gt;

&lt;h2 id=&#34;just-so-stories&#34;&gt;Just-So Stories&lt;/h2&gt;

&lt;p&gt;We have many pleasing &lt;a href=&#34;https://en.wikipedia.org/wiki/Just_So_Stories&#34; target=&#34;_blank&#34;&gt;just-so stories&lt;/a&gt; in NLP. Much has been made of interpretable artifacts such as &lt;a href=&#34;https://aclanthology.org/2022.acl-long.269.pdf&#34; target=&#34;_blank&#34;&gt;syntactic attention distributions&lt;/a&gt; or &lt;a href=&#34;https://openai.com/blog/unsupervised-sentiment-neuron/&#34; target=&#34;_blank&#34;&gt;selective neurons&lt;/a&gt;. But how can we know if such a pattern of behavior is actually used by the model?
Causal modeling can help, but interventions to test the influence of particular features and patterns may target only particular types of behavior explicitly. In practice, it may be possible only to perform certain types of slight interventions on specific units within a representation, failing to reflect interactions between features properly. Furthermore, in staging these interventions, we create distribution shifts that a model may not be robust to, regardless of whether that behavior is part of a core strategy. Significant distribution shifts can cause erratic behavior, so why shouldn&amp;rsquo;t they cause spurious interpretable artifacts? In practice, we find &lt;a href=&#34;https://arxiv.org/pdf/2010.12016.pdf&#34; target=&#34;_blank&#34;&gt;no shortage&lt;/a&gt; of incidental observations construed as crucial.&lt;/p&gt;

&lt;p&gt;Fortunately, the study of evolution has provided a number of ways to interpret the artifacts produced by a model. They might be vestigial, like a human tailbone. They may have dependencies, with some features and structures relying on the presence of other properties earlier in training, like the requirement for light sensing before a complex eye can develop. Some artifacts might represent side effects of training, like how junk DNA constitutes a majority of our genetic code without influencing our phenotypes.&lt;/p&gt;

&lt;p&gt;We have a number of theories for how such unused artifacts might emerge while training models. For example, the &lt;a href=&#34;https://arxiv.org/abs/1703.00810&#34; target=&#34;_blank&#34;&gt;Information Bottleneck Hypothesis&lt;/a&gt; predicts how inputs may be memorized early in training, before representations are compressed to only retain information about the output. These early memorized interpolations may not ultimately be useful when generalizing to unseen data, but they are essential in order to eventually learn to specifically represent the output. We also can infer the possibility of vestigial features, because early training behavior is so distinct from late training: &lt;a href=&#34;https://arxiv.org/abs/1905.11604&#34; target=&#34;_blank&#34;&gt;earlier models are more simplistic&lt;/a&gt;. In the case of language models, they &lt;a href=&#34;https://arxiv.org/abs/2109.06096&#34; target=&#34;_blank&#34;&gt;behave similarly to ngram models&lt;/a&gt; early on and &lt;a href=&#34;https://www.aclweb.org/anthology/2020.emnlp-main.16&#34; target=&#34;_blank&#34;&gt;exhibit linguistic patterns&lt;/a&gt; later. Side effects of such a heteroskedastic training process could easily be mistaken for crucial components of a trained model.&lt;/p&gt;

&lt;h2 id=&#34;the-evolutionary-view&#34;&gt;The Evolutionary View&lt;/h2&gt;

&lt;p&gt;I may be unimpressed by &amp;ldquo;interpretability creationist&amp;rdquo; explanations of static fully trained models, but I have engaged in similar analysis myself. I&amp;rsquo;ve published papers on &lt;a href=&#34;https://arxiv.org/pdf/2010.02180.pdf&#34; target=&#34;_blank&#34;&gt;probing static representations&lt;/a&gt;, and the results often seem intuitive and explanatory. However, the presence of a feature at the end of training is hardly informative about the inductive bias of a model on its own! Consider &lt;a href=&#34;https://openreview.net/forum?id=mNtmhaDkAr&#34; target=&#34;_blank&#34;&gt;Lovering et al.&lt;/a&gt;, who found that the ease of extracting a feature at the start of training, along with an analysis of the finetuning data, has deeper implications for finetuned performance than we get by simply probing at the end of training.&lt;/p&gt;

&lt;p&gt;Let us consider an explanation usually based on analyzing static models: hierarchical behavior in language models. An example of this approach is the claim that &lt;a href=&#34;https://nlp.stanford.edu/pubs/hewitt2019structural.pdf&#34; target=&#34;_blank&#34;&gt;words that are closely linked on a syntax tree have representations that are closer together&lt;/a&gt;, compared to words that are syntactically farther.  How can we know that the model is behaving hierarchically by grouping words according to syntactic proximity? Alternatively, syntactic neighbors may be more strongly linked due to a strong correlation between nearby words because they have higher joint frequency distributions. For example, perhaps constituents like &amp;ldquo;football match&amp;rdquo; are more predictable due to the frequency of their co-occurrence, compared to more distant relations like that between &amp;ldquo;uncle&amp;rdquo; and &amp;ldquo;football&amp;rdquo; in the sentence, &amp;ldquo;My uncle drove me to a football match&amp;rdquo;. In fact, we can be more confident that some language models are hierarchical, because early models encode more local information in &lt;a href=&#34;https://arxiv.org/abs/1811.00225&#34; target=&#34;_blank&#34;&gt;LSTMs&lt;/a&gt; and &lt;a href=&#34;https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html#argument-phase-change&#34; target=&#34;_blank&#34;&gt;Transformers&lt;/a&gt;, and they learn longer distance dependencies more easily when those dependencies can be &lt;a href=&#34;https://arxiv.org/abs/2010.04650&#34; target=&#34;_blank&#34;&gt;stacked onto short familiar constituents&lt;/a&gt; hierarchically.&lt;/p&gt;

&lt;h2 id=&#34;an-example&#34;&gt;An Example&lt;/h2&gt;

&lt;p&gt;I recently had to manage the trap of interpretability creationism myself. My coauthors had found that, when training text classifiers repeatedly with different random seeds, &lt;a href=&#34;https://arxiv.org/abs/2205.12411&#34; target=&#34;_blank&#34;&gt;models can occur in a number of distinct clusters&lt;/a&gt;. Further, we could predict the generalization behavior of a model based on which other models it was connected to on the loss surface. Now, we suspected that different finetuning runs found models with different generalization behavior because their trajectories entered different basins on the loss surface.&lt;/p&gt;

&lt;p&gt;But could we actually make this claim? What if one cluster actually corresponded to earlier stages of a model? Eventually those models would leave for the cluster with better generalization, so our only real result would be that some finetuning runs were slower than others. We had to demonstrate that training trajectories could actually become trapped in a basin, providing an explanation for the diversity of generalization behavior in trained models. Indeed, when we looked at several checkpoints, we confirmed that models that were very central to either cluster would become &lt;em&gt;even more&lt;/em&gt; strongly connected to the rest of their cluster over the course of training. Instead of offering a just-so story based on a static model, we explored the evolution of observed behavior to confirm our hypothesis.&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://nsaphra.github.io/img/clusters.png&#34; alt=&#34;k&#34; /&gt;&lt;/p&gt;

&lt;h2 id=&#34;a-proposal&#34;&gt;A Proposal&lt;/h2&gt;

&lt;p&gt;To be clear, not every question can be answered by &lt;em&gt;only&lt;/em&gt; observing the training process. Causal claims require interventions! In biology, for example, research about antibiotic resistance requires us to deliberately expose bacteria to antibiotics, rather than waiting and hoping to find a natural experiment. Even the claims currently being made based on observations of training dynamics may require experimental confirmation.&lt;/p&gt;

&lt;p&gt;Furthermore, not all claims require &lt;em&gt;any&lt;/em&gt; observation of the training process. Even to ancient humans, many organs had obvious purpose: eyes see, hearts pump blood, and &lt;a href=&#34;https://www.scientificamerican.com/article/aristotle-thought-the-brain-was-a-radiator/&#34; target=&#34;_blank&#34;&gt;brains are refrigerators&lt;/a&gt;. Likewise in NLP, just by analyzing static models we can make simple claims: that particular neurons activate in the presence of particular properties, or that some types of information remain accessible within a model. However, the training dimension can still clarify the meaning of many observations made in a static model.&lt;/p&gt;

&lt;p&gt;My proposal is simple. Are you developing a method of interpretation or analyzing some property of a trained model? Don&amp;rsquo;t just look at final checkpoint in training. Apply that analysis to several intermediate checkpoints. If you are finetuning a model, check several points both early and late in training. If you are analyzing a large language model, &lt;a href=&#34;https://arxiv.org/abs/2106.16163&#34; target=&#34;_blank&#34;&gt;MultiBERTs&lt;/a&gt; and &lt;a href=&#34;https://nlp.stanford.edu/mistral/getting_started/download.html&#34; target=&#34;_blank&#34;&gt;Mistral&lt;/a&gt; both provide intermediate checkpoints sampled from throughout training on masked and autoregressive language models, respectively. Does the behavior that you&amp;rsquo;ve analyzed change over the course of training? Does your belief about the model&amp;rsquo;s strategy actually make sense after observing what happens early in training? There&amp;rsquo;s very little overhead to an experiment like this, and you never know what you&amp;rsquo;ll find!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Sources of Variance in Pretraining and Finetuning (Keynote)</title>
      <link>https://nsaphra.github.io/talk/quebec/</link>
      <pubDate>Wed, 01 Jun 2022 14:30:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/talk/quebec/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Linear Connectivity Reveals Generalization Strategies</title>
      <link>https://nsaphra.github.io/publication/juneja-linear-2022/</link>
      <pubDate>Wed, 01 Jun 2022 00:00:00 +0000</pubDate>
      
      <guid>https://nsaphra.github.io/publication/juneja-linear-2022/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Mathematical Fundamentals of AI</title>
      <link>https://nsaphra.github.io/talk/nyu_mlschool/</link>
      <pubDate>Sat, 01 Jan 2022 15:00:00 -0800</pubDate>
      
      <guid>https://nsaphra.github.io/talk/nyu_mlschool/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Against Monodomainism</title>
      <link>https://nsaphra.github.io/post/monodomainism/</link>
      <pubDate>Wed, 28 Apr 2021 00:00:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/post/monodomainism/</guid>
      <description>&lt;p&gt;Reaching the endpoint of a PhD studying how language models learn, I have spent several years telling people that I study &amp;ldquo;machine learning and natural language processing&amp;rdquo;. However, my colleagues who tried to understand or augment image classifiers would describe themselves only as working in &amp;ldquo;machine learning&amp;rdquo;. I argue that this pattern reflects thinking about what it means to be &amp;ldquo;application&amp;rdquo; work or &amp;ldquo;core&amp;rdquo; machine learning that damages our understanding of statistical modeling and deep learning as a whole.&lt;/p&gt;

&lt;p&gt;Why do we know so little about how language models learn? This gap is in part because consideration of NLP as a domain is historically rare in venues that publish most training dynamics research, or analytic work in learning theory. A current search&lt;sup class=&#34;footnote-ref&#34; id=&#34;fnref:1&#34;&gt;&lt;a href=&#34;#fn:1&#34;&gt;1&lt;/a&gt;&lt;/sup&gt; of ICML 2020 publications returned 169 papers with citations to “Association for Computational Linguistics” or “ACL”, even including citations to many potential sister conferences: NAACL, AACL, or EACL. A search for citations to a single vision conference, “Computer Vision and Pattern Recognition” or “CVPR”, turned up 541 papers. In COLT publications since 2017, the same searches turned up 13 and 23 papers, respectively. In ICML 2020, Wikitext-* or PTB references found only 16 results, while the most popular small corpus for image classification, MNIST, found 264 ICML publications&lt;sup class=&#34;footnote-ref&#34; id=&#34;fnref:2&#34;&gt;&lt;a href=&#34;#fn:2&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;

&lt;p&gt;Linguistics provides us with the salient concept of &lt;em&gt;markedness&lt;/em&gt; &lt;a href=&#34;https://www.degruyter.com/document/doi/10.1515/9783110862010.11/html&#34; target=&#34;_blank&#34;&gt;(Andersen, 1989)&lt;/a&gt;. In language, some forms of a word are the default form, while others are explicitly marked by some additional inflection. An example would be contrast between the word “marked”, which is an &lt;em&gt;unmarked&lt;/em&gt; form compared to “unmarked”, which is &lt;em&gt;marked&lt;/em&gt; by the prefix “un-”. In machine learning, we might call CV an unmarked domain by convention, in contrast to the &lt;em&gt;marked&lt;/em&gt; NLP. This convention means that certain tasks and architectures are considered the default environments to understand. Such a convention privileges understanding continuous data over discrete; ConvNets over LSTMs; ResNets over Transformers; geometric tasks over structured prediction.&lt;/p&gt;

&lt;p&gt;Understanding one machine learning domain will always extend analysis of others. For example, latent tree structure is inherent to both domains, but in CV, it is obscured by the image data from which we must compose eyes and mouth into a face—and subsequently, body and face into a cow &lt;a href=&#34;https://ieeexplore.ieee.org/document/6909858&#34; target=&#34;_blank&#34;&gt;(Vedaldi et al., 2014)&lt;/a&gt;. Image classification is also a language task, because it is our language that provides the intuitions which we use to construct ontologies that turn into image classes; English does not provide us with common distinctions for different packs of wolves, but it names every dog breed, and so the image labels are chosen according to available terminology.&lt;/p&gt;

&lt;p&gt;Many researchers think of text data as arcane, but the unmarked domain of CV displays many idiosyncrasies on which to overfit our understanding of statistical modeling. CV provides us with many interesting geometric phenomena, but the underlying structure of language &lt;em&gt;without&lt;/em&gt; the added noisy channel of an image can provide a clear and simple domain worth analyzing, as well. A true understanding of statistical models must be a multi-domain understanding, not a mono-domain view focused on one task and its peculiarities.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34;&gt;

&lt;hr /&gt;

&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;Searches were performed with Google Scholar.
 &lt;a class=&#34;footnote-return&#34; href=&#34;#fnref:1&#34;&gt;&lt;sup&gt;^&lt;/sup&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;*CL venues have also become distanced from work in computational linguistics &lt;a href=&#34;https://www.aclweb.org/anthology/J07-2013.pdf&#34; target=&#34;_blank&#34;&gt;(Reiter, 2007)&lt;/a&gt;, leaving NLP as a field deprived of new scientific work in its data domain as well as new scientific work in its methodologies.
 &lt;a class=&#34;footnote-return&#34; href=&#34;#fnref:2&#34;&gt;&lt;sup&gt;^&lt;/sup&gt;&lt;/a&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>A Non-Linear Structural Probe</title>
      <link>https://nsaphra.github.io/publication/white-nonlinear-2021/</link>
      <pubDate>Fri, 01 Jan 2021 00:00:00 +0000</pubDate>
      
      <guid>https://nsaphra.github.io/publication/white-nonlinear-2021/</guid>
      <description></description>
    </item>
    
    <item>
      <title>The MultiBERTs: BERT Reproductions for Robustness Analysis</title>
      <link>https://nsaphra.github.io/publication/sellam-multiberts-2021/</link>
      <pubDate>Fri, 01 Jan 2021 00:00:00 +0000</pubDate>
      
      <guid>https://nsaphra.github.io/publication/sellam-multiberts-2021/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Accessible Means Hackable (Keynote)</title>
      <link>https://nsaphra.github.io/talk/pydata/</link>
      <pubDate>Sat, 15 Aug 2020 13:00:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/talk/pydata/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Understanding Privacy-Related Questions on Stack Overflow</title>
      <link>https://nsaphra.github.io/publication/tahaei-understanding-2020/</link>
      <pubDate>Wed, 01 Apr 2020 00:00:00 +0000</pubDate>
      
      <guid>https://nsaphra.github.io/publication/tahaei-understanding-2020/</guid>
      <description></description>
    </item>
    
    <item>
      <title>LSTMs Compose (and Learn) Bottom-Up</title>
      <link>https://nsaphra.github.io/publication/saphra-lstms-2020/</link>
      <pubDate>Wed, 01 Jan 2020 00:00:00 +0000</pubDate>
      
      <guid>https://nsaphra.github.io/publication/saphra-lstms-2020/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Pareto Probing: Trading Off Accuracy for Complexity</title>
      <link>https://nsaphra.github.io/publication/pimentel-pareto-2020/</link>
      <pubDate>Wed, 01 Jan 2020 00:00:00 +0000</pubDate>
      
      <guid>https://nsaphra.github.io/publication/pimentel-pareto-2020/</guid>
      <description></description>
    </item>
    
    <item>
      <title>What Does a Coder Do If They Can&#39;t Type?</title>
      <link>https://nsaphra.github.io/post/hands/</link>
      <pubDate>Thu, 08 Aug 2019 18:11:42 +0100</pubDate>
      
      <guid>https://nsaphra.github.io/post/hands/</guid>
      <description>

&lt;p&gt;In August of 2015, my hands stopped working. I could still control them, but every movement accumulated more pain, so every motion came with a cost: getting dressed in the  morning, sending a text, lifting a glass. I was interning at Google that summer about to begin a PhD in Scotland, but coding all day would have left me in agony. In relating this story, I often mention that for months before I learned to work without my hands, I had nothing to do but go to a bar and order a shot of vodka with a straw in it. This is a very funny joke.&lt;/p&gt;

&lt;p&gt;I have been in pain for four years.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&#34;talon&#34;&gt;Talon&lt;/h2&gt;

&lt;p&gt;Due to this disability, I cannot type or write by hand. Many people have asked me about the stack that enables me to be productive in spite of this limitation. I hope this information is helpful both for people with more severe limitations, and for programmers with mild repetitive stress injuries who can benefit from reducing their keyboard use.&lt;/p&gt;

&lt;p&gt;The star of the show is &lt;a href=&#34;https://talonvoice.com/&#34; target=&#34;_blank&#34;&gt;Talon&lt;/a&gt;, a system which makes it easy to write customized grammars and scripts that work with speech recognition systems to enable programming. Commands range from simple aliases for common symbols to complex meta-commands which repeat a previous utterance or  change dictation modes. For example, just in the case of parentheses, I have separate commands for &lt;code&gt;(&lt;/code&gt;, &lt;code&gt;)&lt;/code&gt;, &lt;code&gt;()&lt;/code&gt;, and &lt;code&gt;()⬅️&lt;/code&gt; (which leaves the cursor between parentheses so my next utterance is bracketed).&lt;/p&gt;

&lt;p&gt;Each Talon user has a number of personal scripts. The most precious script that I&amp;rsquo;ve written is probably my indexed clipboard:&lt;/p&gt;

&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-python&#34; data-lang=&#34;python&#34;&gt; &lt;span style=&#34;color:#f92672&#34;&gt;from&lt;/span&gt; talon.voice &lt;span style=&#34;color:#f92672&#34;&gt;import&lt;/span&gt; Key, press, Str, Context
 &lt;span style=&#34;color:#f92672&#34;&gt;from&lt;/span&gt; talon &lt;span style=&#34;color:#f92672&#34;&gt;import&lt;/span&gt; clip
 &lt;span style=&#34;color:#f92672&#34;&gt;from&lt;/span&gt; .talon_community.utils &lt;span style=&#34;color:#f92672&#34;&gt;import&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;*&lt;/span&gt;

 ctx &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; Context(&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;clipboard&amp;#39;&lt;/span&gt;)

 &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;copy_selection&lt;/span&gt;(m):
     &lt;span style=&#34;color:#66d9ef&#34;&gt;with&lt;/span&gt; clip&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;capture() &lt;span style=&#34;color:#66d9ef&#34;&gt;as&lt;/span&gt; sel:
         press(&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;cmd-c&amp;#39;&lt;/span&gt;)
     &lt;span style=&#34;color:#66d9ef&#34;&gt;if&lt;/span&gt; len(m&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;_words) &lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;1&lt;/span&gt;:
         key &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39; &amp;#39;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;join(parse_words(m))
         value &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; sel&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;get()
         keymap[&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;paste &lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;%s&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;%&lt;/span&gt; key] &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; value
         ctx&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;keymap(keymap)
         ctx&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;reload()
     &lt;span style=&#34;color:#66d9ef&#34;&gt;else&lt;/span&gt;:
         clip&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;set(sel&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;get())

 keymap &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; {
     &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;paste&amp;#39;&lt;/span&gt;: Key(&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;cmd-v&amp;#39;&lt;/span&gt;),
     &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;clip [&amp;lt;dgndictation&amp;gt;]&amp;#39;&lt;/span&gt;: copy_selection,
 }

 ctx&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;keymap(keymap)&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The use is simple. After selecting a particular phrase using my cursor control commands, I say &amp;ldquo;clip [foo]&amp;ldquo;, and every time I want to enter the same phrase after, I say &amp;ldquo;paste [foo]&amp;ldquo;. I therefore only have to dictate a particularly obnoxious variable name once. However, it does introduce a new challenge: every variable has two names, its written name and its spoken name. This unfortunate side effect exacerbates the difficulty of naming variables, which has been called &amp;ldquo;the hardest problem in computer science&amp;rdquo;.&lt;/p&gt;

&lt;p&gt;If you are a vim or Emacs power user, this may all feel familiar to you. I have commands for searching, moving a cursor, selection, and manipulating the clipboard. Learning to dictate code is a lot like learning a new text editor very thoroughly, down to the challenge of customizing for your particular languages and needs.&lt;/p&gt;

&lt;p&gt;The &lt;a href=&#34;https://github.com/dwiel/talon_community&#34; target=&#34;_blank&#34;&gt;Talon community&lt;/a&gt; has specialized commands that take effect depending on application or programming language. For a Perl user, for example, a good starting point might be to borrow settings from Emily Shea:&lt;/p&gt;


&lt;div style=&#34;position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;&#34;&gt;
  &lt;iframe src=&#34;//www.youtube.com/embed/Mz3JeYfBTcY&#34; style=&#34;position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;&#34; allowfullscreen title=&#34;YouTube Video&#34;&gt;&lt;/iframe&gt;
&lt;/div&gt;


&lt;p&gt;My Talon setup relies on Dragon for the speech recognition side. Unfortunately, Nuance has discontinued OSX Dragon editions that make scripting possible. The coder behind Talon, &lt;a href=&#34;https://ryanhileman.com/&#34; target=&#34;_blank&#34;&gt;Ryan Hileman&lt;/a&gt;, is working on a suitable replacement but at time of writing, it is not yet ready.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&#34;interlude&#34;&gt;Interlude&lt;/h3&gt;

&lt;p&gt;People often ask for my diagnosis, but it officially depends on the country I&amp;rsquo;m in. After an initial assumption that carpal tunnel was to blame, a rheumatologist gave me my first American diagnosis: &lt;em&gt;fibromyalgia&lt;/em&gt;, a word which is Doctorspeak for &amp;ldquo;go away&amp;rdquo;.&lt;/p&gt;

&lt;p&gt;I did not go away. A neurologist performed a skin biopsy that led to my official American diagnosis of &amp;ldquo;idiopathic small fiber neuropathy&amp;rdquo;, meaning that I am missing crucial nerve fibers that transmit heat and pain but nobody knows why. &lt;em&gt;Idiopathic&lt;/em&gt; is also Doctorspeak for &amp;ldquo;go away&amp;rdquo;.&lt;/p&gt;

&lt;p&gt;I went away to the UK. I brought my medical records from America, but my British neurologist did not read my records or perform examinations. After a brief conversation, he gave me my British diagnosis by submitting a note that he had no evidence of any physical cause, and he &amp;ldquo;suspected significant functional overlay&amp;rdquo;, which is how they teach you to call someone delusional in medical school.&lt;/p&gt;

&lt;p&gt;My GP read the note and informed me: He would not prescribe me painkillers. He would not send me for a second opinion from a neurologist, or treatment from any other specialist. The only referral he would write would be to a psychologist to help me &amp;ldquo;resolve the underlying issues behind my pain&amp;rdquo;.&lt;/p&gt;

&lt;p&gt;He then kicked me out of his office for using the word &amp;ldquo;fucking&amp;rdquo;. &amp;ldquo;We do not tolerate cursing&amp;rdquo;, said a sign in the lobby.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&#34;equipment&#34;&gt;Equipment&lt;/h2&gt;

&lt;p&gt;For dictating, I use two different microphones. In the office, I use a &lt;a href=&#34;https://en-uk.sennheiser.com/me-3-ii&#34; target=&#34;_blank&#34;&gt;Sennheiser ME-3&lt;/a&gt;, while for travel I use a Bluetooth headset, the &lt;a href=&#34;https://en-uk.sennheiser.com/mb-pro-1-uc-ml-and-mb-pro-2-uc-ml&#34; target=&#34;_blank&#34;&gt;Sennheiser MB Pro 2&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Another essential piece of equipment for me is my foot pedal, a &lt;a href=&#34;https://www.pageflip.com/products/firefly&#34; target=&#34;_blank&#34;&gt;PageFlip Firefly&lt;/a&gt;. It is programmable, so I have modified the settings to include one that is useful for reading papers in &lt;a href=&#34;https://skim-app.sourceforge.io/&#34; target=&#34;_blank&#34;&gt;Skim&lt;/a&gt;, with the left pedal corresponding to a click and the right pedal corresponding to down arrow. I can use my feet to scroll, and to click for annotations. Another pedal setting I have added maps the pedals to click and shift+enter. This setting is useful for Jupyter notebooks and writing my research notes and mathematical scratch work in &lt;a href=&#34;https://happenapps.com/&#34; target=&#34;_blank&#34;&gt;Quiver&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;When my hands are unusually aggravated, I cannot nudge my mouse around anymore and I fall back on &lt;a href=&#34;https://shortcatapp.com/&#34; target=&#34;_blank&#34;&gt;shortcat&lt;/a&gt;, which allows me to press buttons by dictating keyboard strokes instead of using a mouse.&lt;/p&gt;

&lt;p&gt;My final essential piece of equipment is a pair of &lt;a href=&#34;https://www.futuro-usa.com/3M/en_US/futuro-us/products/~/FUTURO-Night-Wrist-Support/?N=4318+3294508029+3294529207&amp;amp;rt=rud&#34; target=&#34;_blank&#34;&gt;large wrist braces&lt;/a&gt;. The primary purpose of my braces is to discourage me from habitual hand use. I always wear them at conferences, because wearing them is easier than constantly repeating, &amp;ldquo;I cannot shake hands due to a disability&amp;rdquo;.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&#34;interlude-1&#34;&gt;Interlude&lt;/h3&gt;

&lt;p&gt;I struggle with sleep. I dream that my thumbs fall off. I dream that every bone in my hands breaks. I dream that my arms break out in open bleeding sores. I wake up and the pain remains like an invisible nightmare.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&#34;limitations&#34;&gt;Limitations&lt;/h2&gt;

&lt;p&gt;Maybe ironically, the largest concern if you begin to dictate code is that you do not develop a repetitive stress injury in your vocal tract. Speaking quietly can actually cause more damage, hydration is important, and better posture will prevent damage in your voice as well as the rest of your body. I strongly recommend finding a vocal coach who teaches actors and singers how to protect their voices. It is important to take breaks, and you may find talking tiring outside of work.&lt;/p&gt;

&lt;p&gt;Speech recognition technology is not perfect, and the error rate is even higher if you have an unusual accent. Furthermore, it may force you to take time off from programming every time you develop a cold or sore throat. I live in fear of even minor colds.&lt;/p&gt;

&lt;p&gt;Having a private space to dictate in is essential. I was unable to be productive working from home, but as soon as I had a private office I developed momentum on several research projects. I know that this is a huge limitation for a lot of people because of the productivity-destroying, soul-sucking trend towards open offices for all programming work. If your workplace has fallen prey to this trend, you may still have options. In many countries, large companies will be obligated to provide a space to work in if you are disabled.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&#34;addendum&#34;&gt;Addendum&lt;/h3&gt;

&lt;p&gt;Life with my disability is not easy, but thanks to &lt;a href=&#34;https://en.wikipedia.org/wiki/Hedonic_treadmill&#34; target=&#34;_blank&#34;&gt;hedonic adaptation&lt;/a&gt; as well as satisfying &lt;a href=&#34;https://nsaphra.github.io/publication/&#34; target=&#34;_blank&#34;&gt;work&lt;/a&gt; and &lt;a href=&#34;https://auldreekierollerderby.com/2019/08/10/the-one-gift-i-received-along-with-my-disability/&#34; target=&#34;_blank&#34;&gt;hobbies&lt;/a&gt;, I am actually very happy. If you have recently developed a disability or chronic pain condition, it may feel like you could never adjust to the lifestyle required. That is why I have tried to give you a lens into my challenges as well as my successes. It is easy to respond to anyone who has overcome adversity with one of two reactions: &amp;ldquo;It can&amp;rsquo;t be that hard,&amp;rdquo; or &amp;ldquo;I could never do that&amp;rdquo;. Move past both reactions. It is that hard. You can do it.&lt;/p&gt;

&lt;p&gt;If you are currently able-bodied, please support your disabled colleagues, coworkers, and anyone you have power over in their quest to do valuable and fulfilling work. I encourage other disabled scientists and programmers to reach out to me with any questions they have.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;em&gt;Thank you for comments on early drafts: &lt;a href=&#34;https://www.cs.jhu.edu/~vandurme/Carrell.html&#34; target=&#34;_blank&#34;&gt;Annabelle Carrell&lt;/a&gt;, &lt;a href=&#34;https://www.craiginnes.com/&#34; target=&#34;_blank&#34;&gt;Craig Innes&lt;/a&gt;, &lt;a href=&#34;https://twitter.com/uscm_&#34; target=&#34;_blank&#34;&gt;Matthew Summers&lt;/a&gt;, &lt;a href=&#34;https://www.dinalevitan.com/&#34; target=&#34;_blank&#34;&gt;Dina Lev&lt;/a&gt;, &lt;a href=&#34;https://www.ims.uni-stuttgart.de/institut/mitarbeiter/schlecdk/index.en.html&#34; target=&#34;_blank&#34;&gt;Dominik Schlechtweg&lt;/a&gt;, and &lt;a href=&#34;https://americanstudies.yale.edu/people/yuhe-faye-wang&#34; target=&#34;_blank&#34;&gt;Yuhe Faye Wang&lt;/a&gt; (who is in The Humanities!). Thank you to &lt;a href=&#34;https://www.recurse.com/&#34; target=&#34;_blank&#34;&gt;The Recurse Center&lt;/a&gt; for providing a private space for me to learn to dictate code. Thank you to my PhD advisor, &lt;a href=&#34;https://alopez.github.io/&#34; target=&#34;_blank&#34;&gt;Adam Lopez&lt;/a&gt;, who has unfailingly supported me and made all of this possible.&lt;/em&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Blackbox NLP Panel Discussion</title>
      <link>https://nsaphra.github.io/talk/florence/</link>
      <pubDate>Thu, 25 Jul 2019 13:00:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/talk/florence/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Get Hooked On Neural Net Inspection! That was a pun!</title>
      <link>https://nsaphra.github.io/talk/bangbangwest/</link>
      <pubDate>Tue, 28 May 2019 13:00:00 -0700</pubDate>
      
      <guid>https://nsaphra.github.io/talk/bangbangwest/</guid>
      <description></description>
    </item>
    
  </channel>
</rss>