diff --git a/README.md b/README.md index eec6db5..92968b8 100644 --- a/README.md +++ b/README.md @@ -8,15 +8,20 @@ This [Julia](http://julialang.org/) package provides computational routines for ## Installation -Within Julia, use the package manager to install VarianceComponentModels: +This package requires Julia v0.7.0 or later, which can be obtained from +https://julialang.org/downloads/ or by building Julia from the sources in the +https://github.com/JuliaLang/julia repository. - Pkg.clone("https://github.com/OpenMendel/VarianceComponentModels.jl.git") - -This package supports Julia v0.7 and v1.0. +The package has not yet been registered and must be installed using the repository location. +Start julia and use the `]` key to switch to the package manager REPL +```julia +(v0.7) pkg> add https://github.com/OpenMendel/VarianceComponentModels.jl.git#Julia-0.7 +``` +Use the backspace key to return to the Julia REPL. ## Citation -If you use this analysis package in your research, please cite the following reference in the resulting publications: +If you use [OpenMendel](https://openmendel.github.io) analysis packages in your research, please cite the following reference in the resulting publications: *Lange K, Papp JC, Sinsheimer JS, Sripracha R, Zhou H, Sobel EM (2013) Mendel: The Swiss army knife of genetic analysis programs. Bioinformatics 29:1568-1570.* diff --git a/docs/heritability.ipynb b/docs/heritability.ipynb index bc77f75..d769838 100644 --- a/docs/heritability.ipynb +++ b/docs/heritability.ipynb @@ -2,32 +2,22 @@ "cells": [ { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ - "# Heritability Analysis" + "# Heritability Analysis\n" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "As an application of the variance component model, this note demonstrates the workflow for heritability analysis in genetics, using a sample data set `cg10k` with **6,670** individuals and **630,860** SNPs. Person IDs and phenotype names are masked for privacy. `cg10k.bed`, `cg10k.bim`, and `cg10k.fam` is a set of Plink files in binary format. `cg10k_traits.txt` contains 13 phenotypes of the 6,670 individuals." ] }, { "cell_type": "code", - "execution_count": 31, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 1, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -46,37 +36,28 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "Machine information:" ] }, { "cell_type": "code", - "execution_count": 32, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 3, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Julia Version 0.6.0\n", - "Commit 903644385b (2017-06-19 13:05 UTC)\n", + "Julia Version 1.1.0\n", + "Commit 80516ca202 (2019-01-21 21:24 UTC)\n", "Platform Info:\n", - " OS: macOS (x86_64-apple-darwin13.4.0)\n", - " CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz\n", + " OS: macOS (x86_64-apple-darwin14.5.0)\n", + " CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz\n", " WORD_SIZE: 64\n", - " BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)\n", - " LAPACK: libopenblas64_\n", " LIBM: libopenlibm\n", - " LLVM: libLLVM-3.9.1 (ORCJIT, haswell)\n" + " LLVM: libLLVM-6.0.1 (ORCJIT, skylake)\n" ] } ], @@ -86,36 +67,26 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Read in binary SNP data" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ - "We will use the [`SnpArrays.jl`](https://github.com/OpenMendel/SnpArrays.jl) package to read in binary SNP data and compute the empirical kinship matrix. Issue \n", + "We will use the [`SnpArrays.jl`](https://github.com/OpenMendel/SnpArrays.jl) package to read in binary SNP data and compute the empirical kinship matrix. The package has not yet been registered and must be installed using the repository location. Start julia and use the `]` key to switch to the package manager REPL\n", "```julia\n", - "Pkg.clone(\"https://github.com/OpenMendel/SnpArrays.jl.git\")\n", + "(v0.7) pkg> add https://github.com/OpenMendel/SnpArrays.jl.git#juliav0.7\n", "```\n", - "within `Julia` to install the `SnpArrays` package." + "Use the backspace key to return to the Julia REPL." ] }, { "cell_type": "code", - "execution_count": 33, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 5, + "metadata": {}, "outputs": [], "source": [ "using SnpArrays" @@ -123,80 +94,69 @@ }, { "cell_type": "code", - "execution_count": 34, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 7, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - " 22.902730 seconds (51.62 k allocations: 1005.845 MiB, 0.11% gc time)\n" + " 0.030779 seconds (54 allocations: 19.467 MiB, 28.08% gc time)\n" ] }, { "data": { "text/plain": [ - "6670×630860 SnpArrays.SnpArray{2}:\n", - " (false, true) (false, true) … (true, true) (true, true) \n", - " (true, true) (true, true) (false, true) (true, false)\n", - " (true, true) (true, true) (true, true) (true, true) \n", - " (true, true) (true, true) (false, true) (true, true) \n", - " (true, true) (true, true) (true, true) (false, true)\n", - " (false, true) (false, true) … (true, true) (true, true) \n", - " (false, false) (false, false) (true, true) (true, true) \n", - " (true, true) (true, true) (true, true) (false, true)\n", - " (true, true) (true, true) (true, true) (true, true) \n", - " (true, true) (true, true) (false, true) (true, true) \n", - " (true, true) (true, true) … (true, true) (true, true) \n", - " (false, true) (false, true) (true, true) (false, true)\n", - " (true, true) (true, true) (true, true) (false, true)\n", - " ⋮ ⋱ \n", - " (false, true) (false, true) (false, true) (false, true)\n", - " (false, true) (false, true) (false, true) (true, true) \n", - " (true, true) (true, true) … (false, true) (true, true) \n", - " (false, true) (false, true) (true, true) (false, true)\n", - " (true, true) (true, true) (false, true) (true, true) \n", - " (true, true) (true, true) (false, false) (false, true)\n", - " (true, true) (true, true) (true, true) (false, true)\n", - " (true, true) (true, true) … (true, true) (true, true) \n", - " (true, true) (true, true) (false, true) (true, true) \n", - " (true, true) (true, true) (true, true) (false, true)\n", - " (false, true) (false, true) (true, true) (true, true) \n", - " (true, true) (true, true) (true, true) (true, true) " + "6670×630860 SnpArray:\n", + " 0x02 0x02 0x03 0x03 0x02 0x03 … 0x02 0x03 0x03 0x03 0x03 0x03\n", + " 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x02 0x02 0x03 0x02 0x01\n", + " 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x03 0x03 0x03 0x03 0x03\n", + " 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02 0x02 0x02 0x02 0x03\n", + " 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02\n", + " 0x02 0x02 0x03 0x03 0x02 0x03 … 0x03 0x03 0x03 0x03 0x03 0x03\n", + " 0x00 0x00 0x03 0x03 0x00 0x03 0x03 0x02 0x03 0x03 0x03 0x03\n", + " 0x03 0x03 0x03 0x03 0x03 0x02 0x03 0x03 0x03 0x03 0x03 0x02\n", + " 0x03 0x03 0x02 0x03 0x02 0x03 0x02 0x02 0x03 0x03 0x03 0x03\n", + " 0x03 0x03 0x03 0x00 0x03 0x03 0x02 0x02 0x03 0x03 0x02 0x03\n", + " 0x03 0x03 0x02 0x03 0x02 0x02 … 0x02 0x03 0x03 0x03 0x03 0x03\n", + " 0x02 0x02 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x03 0x03 0x02\n", + " 0x03 0x03 0x03 0x00 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02\n", + " ⋮ ⋮ ⋱ ⋮ \n", + " 0x02 0x02 0x03 0x03 0x02 0x03 0x03 0x02 0x03 0x03 0x02 0x02\n", + " 0x02 0x02 0x02 0x03 0x00 0x02 0x02 0x03 0x03 0x03 0x02 0x03\n", + " 0x03 0x03 0x02 0x02 0x02 0x03 … 0x03 0x02 0x03 0x03 0x02 0x03\n", + " 0x02 0x02 0x03 0x03 0x02 0x03 0x03 0x02 0x03 0x03 0x03 0x02\n", + " 0x03 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x03 0x03 0x02 0x03\n", + " 0x03 0x03 0x01 0x02 0x02 0x01 0x03 0x00 0x00 0x02 0x00 0x02\n", + " 0x03 0x03 0x03 0x02 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02\n", + " 0x03 0x03 0x02 0x02 0x02 0x03 … 0x00 0x03 0x03 0x03 0x03 0x03\n", + " 0x03 0x03 0x03 0x02 0x02 0x03 0x02 0x02 0x02 0x02 0x02 0x03\n", + " 0x03 0x03 0x03 0x02 0x03 0x02 0x02 0x03 0x02 0x03 0x03 0x02\n", + " 0x02 0x02 0x03 0x03 0x02 0x03 0x03 0x03 0x01 0x03 0x03 0x03\n", + " 0x03 0x03 0x03 0x03 0x03 0x02 0x03 0x00 0x03 0x03 0x03 0x03" ] }, - "execution_count": 34, + "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# read in genotype data from Plink binary file (~50 secs on my laptop)\n", - "@time cg10k = SnpArray(\"cg10k\")" + "@time cg10k = SnpArray(\"cg10k.bed\")" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Summary statistics of SNP data" ] }, { "cell_type": "code", - "execution_count": 35, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 8, + "metadata": {}, "outputs": [ { "data": { @@ -204,7 +164,7 @@ "(6670, 630860)" ] }, - "execution_count": 35, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -213,36 +173,143 @@ "people, snps = size(cg10k)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The positions of the missing data are evaluated by" + ] + }, { "cell_type": "code", - "execution_count": 36, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 9, + "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - " 24" - ] + "data": { + "text/plain": [ + "6670×630860 SparseArrays.SparseMatrixCSC{Bool,Int32} with 5524131 stored entries:\n", + " [5688 , 1] = true\n", + " [6445 , 1] = true\n", + " [109 , 3] = true\n", + " [189 , 3] = true\n", + " [216 , 3] = true\n", + " [254 , 3] = true\n", + " [331 , 3] = true\n", + " [522 , 3] = true\n", + " [525 , 3] = true\n", + " [597 , 3] = true\n", + " [619 , 3] = true\n", + " [672 , 3] = true\n", + " ⋮\n", + " [4929 , 630860] = true\n", + " [5594 , 630860] = true\n", + " [5650 , 630860] = true\n", + " [5780 , 630860] = true\n", + " [5854 , 630860] = true\n", + " [5867 , 630860] = true\n", + " [6084 , 630860] = true\n", + " [6175 , 630860] = true\n", + " [6178 , 630860] = true\n", + " [6239 , 630860] = true\n", + " [6478 , 630860] = true\n", + " [6511 , 630860] = true" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ - "# summary statistics (~50 secs on my laptop)\n", - "@time maf, _, missings_by_snp, = summarize(cg10k);" + "mp = missingpos(cg10k)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The number of missing data values in each column can be evaluated as" ] }, { "cell_type": "code", - "execution_count": 37, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1×630860 Array{Int64,2}:\n", + " 2 0 132 77 0 27 2 2 6 27 2 … 6 4 5 11 0 0 4 29 0 5 43" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "missings_by_snp = sum(mp, dims=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Minor allele frequencies (MAF) for each SNP." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "630860-element Array{Float64,1}:\n", + " 0.1699160167966407 \n", + " 0.17098950524737633 \n", + " 0.11402569593147749 \n", + " 0.2686940694676172 \n", + " 0.21926536731634183 \n", + " 0.23934969140448592 \n", + " 0.19061187762447507 \n", + " 0.20200959808038388 \n", + " 0.027160864345738278\n", + " 0.2997139846454915 \n", + " 0.24625074985003004 \n", + " 0.05555555555555558 \n", + " 0.3659067046647667 \n", + " ⋮ \n", + " 0.22547254725472543 \n", + " 0.4035864345738295 \n", + " 0.20799579957995795 \n", + " 0.44801200300075017 \n", + " 0.2954647845021775 \n", + " 0.14265367316341826 \n", + " 0.1709145427286357 \n", + " 0.2814281428142814 \n", + " 0.06113537117903933 \n", + " 0.052473763118440764\n", + " 0.13930982745686427 \n", + " 0.1324128564961521 " + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "maf_cg10k = maf(cg10k)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, "outputs": [ { "data": { @@ -250,53 +317,45 @@ "([0.00841726 0.124063 … 0.364253 0.5], 0.24536516625042462)" ] }, - "execution_count": 37, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 5 number summary and average MAF (minor allele frequencies)\n", - "quantile(maf, [0.0 .25 .5 .75 1.0]), mean(maf)" + "using Statistics\n", + "Statistics.quantile(maf_cg10k, [0.0 .25 .5 .75 1.0]), mean(maf_cg10k)" ] }, { "cell_type": "code", - "execution_count": 38, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 15, + "metadata": {}, "outputs": [ { "data": { - "text/html": [ - "" - ] + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAH0CAIAAABuMsSDAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deXxTdb7/8W/SJd2g0MpaoQwglEVEATdQRNSrYCmI1xkEkRFnUFyBhyOOiAsMUJhRAcfLD6UjOoNAK6UoDIvQCoIbiqAIVBSklJaldG+znZzfH7nmdpKT0CU9Sfp9PR/8Eb6fnHM++eacvJuTzaCqqgAAQFbGQDcAAEAgEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpteQgPHXqVGlpqX/Xabfb+VI6IYTD4VAUJdBdBAWbzRboFoKCoigOhyPQXQSeqqp2uz3QXQSFEHq0bMlB+Pzzz3/00Uf+XWdZWRl7uRDCbDZXVVUFuougcOHChUC3EBQqKystFkuguwg8m81WXl4e6C6CwsWLF0Plb6OWHIQAAFwSQQgAkBpBCACQGkEIAJAaQQgAkBpBCACQGkEIAJAaQQgAkBpBCACQGkEIAJBaeKAbQOCZzWZvX44VHh4eGxurcz8AoCeCUBZnzpzJz8/3HDebzQ89NqPk7BnPkupQBlx19f7P9jR/dwAQMAShLIbflXbOFmEMi3AbV6y1lYUF4g2tn+n48VNL3lw9mgOAwCEIZVFccLLqhYOiVTv3wtl8Me+6QHQEAEGBN8sAAKRGEKIxvv322w5JXYxeLF26NNANAkB9cWoUjfH1118br/2tOuENz5Ip65lQ+VlqABAEIZrEwBkFACGPBzIAgNR4RggAQWThwoXr168PdBd+oChKWFhYIxbs3bv32rVr/d6PDyEfhAcOHBg2bFh1dXWgG2mJqkryj/8U1/Yyz4rdYlYGjde/I6DFKygouOuuu/77v/870I0ERn5+/oIFC3TeaGgHYXFx8csvv1xTUxPoRlqosjP2mETrrO0apX8+3pi/9ADUw+WXX3711VcHuovAMBoD8IJdCL9GaDabp02btmzZskA30pIZjGEi7jKNf+GmQLcGAP4RwkH4xBNPzJgxo2vXroFuBAAQwkL41GhmZubbb7/tvGwwGGpra6OiogLbEoQQQrFt+mjL6aJiz4pBiD8+PPWKK67QvykA8CaEg7CsrMx5wWAw8Anu4GH/+avcVh1zj7fyLEV9/s7tI28lCAEEFf+fGlUUJSUlRbM0dOhQw68eeeSRhq6ntLQ0NTU1ISFhzJgxpaVav5aAINFnhLjrT57/TO27BbozAHDn52eES5cuXbNmzbFjxzxLqqoePXr09OnT8fHxQojwcF+b1lxPenp6cnJyVlbWrFmzFi9evHDhQteaNVdis9l27NjheuLocu2111555ZX1v1F11dbWRkRERES4/5hRkMjds+/NlW9plsxms5831ogn4apqsVha2Lt8a2trW9gtapza2lpFUTg3Y7Vam7hL2O12P/YTilRVbcQEmkymxn1sUfg9CAcMGNCjR4/U1FTP0tmzZ61Wa1pa2rFjx0aOHLly5UrnS3p2u90Viq7LmuvJzs7OyckxmUyPP/54WlqaKwi9URTl5MmTnlPTpUsXb89ZL8lms9lstsYtq4PZL7z0ffvholNvjZryoe7taLDb7cE8gY0Q5LuEbmw2m9FoZCpsv2r0GhwOh9vIa0uX/TMrp2l91ZdRiBeeeWrMmDH6bE6TqqqNmMCIiIhgCcIRI0Z4KxUXFw8ZMuTVV1/t2rXrjBkznnrqqffff18IMW3atFGjRo0fPz49Pd1iscydO9fbegoLC5OTk4UQycnJRUVFl2wmKirqD3/4wwMPPND42+PBZrPFx8cH7TNCY1iY6Hur6HWzRu3dS5yLbjBDIxYxxMbGOk8JtBg1NTUt7BY1jqqqJpMpOjo60I0EmNVqNRqNTdklIiMj3UYOHT76jamvuOruprVWv63nvXH69GkdNuRDEyewEfR7s8zAgQN37drlvLxo0aJ+/fo5Ly9ZsuTOO+/MyckpKSnZsGGDjzWoqmowGJwXFEVp7oYBIFh07CX63KrDdsIOZmuOGwyGCRMmrFmzxjUyceLENWvWuE6G2+32pKSkhISEH374wflA7VzK7VlacJ741e9zhN98882+ffuclyMjI02m//1EdkJCwrhx49atWzdlyhTXoKbOnTsXFBQIIQoLC5OSkpq7YQCAS15ensVicV62Wq15eXl1q5988slll1124cKFw4cP1x23/yfdum0QPZ4R5uXl3XLLLdXV1ffee29eXl6PHj3mzZs3duxYZ3XFihW5ubn5+fljx46NjY0dNWqUt/WkpqZmZGQsWLAgIyMjLS1Nh87hXw6bedU/3tmZm+dZMhrE7Gef5RwjELSGDRu2Y8eOu+++Wwixa9euoUOHZmZmuqrr169/8MEHT5w4kZmZ2b9//8C12Rh6BOGIESNUVR02bNiLL76YmppaXl4+atSo5cuXO6tdunTZuHFjTEzMli1bNN9u6jJ37tyJEyd26dLlmmuuee+993ToHP5VU3xqfZsewqLxMlLk9r9Nf/RRghAIWvfcc88HH3zgDMLs7Oxx48a5gtBms23cuPGbb7758ccfH3300Zdeesl1djQkNEsQur2F2vlfg8Ewffr06dOnu1159OjRzgudOnXq1KmTj/W0adNm8+bN/m8Xehpyn+h3u+dwxKcr9e8FQP2NHj161qxZNpstLCxs69atixcvdpV27do1YMCApKSkjh07lpeXHz582PWksG4ivvXWWw8//LDefddDCH+zDFoSi0MMGHKD5hfPt2kV99OxH/RvCUBdrVq1GjRoUF5eXkxMTP/+/euev1m/fv3HH3/syry6Z0dD4qOlBCGCglJdXjbz3yK+k3vBUl254DrXiXQ3HTp0uO+++5q9OQBCiF/PjsbFxY0bN841aLVas7Ozf/nlF+dPIGzbtu3pp58OrbOjBCGCRnxnkXC5+2B5sU1x/OnDo55Xd1Sc+435F4IQ0M2YMWPmzJkTExPzpz/9yTW4Y8eOrl27un4IaPjw4adOnap7djT4EYQIcqpQhfm+1zUqJ75SNz+tez+AvBISEvr27WuxWNq3b+8aXL9+fd1vAYuKiho5cmRovXeUIAQAXILrpb7t27e7Da5evdrtyps2bXJbKsgRhAhZ5srTxee79blKszhhfNrC+a/o3BHQHBISE0y73o345XMdtmX/+SshBuiwoaBCECJkXSwwOwy/3JehUTr44Y8nTurdD9A8Frz0wtX9Gvk7AQ1317XXXqvXtoIFQYhQFmESXbT+ei34VlhO6t0M0DxMJtOkSZMC3UVLpt93jQIAEIQIQgCA1AhCAIDUeI0QLdMvPx9ftmyZZmno0KGDBg3SuR8AQYsgREv085cHTp777qN8z4rj+GfzamsJQgAuBCFaIsVmSL7act9rnpXwDX/Wvx0AwYwgBIAgEh0dnZmZefSoxvfryuDChQv6b5QgBIAgMn/+/LfeeivQXfhBZWVlbGys5m+r+dCzZ8+631yqD4IQAIJIdHT0k08+Gegu/ODcuXOJiYlhYWGBbuTSCEJIx2az1dTUaJZMJlNIHLcA/IggDD2Kohw/flyzZLFYdG4m5Kjnfp770qaX5y/wLDls1uzsDWPGjNG/KwABRBCGntf+vuKFl+ZFxMZ7lqpKz4vb9O8olKhlZ9SxL9vvfMazFL/yHv37ARBwBGHoOXToO/PoF8zD/+hZCnu2m6J/QwAQyviKNQCA1AhCAIDUODUK/C/FUpO+9O/vrc/2LEUYxet/TW/fvr3+XQFobgQh8L/MxSf2tRstIq7yLEVtfP6Vuc8ThECLRBACdfS5VVx1t+dw+PZF+vcCQB+8RggAkBpBCACQGkEIAJAarxECl6Yq9ry8vPx8jV/6LSsru//++/VvCYC/EITApdWazU+/tjo8Ks6jolYe3EkQAiGNIAQuTbWZqye+Jdr3cC8oNuNjGl/6CiCEEIRAU50+fVpzPDIyko8eAsGPIASawKE4TLG9r7nes6LabXGxsWPHaHwqUQhx45BBU6Y82MzNAagXghBoAtUhaitrlp7XKO3PMv/ribcu/kajdOrA6eJ/E4RAkCAIgeZiiI5XRz6uUfhqvbjwoe7tANDG5wgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFLjc4RAACh2e3l5uWYpKirKZDLp3A8gM4IQ0N2Fkzs/3tE+qatnRbFa5s7589y5cz1LFRUV27dv97bKO+64o3Xr1v5sEpAGQQjorrzI0Hu4dfoHGqXsuVu2f1xSVuFZ2Z736clKR+RlXTxL5u8+/uaLff369fN7p4AMCEIgmPy494vI6C8K2nlWjCcLHPf9zTx4vGep9fyBzd8Z0GIRhECQ6XG9+K+ZnsPGT1c59G8GkADvGgUASI0gBABIjVOjQMiz2mwTpj4aHR3tWYoKE3k7thoMBv27AkIFQQiEPEtFyXe9ZohEjc9jiKWpS5Ys0QzCxMTEhx56qNmbA4IeQQi0CD1vFJ37aowbjHNyz3oOq9UX2xd+RhACgiAEWjiHYrtngTB4vBug+Jj4x72BaAgIOgRhkCorK3v4iZnm2lrP0sGDh8T1V+rfEgC0SARhkHr33Xc3HSy0Xf+AZ8lY8aX+/QBAS0UQBi9jxyvEtb/1HDdsW6J/MwDQUhGEgKSs5tr33ntPs9S1a9fhw4fr3A8QKAQhIKXS0yVlFdPf2uZZsZcUDP1NAkEIeRCEgJTsNmGKrZr8D43SgRzHyX/p3hAQMHzFGgBAagQhAEBqBCEAP1jxj/diYmIMXnz99deBbhDwitcIAfjB7j2fiqnviOvv9yzFp1+nfz9A/fGMEAAgNZ4RAmgAVVVVVdWseF1Ese/evbuwsFCzevfddxuN/EWOQCIIAdTXnj17brllhEN1aNRMsWLiaM2lqivKnl/5QWSrtp6lih8+7ZPSx6AVhL+5vNOHGzKb1i9QLwQhgP9UW3Hg+x/6DRnqWblQdFq9c5YYN9+zFP7iALuX9ak2c+29f6tNvkajNi3mh1teEuGR7uPlZ4s3PduwtoHGIggB/Keio5Xh8T+MXKBRWv+MQfu8aBP0uF5ERLsPlvzi560A3hGEANwZYtqInjdqFKLjde8FaHYEIYDgY60pr7UmdOqiWbxj5K1r/7la547QghGEAIJPdalis5bO/kqj9P2/d+554+b/StVc7sH7xk2d+lDz9oYWhyAEEJQMRpFwucZ4+dmLNuOeXlM1St/mXPX94ebuCy0PQQggxBjiEsRVWh/VOJsvxFnd20HI43OsAACpEYQAAKkRhAAAqfEaIQD4044dO6qqqjRL/fr1u+GGG3TuB5dEEAKANrvdbrFYNEtGozE62uMLcYT47LPPpj49W/S5VWNthUcmjxxEEAYhghCAFGw2m+b4x3l7Vry9SgiDZ2nv51+Wnyv0LKiqGh8ff+FskeciFoslvF23ion/T2NLu948eWLb2rVrNdsYPHhwz549fd2AhqioqPD2rDQyMvKyyy7z14ZaBoIQQMu3YtXq6X+cajBopJ3DFCdGPi7aa4XQmSzxtwIR08Z9vLzItqjhvzZ88uvcn499sTzbs2I78fVfn5/hryCsqam58tphFy5c8Cw57LY+vXt988Vev2yoxSAIAbR8ebt3qw+8qQ77vWfJ+Eyyo9/toofWGcvV0/zZhLVGpNxaOelNz0rUuif9uB2z2Xz2zGnLq1ofqTy+z7Lzz37cVstAEAJoKSrO/SN37foPNmpUqqrEmIY/h5OM1Wr18qvLIjw8PCwsTOd+dEMQAmgpio/W9r6terTGM56wN9L0bye0fPXVVzffervNUutZUhVlyZLFM2fO1L8rfRCEAFqQ6HjRrrvnsCHMz491DsWel5fnOX7o0CGHQ/HvtvTx7bffGoeMV+5f4VmKzHpG/370RBACQAPVlFVbbOOeeNGzYr5w2hadoE8XP/744xdffOE5XlVVFaJhHCgEIQA0kN0qhCh76mON0vbXwvZn6tPFiLvSyi7rY4iIchtXzVU2m12fHs6fP5+Tk6NZqqysfOihh+LjQ+DHnAlCAAgw1Wb+1/oPvv7uB89SmEH8aebTV1xxhWepsuxi9aOvi/iO7oXzP4tD13jblmK3FxVpfAKyvLzcyxtlfJn4h8f2nCg3JiR5lmxfrE9LSyMIAQCXZj/57Wften5W2duzFJ23/P777tUMwsYoO3P8+PGe/TVi0lpdLgaNb+j6KioqzCOeEP3/y7MU84PWM+agRBACQBDodZMY8ajnsOm7Df7cStVFtW1SzdyvNUoZv/eWB46Lp2fPefv5F1/xLFlFuBjgzwYDgiAEgOCl2m27du0qLi72LNns2l8a5/8eSk7Z73zWdssjnqWweYP16aFZEYQAELyqS4qXfLA7MvdHz1JtrVm/PiKiNL5qTgjN72gNOQQhAAQv1W6xjJpjSblFo7ZfM5nQYPwwLwBAajwjDKSKioppTz9bUHDKs1RUcFJJuln/lgBANgRhIH3xxRcffvJZ9eiXNWon08Ma8aEeAEADEYQBFt7qMjFglEZh72rdewEAGfEaIQBAagQhAEBqBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAanyO0G9KSko0vyFeCBEREdGrVy+d+wEA1AdB6DdDb7/7zIUyg9H9Sbaq2CNtVReKCwPSFQDAN4LQb07mH7YsPCGiW7sXys60WXJjIDoCAFwarxECAKRGEAIApEYQAgCkRhACAKTGm2X0oNhtH3zwgef4wYMHFcWufz8AABeCsPlVlVTVWh5a8i/PiuVMvt0Up39HAAAXgrD5KTZhCKt4eJ1G6d9Lwg5t0r0hAMD/4TVCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1EI+CA8cOBAbGxvoLgAAoSq0g7C4uPjll1+uqakJdCMAgFAVwkFoNpunTZu2bNmyQDcCAAhhIRyETzzxxIwZM7p27RroRgAAISyEgzAzM3PEiBEGg0EIYTAYzGZzoDsCAISeEP6u0bKyMucFg8GgqmpgmwEAhCj/PyNUFCUlJUWzVFpampqampCQMGbMmNLS0oaup0GLAwBQH35+Rrh06dI1a9YcO3ZMs5qenp6cnJyVlTVr1qzFixcvXLiwQevxtri3p4O1tbVvvPHGRx995DY+fvz422+/vQG3qo6ysjKHwxEREaFRa9yz0tB9KtuYzvW7td7uDR8dNENz/l2ljrOnyyJ6rtLn0Rncu3IQ8HnKzXtJVcvLy3V70hIXF6f9yFwPfg7CAQMG9OjRIzU1VbOanZ2dk5NjMpkef/zxtLQ0Z5LZ7fbw8P9tw3VZcz2ai/sQERExYMCAG264wW38iiuuiI6ObtwNjI6Ojo6O9jLdhsassVEL4ZIMXibWoOtDGPduSGjM3aTvjhRgBm+H06VERUU1+sG2oYzGxp/g9HMQjhgxwke1sLAwOTlZCJGcnFxUVOQcnDZt2qhRo8aPH5+enm6xWObOnettPZqL+xAeHn7zzTc/8MADjbgh3phMpqioKO0glO1Br5GPHsErqJsTQs8GG7GlZmjOn6ts7CM5Lsn7zBoMzgdMHZtpJF3fLKOqqvMvC1VVFUVxDi5ZsuTOO+/MyckpKSnZsGFDQxcHAKApdP34ROfOnQsKCoQQhYWFSUlJzsGEhIRx48atW7duypQpJpOpoYsDANAUegRhXl6e80JqampGRoaqqhkZGWlpac7BFStW5Obm5ufnL1iwYMuWLT7Wo7k4AABNoUcQul7wmzt37qFDh7p06XL48OE5c+Y4B7t06bJx48bk5OQtW7bExMT4WI/m4gAANEWzvEbo9l5b13/btGmzefNmtyuPHj3aeaFTp06dOnXysR7NxQEAaIoQ/oo1AACajiAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASC2Eg9DhcEyZMmXw4MEpKSmZmZmBbgcAEJJCOAh37Nhhs9n279+fmZk5ffr0QLcDAAhJIRyEV1111aJFi8xmc0FBQadOnQLdDgAgJIUHuoHG69ixoxDi7rvv3rx58/bt2wPdDgAgJIXwM0JFURwOR05OTlZW1iOPPBLodgAAISmEg/DNN9989tlnw8LCevToUVNTE+h2AAAhyc9BOHToUMOvPJ+llZaWpqamJiQkjBkzprS01PeqFEVJSUnxsfjkyZN/+umnAQMGTJgwYeXKlf69IQAASfjzNUJVVY8ePXr69On4+HghRHi4+8rT09OTk5OzsrJmzZq1ePHihQsXelvV0qVL16xZc+zYMd+Lb9iwwUc/VVVVM2bMeP75593Gn3766UmTJjXstv3q4sWLFoslIiJCo6aq3pfzXvKxUMP5aqFxK2xkzZ/LNI63qfB1J/m/O69rVBuzMR1nr+Hbaobm/LlKn/Md+F1Zv7vWJ29t+Nxdfe3kJSUlcXFxTeyqnuLj400mU+OW9WcQnj171mq1pqWlHTt2bOTIkStXroyKiqp7hezs7JycHJPJ9Pjjj6elpTmD0G63uyLTdXnAgAE9evRITU295OI+xMbGzp49e+zYsW7jCQkJzqhuBFVV4+PjtYPQYPC+nPeSj4UazlcLjVthI2v+XKZxvE2FwceB6//uvK7R0Ji7SsfZ03Fb3vmzB5/z3chd2Y/pFQzTLby34XN39bWTt23bNjExsYld1ZPR2PgTnP4MwuLi4iFDhrz66qtdu3adMWPGU0899f7779e9QmFhYXJyshAiOTm5qKjIOTht2rRRo0aNHz8+PT3dYrHMnTtXCDFixAjP9Wsu7oPBYOjQoUPPnj2bftNcwn7lx3UiSATJg1GIYvbgyWg0hsSjpT+DcODAgbt27XJeXrRoUb9+/dyuoKqq8y8LVVUVRXEOLlmy5M4778zJySkpKfF9qlNzcQAAmsKfb5b55ptv9u3b57wcGRnpebq2c+fOBQUFQojCwsKkpCTnYEJCwrhx49atWzdlyhTfZ3g1FwcAoCn8GYTV1dXjxo07cuSI1WqdN2+e68W5vLw854XU1NSMjAxVVTMyMtLS0pyDK1asyM3Nzc/PX7BgwZYtW3ysX3NxAACawp9BOGzYsBdffDE1NTUpKam0tDQ9Pd057nrBb+7cuYcOHerSpcvhw4fnzJnjHOzSpcvGjRuTk5O3bNkSExPjY/2aiwMA0BT+fI3QYDBMnz7d8/uvXW+9bdOmzebNm92qo0ePdl7o1KmT21eGur1nV3NxAACaIoS/WQYAgKYjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSCw90AyFm45btGe/+02gweJZsNpv+/QAAmoggbJj0v712POU+0aGXRi3vY93bAQA0FUHYcD1uEN2vC3QTAAD/4DVCAIDUQjsIS0tLU1NTExISxowZU1paGuh2AAChJ7SDMD09PTk5uaioqGvXrosXLw50OwCA0BParxFmZ2fn5OSYTKbHH388LS1t4cKFdauqqpaUlJw6dcptqfj4+Li4uEZuUlVF5QVRdka7Wl4sLBWAbDcAABFESURBVFXug5XnhXBoL2KuFHabdslWKyzV2iWHImrKNEuqKkRVidf2Ks4KxeOtrRXnhapqL1Jbripe2rPWCDVSu6TYRW25dkl1iOqLXtpTRcVZYfT4y6zinBBe2qspE4rdW3uqrVZ7ihSrqK30Mns+2hOi4qyIjPbYkFkIob1I9UWhKtolS7VqM2uX7BZhqfJ6v/tq75yIaaNdKisSnu9zri7x3l6lard4b097t1Qdiqgp9dpe5XmvpbIiERHlPlh1Xqg+jhqrl6PG6+yp3o8aoTp8HTXlxcJa4z5Y6eOoqRDejhpbrbCavLTn46hRvbanClFxVqgO9/GKs16Pmtpyr+1Za1WjwctRY/N21Pg4qK12x4BB1xoMGk+3WsXGFJz8SWNtTWA0Gg1a7+evD4Oqqv7tRk9xcXHnz5+Pjo6ura3t0KFDRUVF3eq4ceNyc3Ojo90fvGbOnPnggw82bovj7pvw/eEfhNZsV1VUxLVq7VlSHY6aqqrY1q09F1FsVqvVFh0b61mymWsdwmiKMnmWLLW1xrDwiMgIz1JtTW1kZGRYeJhnqbqyMiYuznNHUVVHTWVlbOt4jfbsNqvF6q09RTVERXs8fglhqa0xhkVotmeuqY3w1l5VVUxMrMHo2Z5aXVEeF6/xEK/YbRazJUbrbxqbxexwCJNWe9baGhEWHhkZqdFebW14RGS4dnvVMTExnu0JIarKSuPatPUcdyh2c61Zsz271WJXHFEee6YQwlpTLcIiIk1e2guPCI/Q+OO1promKirKGKbxiFNVVhoX39Zzt3QoirmmJqZVK+327EpUTIxGe7U1whjurb2w8PCICI373Vd75eVxreM12nMotdXVsa00jhq7zWq32bXbM9eoIkzzqPHRXm1NTaQpKkyrveqKilivB7WXo8Zqtdp0PKhj47SOGp8HtdkaHafVnsXsUIUpSvOoqTV4ac9cY46IjPDSXlVMrMZBLYSIi4468NUXnuNNER8fbzJpzG19hPYzQlVVnY/sqqoqiuJWjYuLW758+QMPPODHLeZkrYuPj9c8nKRSU1Njs9ni4zWONNkUFRV16tQp0F0EXllZmclk8vy7UzZWq7WysjIxMTHQjQTeuXPnEhMTw8I0MjLYhPZrhJ07dy4oKBBCFBYWJiUlBbodAEDoCe0gTE1NzcjIUFU1IyMjLS0t0O0AAEJPaAfh3LlzDx061KVLl8OHD8+ZMyfQ7QAAQk9ov0bYpk2bzZs3B7oLAEAIC+1nhPr7y1/+cvz48UB3EXi5ubnvvPNOoLsIPEVRHn300UB3ERRWrly5d+/eQHcReIcPH16yZEmguwgKzz777IULFwLdRb0QhA2zZ8+ekpKSQHcReD///PO3334b6C4Cz+FwcE7Caf/+/Z6f2ZXQ+fPn+YPAaceOHdXV1YHuol4IQgCA1AhCAIDUCEIAgNQIQgCA1EL74xO+Wa3WTZs2Ob96xl+Ki4vffffd3bt3+3Gdoeizzz47c+bMggULAt1IgCmK4nA4mAchxNGjRxVFKS4uDnQjAfbjjz8WFBSwSwghKisr33zzzYSEBH02d++99/bq1atxy4b2l277dv78+VdffbXR30euyeFwGD1/IUE+zt3Gv3MbotglnNglXNglnHSeh0mTJvXt27dxy7bkIAQA4JL4swUAIDWCEAAgNYIQACA1ghAAIDWC0JfS0tLU1NSEhIQxY8aUlpY2qNrCXPLGKoqSkpKif2N68j0JOTk5/fv3b9Omzc0335yfnx+QDnXjeyq2bt3at2/fNm3a9O3bd/v27QHpUB/1eRD4/vvvY2NjdW5Mf76nYujQoYZfPfLIIwHp0AeC0Jf09PTk5OSioqKuXbsuXry4QdUWxveNXbp06Y033njs2LGA9KYbH5Nw6tSpSZMmvfXWW0VFRWPGjPn9738fqCb14WMqHA7HxIkTly9ffvHixVdeeaVlT8UlHwTKy8unTJlSU1Ojf2868zEVqqoePXr09OnTlZWVlZWVr7/+eqCa9EqFd7169Tpy5IiqqkeOHOnVq1eDqi2M7xu7a9euDz/8sMXvTj4mITc39+GHH3ZePnfuXGJiYgD605GPqbBYLJs3b3Y4HBUVFZs2berbt2+AetSD7+PC4XCMHTs2MzOzxR8aqs+pKCoqiouLGzRoUFxcXFpa2tmzZwPUo1ct/+5pitjY2JqaGlVVa2pqWrVq1aBqC1OfG9vij/b6TILdbn/kkUemT5+ub2t6u+RUVFZWCiEMBsPevXt1704/vudh4cKFs2bNUiU4NFSfU3HgwIERI0YcOHCgpKRk8uTJv/vd7wLUo1ecGvVFVVXnN2WoqqooSoOqLYxUN9abS07Cxx9/PGTIkPj4+KVLl+rena4uORVxcXFVVVXz589/6qmndO9OPz7mITc3d+vWrQsXLgxQa3rzMRUDBw7ctWvXwIEDExISFi1atG3btgD16BVB6Evnzp2dX1VaWFiYlJTUoGoLI9WN9cbHJKiq+txzz73yyitr165dtGhReHhL/hZf4XMqTp48+cwzzwghYmNjp06deuTIkcC0qAsf87Bz585PPvkkMjLSGQ8Gg+HTTz8NTJe68DEV33zzzb59+5yXIyMjTSZTAPrziSD0JTU1NSMjQ1XVjIyMtLQ052BeXp6Pakvleyok4WMS9u3bl52dvWnTps6dO1dVVVVVVQWy0ebnYyo6d+68atWqTz75RFXVdevWXX311YFstJn5mIf58+e7zrwJIVRVHTZsWABbbW4+pqK6unrcuHFHjhyxWq3z5s0bO3ZsIBvVpMPp19BVWlo6atSopKSk1NTUsrIy56Br0jSrLZXvqdD8b8vjYxLmz58v1ZHle3/Iy8u75ppr2rZte8MNNzjfQNFS1ee40BxpeXxMhcPh+Pvf/96jR4/LLrts8uTJ5eXlAe1UA1+6DQCQGqdGAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgRpAwGg8lkKisrcxsvKyszmUzOHzt1Xu3o0aPN3Uz//v0NBsOqVas8m3RtvZ6dNKJhgxcNWklosVqtQ4YMOXfunKj3nuDi487yNo1nz54dPHiw1WptthuEoNbCf0cbIc1qtW7YsOGhhx6qO5idna3zA9bhw4cPHz4cGxubmZk5depUPTftsnz58ssvvzwgmw6I5cuX33bbbe3bt3f+t/57gu87y9s0dujQ4dZbb33jjTdmzpzpvxuB0BHoH0QEtAkhrr766pEjR7qN33777c4fPXf+t7KyUlGUZu3khRdeiI+P/8tf/hIWFnbhwgW3Jl0/PFv3sg/1vFoTFwlpNputS5cuJ06ccP63nnuCUz3vLE8//fRT165d7Xa7f24DQgqnRhG87r///l27dhUVFblGzp07t3PnzgkTJrhG4uLijMZm3I1VVV23bt299957//33K4qSnZ3dfNuC07Zt2y6//PJu3bq5RuqzJ4im3Vndu3fv1KnTtm3bmtw+Qg9BiOB1xx13tG3bdt26da6RrKys1q1b33HHHa4Rt1fpvvrqq9/+9rcJCQndu3evu6DD4Xj99df79esXFxc3aNCgrKysumv46quvUlNTr7vuOs8eDh48mJ+fP2HChG7dug0ePDgzM7M+nauqumLFij59+sTExFx99dWrV69WVbXRV/PNrX8f6/zyyy9HjhzZtm3b7t27/+EPf/j88881X2o9evRo3RfefKzQx4Srqrps2bK+ffu2bt36pptu+vTTT4UQc+bM+c1vfuNa/NSpU0ajMScnx+0Wbdu2bfjw4XVH6rMniMbeWS7Dhw/fvn17gxZBCxGYJ6LApQghjhw58sc//nHIkCGuwWHDhk2dOvXIkSOuXVf858nJ6667bsWKFfv27Zs8eXJkZGRVVZWztGTJkri4uMWLF2/ZsmX27NlhYWEffviha6nBgwdPnTr1X//6l2cbs2fP7tixo/OM2aJFi8LCws6fP+/WpOfld999t2/fvqtXr/73v//t3Nz//M//NOhqblOxefPmI//JbDZr9u9tnfv37w8LCxs7duyaNWsyMzMnTJiQmJioOY11p/eSt8XbhC9dujQ+Pn7ZsmVZWVnjxo0LCws7ePDg999/L4T4/PPPnddZsGBBhw4drFar2+295pprsrKy3Cb5kntCfe4sH9Ooquq6desGDRrkOf9o8QhCBCnnw19eXp4QIj8/X1XVU6dOCSE+/vhjH0H42muvOS9XVFS4Sg6HIzEx0flUxmn27Nk33XSTa6lnnnlGsweHw9G9e/cnn3zS+d/jx48LIVauXOnWpOfla6655vjx466rzZw5c9iwYQ26mttUePruu+80+/e2zrvuumvy5Ml1V+t8L4lnV27p4vu2eJvwjh07rlmzxllSFOWuu+567733VFUdMGDAjBkznNdJSUnRnPl27drt3bu37s2vz55QnzvLxzSqqrp79+727dt79oMWjyBEkHI+/CmKkpSU9PLLL6uqumTJkg4dOthsNh9BuH//frc1qKp69uxZIURJSYmrtHv37sTERNfVdu/erdnDl19+6fnQefvtt3tuwu1ybGys21IdOnRo0NU8p8LHRNXt39s627dvn5ubW3fBXbt21ScIfd8WzQk/f/68EMLtvSpOCxcuTEpKUhTFObc//PCD53XCw8MPHTrkttpL7gkNurM0HTx4MDw83McV0FLx8QkENaPROGHChDVr1rzwwgtr16697777wsN97bSej9reVqsoiuu/7dq107zaunXr2rVr98knn7heM1u9evWSJUvOnz/vbRGnmJiYPXv2REdH+26jnle7pLrNeFun1Wp1+8hdWFiY5tpqamrq36TmhNvtdm/r/93vfvfcc8/t3bs3MzPzhhtu6NOnj+d12rRpU1VV5TZ4yT2h0XeWS2VlZdu2betzTbQwvFkGwe7+++8/duzY+vXrv/76a7d3CdZTu3btEhMTt2zZ4hrZvHlz//79fS/lcDjWr1+flpbWp0+flF89+OCD9Xk7Yr9+/QoLC52L9O7de/ny5atXr2701RrE2zqvvPLKd955p+4116xZU/e/rvz77LPPmthkhw4d2rZtu3PnTud/HQ7H4MGDFyxYIITo1q3bjTfe+M9//vP999/39qHMyy+/vO4bRF187AlNubNcioqKpPqwJv5PoJ+SAtpEnRecevfu3bZt2+TkZIfDof7nuTvh5eSk238XL17cqlWrv/71r5pvltE8Y7Z3714hxObNm+sOOhyOnj173nbbbb63npOT06pVq9deey07O3vy5MlCCM3N+bia5lT4nijf69yxY4cQ4p577lm7dm1WVtbEiRPj4+Nd05icnDxy5MitW7e+8847bh/Oq+dtcfvvggULEhIS3njjja1btz744IORkZGuV+OWL19uNBpjYmIqKio0b9H06dP//Oc/e67Wx57Q0DtL03PPPffYY4/5uAJaKoIQQaruw9Yrr7wihHj22Wed/21EECqK8uqrrzo/AzBw4EDPNyV6NvDkk0+2atWq7rsKnWbNmmU0Gs+dO+d766tWrUpJSYmJiRk0aNAHH3zgbXPeruZtKupT9bbOrVu3Xn/99XFxcd26dZs0adKePXtc07hz586UlJS4uLiRI0f+/PPPbn8i1/O2uE34okWLevbsGRMTM2TIkG3btrmuVlxcbDQap0yZ4u0WZWdn133TUH32hIbeWZqGDRu2ceNGH1dAS2VQG/65JQAtwNGjR/v06aP/I8CJEye6d+++e/fum266SfMKNpstOTn5yy+/1PNE5alTp66//vpffvklIiJCt40iSPAaIQCd2O32ysrKefPmpaSkDBs2zNvVIiIipk+f/vbbb+vZ26pVqx577DFSUE7/Hz0kdvMCJ2OgAAAAAElFTkSuQmCC" }, - "execution_count": 38, + "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "# Pkg.add(\"Plots\")\n", - "# Pkg.add(\"PyPlot\")\n", - "using Plots\n", - "pyplot()\n", + "#using Pkg\n", + "#pkg \"add Plots\"\n", + "#pkg\"add PyPlot\"\n", "\n", - "histogram(maf, xlab = \"Minor Allele Frequency (MAF)\", label = \"MAF\")" + "using Plots\n", + "gr(size=(600,500), html_output_format=:png)\n", + "histogram(maf_cg10k, xlab = \"Minor Allele Frequency (MAF)\", label = \"MAF\")" ] }, { "cell_type": "code", - "execution_count": 39, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 16, + "metadata": {}, "outputs": [ { "data": { @@ -304,7 +363,7 @@ "0.0013128198764010824" ] }, - "execution_count": 39, + "execution_count": 16, "metadata": {}, "output_type": "execute_result" } @@ -316,12 +375,8 @@ }, { "cell_type": "code", - "execution_count": 40, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 17, + "metadata": {}, "outputs": [ { "data": { @@ -329,22 +384,19 @@ "0.07228069619249913" ] }, - "execution_count": 40, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# proportion of rare SNPs with maf < 0.05\n", - "countnz(maf .< 0.05) / length(maf)" + "count(!iszero, maf_cg10k .< 0.05) / length(maf_cg10k)" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Empirical kinship matrix\n", "\n", @@ -353,101 +405,127 @@ }, { "cell_type": "code", - "execution_count": 41, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 18, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "396.943890 seconds (8.43 G allocations: 127.378 GiB, 4.38% gc time)\n" + "568.176660 seconds (2.91 M allocations: 494.981 MiB, 0.40% gc time)\n" ] }, { "data": { "text/plain": [ "6670×6670 Array{Float64,2}:\n", - " 0.503024 0.00335505 -0.000120075 … -5.45185e-5 -0.00278072 \n", - " 0.00335505 0.498958 -0.00195952 0.000868471 0.0034285 \n", - " -0.000120075 -0.00195952 0.493828 0.000174648 -0.000381467\n", - " 0.000923828 -0.00329169 -0.00194166 -0.00223595 -0.00123508 \n", - " -8.39649e-5 -0.00353358 0.0018709 0.00222858 -0.00171176 \n", - " 0.00204208 0.000572952 0.00254025 … 0.000861385 2.99785e-5 \n", - " 0.000569323 0.0024786 -0.00185743 0.00117649 -0.00118027 \n", - " -0.000642144 0.00317992 -0.00099777 0.00354182 -0.000260645\n", - " -0.00102913 -0.00123475 -0.00061138 0.00173885 0.00177727 \n", - " -0.00139442 0.00208423 0.000124525 -0.00145156 -0.001011 \n", - " -0.00204555 0.00011055 -0.000419398 … -0.000198235 -0.00110353 \n", - " 0.000947587 0.00167346 0.00184451 -0.000690143 -0.00304087 \n", - " 0.000322759 -0.000899805 0.00303981 0.000739331 -0.00118835 \n", + " 0.502735 0.00328112 -6.79435e-5 … -6.09591e-5 -0.00277687 \n", + " 0.00328112 0.49807 -0.00195548 0.000884994 0.00341591 \n", + " -6.79435e-5 -0.00195548 0.492348 0.000198191 -0.000337529\n", + " 0.00087876 -0.00322605 -0.00192088 -0.00235314 -0.00124267 \n", + " -5.03227e-5 -0.00352498 0.00184588 0.00219109 -0.00163189 \n", + " 0.00203199 0.000597697 0.00251237 … 0.00088428 2.21226e-5 \n", + " 0.000560862 0.00244517 -0.00183233 0.00120214 -0.00120686 \n", + " -0.000656949 0.00322249 -0.00101472 0.00355832 -0.000240444\n", + " -0.00103874 -0.00125164 -0.000599731 0.00176492 0.00176928 \n", + " -0.00137058 0.00209596 0.000146711 -0.0014453 -0.00103066 \n", + " -0.00209312 0.000140721 -0.000442031 … -0.000214963 -0.00107391 \n", + " 0.000933587 0.00168842 0.00185731 -0.000787908 -0.00311063 \n", + " 0.000334572 -0.000886623 0.00304182 0.000752285 -0.00123334 \n", " ⋮ ⋱ \n", - " 0.00298012 0.00130003 0.000998861 4.18454e-6 0.00303991 \n", - " -0.00207748 0.00274717 -0.00191741 -0.00107073 0.00368267 \n", - " 0.000545569 -0.00244439 -0.00299578 … -0.000669885 0.00221027 \n", - " -0.00423186 -0.00208514 -0.00108833 -0.000622127 -0.000567483\n", - " -0.00325644 -0.000781353 0.0030423 0.000501423 -0.00010267 \n", - " 0.00041055 -0.00200772 0.00274867 -0.00624933 -0.00521365 \n", - " 0.00210519 0.000879889 -0.00107817 -0.000797878 -0.000557352\n", - " -0.00230058 -0.000119132 0.000116817 … 0.000867087 -0.00233512 \n", - " -0.0020119 0.00230772 -0.00128837 0.00194798 -0.00048733 \n", - " -0.000944942 -0.000928073 -0.000175096 0.00126911 -0.00303766 \n", - " -5.45185e-5 0.000868471 0.000174648 0.500829 0.000469478\n", - " -0.00278072 0.0034285 -0.000381467 0.000469478 0.500627 " + " 0.00298109 0.00121739 0.00102822 9.66935e-6 0.00306953 \n", + " -0.00209728 0.00271452 -0.00182325 -0.00108139 0.00366901 \n", + " 0.000549425 -0.00244419 -0.00301368 … -0.000631961 0.00215641 \n", + " -0.00423362 -0.00208073 -0.00107904 -0.000619315 -0.000593852\n", + " -0.00326697 -0.000769552 0.00310511 0.000520658 -0.000113441\n", + " 0.000430563 -0.0020236 0.00265425 -0.00635493 -0.00520252 \n", + " 0.00218746 0.000798767 -0.00105684 -0.000918245 -0.00061484 \n", + " -0.00230525 -0.000101149 0.000117936 … 0.000879829 -0.00233479 \n", + " -0.00201305 0.00233864 -0.00134496 0.00197044 -0.000486275\n", + " -0.000990534 -0.000924159 -9.12302e-5 0.00122311 -0.00298296 \n", + " -6.09591e-5 0.000884994 0.000198191 0.499289 0.000481492\n", + " -0.00277687 0.00341591 -0.000337529 0.000481492 0.499799 " ] }, - "execution_count": 41, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "# GRM using SNPs with maf > 0.01 (default) (~10 mins on my laptop)\n", - "srand(123)\n", + "## GRM using SNPs with maf > 0.01 (default) (~10 mins on my laptop)\n", + "using Random \n", + "Random.seed!(123)\n", "@time Φgrm = grm(cg10k; method = :GRM)" ] }, { "cell_type": "markdown", - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Phenotypes" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "Read in the phenotype data and compute descriptive statistics." ] }, { "cell_type": "code", - "execution_count": 42, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 20, + "metadata": {}, "outputs": [ { "data": { "text/html": [ - "
Trait1Trait2Trait3Trait4Trait5Trait6Trait7Trait8Trait9Trait10Trait11Trait12Trait13
1-1.81573145026234-0.946150461472831.11363077580442-2.098671211191590.7444166141117480.001391718840801310.934732480409667-1.226773154181031.1160784277875-0.44362803350290.824465656443384-1.02852542216546-0.394049201727681
2-1.244400943787290.1096599925471790.467119394241789-1.621313040975891.05667583556830.9789469794191811.000146339460470.324874271402281.162321752196962.69227069487053.082636724610471.090649547860130.0256616415357438
31.455669145023051.538669329232431.094029593765550.586655272226893-0.32796454430367-0.30337709778827-0.0334354881314741-0.464463064285437-0.3319396273436-0.486839089635991-1.10648681564373-1.42015780427231-0.687463456644413
4-0.7688092766985480.5134908855142490.244263028382142-1.317402544756911.193937743268451.173441277342881.087374266752320.5360225837322610.8027592407620680.2341594117498150.394174866891074-0.7673658924760290.0635385761884935
5-0.264415132547719-0.348240421825694-0.02390650834136060.004739158022449481.256191917121931.20388836676311.298007390426270.3101136602473110.6261598610593520.8992891298312240.549967833508120.5406878095420480.179675416046033
6-1.37617270917293-1.471919677445640.291179894254146-0.803110740704731-0.264239977442213-0.260573027836772-0.165372266287781-0.2192572941183621.04702422290318-0.09858155346164820.9473934380684480.5940148120314380.245407436348479
70.1009416296374-0.191615722103455-0.5674213215966770.378571487240382-0.246656179817904-0.6088107500538580.189081058215596-1.27077787326519-0.4524761991439650.7025628772977240.3326362189571790.00269165036261810.317117176705358
8-0.3198182763674641.357744806572830.818689545938528-1.155655316443520.634483681022590.2914619086346790.933323714954726-0.7410832896824920.647477683507572-0.9708776270779660.2208611654113040.852512250237764-0.225904624283945
9-0.2883341733420320.5660825380907520.254958336116175-0.6525783028697140.6689215592773470.9783091991705580.1228629660419381.47909263782140.06721324241734490.07959039175278270.1675324552432320.2469155794421390.539932616458363
10-1.15759732583991-0.781198583545165-0.595807759833517-1.005549802604020.7898288859333210.5710584133790440.951304176233755-0.2959629829848160.990420024797070.5613093669889830.733100030623233-1.73467772245684-1.35278484330654
110.7405691504590311.408738467554150.7346899994400880.0208322841295094-0.337440968561619-0.458304040611395-0.142582512772326-0.580392297464107-0.684684998101516-0.00785381461893456-0.712244337518008-0.313345561230878-0.345419463162219
12-0.6758924864549950.2798926138296820.267915996308248-1.041036653929850.9107417156458880.8660276185131711.074144317020050.03817510035383020.766355377018601-0.340118016143495-0.8090139585050590.548521663785885-0.0201828675962336
13-0.795410435603455-0.6999899397627380.3991295030063-0.5104762619007361.515522454168441.287430329394671.537723932509030.1339891601177021.020257368860370.499018733899186-0.36948273277931-1.10153460436318-0.598132438886619
14-0.193483122930324-0.286021160323518-0.6914942252629950.01315816787006991.523374706867821.40106380722621.531146204518960.3330664834780751.043724803810990.163206783570466-0.422883765001728-0.383527976713573-0.489221907788158
150.1512462033797182.091851089936142.03800472474384-1.124747171435311.665570243907131.625356751095761.587510704836550.6358521860437760.8425777846059790.450761870778952-1.39479033623028-0.5609841075677680.289349776549287
16-0.4646087408127120.361276947723031.2327673928287-0.8260337310863831.434752247099831.744518238188460.2110968874846382.648164251405481.025114331460960.119757316031840.0596832073448267-0.631231612661616-0.207878671782927
17-0.732977488012215-0.5262234258897790.61657871336593-0.554479743325930.9474848590251040.9368332141381730.9725168063355240.2902510138652271.012853597257230.516207422283291-0.03006891719881940.87873225245830.450254629309513
18-0.1673264596221190.1753271654872370.287467725892572-0.4026525320842460.5511815094180560.5222047432909750.4368376600946530.2995649338455790.583109520896067-0.704415820005353-0.730810367994577-1.95140580379896-0.933504665700164
191.411594857874181.787224079010170.843976395853640.481278083772991-0.0887673728508268-0.499577574268580.304195897924847-1.23884208383369-0.153475724036624-0.8704861027883290.0955473331150403-0.983708050882817-0.3563445644514
20-1.42997091652825-0.4901470450342130.272730237607695-1.610299929541530.9907878171977480.7116875326081841.1885836012715-0.3712291880756381.24703459239952-0.03891623322715160.8834957490728722.589880263210173.33539552370368
21-0.1472472881767650.123284304156520.617549051912237-0.187130771782620.2564381075866940.177949837350830.412611806463263-0.2448091245597370.09476248061364920.723017223849532-0.6839483546334360.0873751276309269-0.262209652750371
22-0.187112676773894-0.270777264595619-1.015568185516060.06028505686002330.2724197577579780.869133161879197-0.6575194614142342.32388522018189-0.9999360115250341.446718441783060.971157886040772-0.358747904241515-0.439657942096136
23-1.82434047163768-0.9334804460680671.29474003766977-1.945452211510360.335846511896540.3592016543028440.513652924365886-0.0731976966969581.571390428120051.533293713267281.820768218595282.227403018678291.50063347195857
24-2.29344084351335-2.491618423444180.40383988742336-2.364880747529481.41052548319561.422441171477921.170241662721720.844766501768551.790268754324950.648181858970515-0.0857231057403538-1.027895352926170.491288088952859
25-0.4341359328883050.7408819890346520.699576357578518-1.024055431877750.7595292239837130.9566561108952880.6332995686565890.7707339322685160.8249885117145261.842874376347691.91045942063443-0.5023172078693660.132670133448219
26-2.1920969546557-2.494656642722710.354854763893431-1.931558486357140.9419794002899380.9789171014141060.8948600972897360.4632394028318731.125371333171631.705284461919550.7177927144791230.6458880491082610.783968250169388
27-1.46602269088422-1.249216771018970.307977693653039-1.550973646609890.6189084944747980.6625081716620420.4759571739060780.4847186745977070.4015648920282490.55987973254026-0.376938143754217-0.9339826292282180.390013151672955
28-1.83317744236881-1.532687878287012.55674262685865-1.518277457838350.7894096017464550.9087477997285880.6499719229414790.6683736499316671.200583035199030.2779632560756371.25049531982753.313704450716382.22035828885342
29-0.7845466282431780.2765825795439313.01104958800057-1.119788432067580.9208238584227070.7502176898861511.26153730009639-0.4033638829224170.400667296857811-0.217597941303479-0.724669537565068-0.391945338467193-0.650023936358253
300.4644559163451351.3326356122229-1.23059563374303-0.3579759589374141.182497469771041.54315938069757-0.603390411540623.383088459584220.823740765148641-0.129951318508883-0.657979878422938-0.499534924074273-0.414476569095651
⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮
" + "

6,670 rows × 13 columns

Trait1Trait2Trait3Trait4Trait5Trait6Trait7Trait8Trait9Trait10Trait11Trait12Trait13
Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰
1-1.81573-0.946151.11363-2.098670.7444170.001391720.934732-1.226771.11608-0.4436280.824466-1.02853-0.394049
2-1.24440.109660.467119-1.621311.056680.9789471.000150.3248741.162322.692273.082641.090650.0256616
31.455671.538671.094030.586655-0.327965-0.303377-0.0334355-0.464463-0.33194-0.486839-1.10649-1.42016-0.687463
4-0.7688090.5134910.244263-1.31741.193941.173441.087370.5360230.8027590.2341590.394175-0.7673660.0635386
5-0.264415-0.34824-0.02390650.004739161.256191.203891.298010.3101140.626160.8992890.5499680.5406880.179675
6-1.37617-1.471920.29118-0.803111-0.26424-0.260573-0.165372-0.2192571.04702-0.09858160.9473930.5940150.245407
70.100942-0.191616-0.5674210.378571-0.246656-0.6088110.189081-1.27078-0.4524760.7025630.3326360.002691650.317117
8-0.3198181.357740.81869-1.155660.6344840.2914620.933324-0.7410830.647478-0.9708780.2208610.852512-0.225905
9-0.2883340.5660830.254958-0.6525780.6689220.9783090.1228631.479090.06721320.07959040.1675320.2469160.539933
10-1.1576-0.781199-0.595808-1.005550.7898290.5710580.951304-0.2959630.990420.5613090.7331-1.73468-1.35278
110.7405691.408740.734690.0208323-0.337441-0.458304-0.142583-0.580392-0.684685-0.00785381-0.712244-0.313346-0.345419
12-0.6758920.2798930.267916-1.041040.9107420.8660281.074140.03817510.766355-0.340118-0.8090140.548522-0.0201829
13-0.79541-0.699990.39913-0.5104761.515521.287431.537720.1339891.020260.499019-0.369483-1.10153-0.598132
14-0.193483-0.286021-0.6914940.01315821.523371.401061.531150.3330661.043720.163207-0.422884-0.383528-0.489222
150.1512462.091852.038-1.124751.665571.625361.587510.6358520.8425780.450762-1.39479-0.5609840.28935
16-0.4646090.3612771.23277-0.8260341.434751.744520.2110972.648161.025110.1197570.0596832-0.631232-0.207879
17-0.732977-0.5262230.616579-0.554480.9474850.9368330.9725170.2902511.012850.516207-0.03006890.8787320.450255
18-0.1673260.1753270.287468-0.4026530.5511820.5222050.4368380.2995650.58311-0.704416-0.73081-1.95141-0.933505
191.411591.787220.8439760.481278-0.0887674-0.4995780.304196-1.23884-0.153476-0.8704860.0955473-0.983708-0.356345
20-1.42997-0.4901470.27273-1.61030.9907880.7116881.18858-0.3712291.24703-0.03891620.8834962.589883.3354
21-0.1472470.1232840.617549-0.1871310.2564380.177950.412612-0.2448090.09476250.723017-0.6839480.0873751-0.26221
22-0.187113-0.270777-1.015570.06028510.272420.869133-0.6575192.32389-0.9999361.446720.971158-0.358748-0.439658
23-1.82434-0.933481.29474-1.945450.3358470.3592020.513653-0.07319771.571391.533291.820772.22741.50063
24-2.29344-2.491620.40384-2.364881.410531.422441.170240.8447671.790270.648182-0.0857231-1.02790.491288
25-0.4341360.7408820.699576-1.024060.7595290.9566560.63330.7707340.8249891.842871.91046-0.5023170.13267
26-2.1921-2.494660.354855-1.931560.9419790.9789170.894860.4632391.125371.705280.7177930.6458880.783968
27-1.46602-1.249220.307978-1.550970.6189080.6625080.4759570.4847190.4015650.55988-0.376938-0.9339830.390013
28-1.83318-1.532692.55674-1.518280.789410.9087480.6499720.6683741.200580.2779631.25053.31372.22036
29-0.7845470.2765833.01105-1.119790.9208240.7502181.26154-0.4033640.400667-0.217598-0.72467-0.391945-0.650024
300.4644561.33264-1.2306-0.3579761.18251.54316-0.603393.383090.823741-0.129951-0.65798-0.499535-0.414477
⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮
" + ], + "text/latex": [ + "\\begin{tabular}{r|ccccccccccccc}\n", + "\t& Trait1 & Trait2 & Trait3 & Trait4 & Trait5 & Trait6 & Trait7 & Trait8 & Trait9 & Trait10 & Trait11 & Trait12 & Trait13\\\\\n", + "\t\\hline\n", + "\t& Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰ & Float64⍰\\\\\n", + "\t\\hline\n", + "\t1 & -1.81573 & -0.94615 & 1.11363 & -2.09867 & 0.744417 & 0.00139172 & 0.934732 & -1.22677 & 1.11608 & -0.443628 & 0.824466 & -1.02853 & -0.394049 \\\\\n", + "\t2 & -1.2444 & 0.10966 & 0.467119 & -1.62131 & 1.05668 & 0.978947 & 1.00015 & 0.324874 & 1.16232 & 2.69227 & 3.08264 & 1.09065 & 0.0256616 \\\\\n", + "\t3 & 1.45567 & 1.53867 & 1.09403 & 0.586655 & -0.327965 & -0.303377 & -0.0334355 & -0.464463 & -0.33194 & -0.486839 & -1.10649 & -1.42016 & -0.687463 \\\\\n", + "\t4 & -0.768809 & 0.513491 & 0.244263 & -1.3174 & 1.19394 & 1.17344 & 1.08737 & 0.536023 & 0.802759 & 0.234159 & 0.394175 & -0.767366 & 0.0635386 \\\\\n", + "\t5 & -0.264415 & -0.34824 & -0.0239065 & 0.00473916 & 1.25619 & 1.20389 & 1.29801 & 0.310114 & 0.62616 & 0.899289 & 0.549968 & 0.540688 & 0.179675 \\\\\n", + "\t6 & -1.37617 & -1.47192 & 0.29118 & -0.803111 & -0.26424 & -0.260573 & -0.165372 & -0.219257 & 1.04702 & -0.0985816 & 0.947393 & 0.594015 & 0.245407 \\\\\n", + "\t7 & 0.100942 & -0.191616 & -0.567421 & 0.378571 & -0.246656 & -0.608811 & 0.189081 & -1.27078 & -0.452476 & 0.702563 & 0.332636 & 0.00269165 & 0.317117 \\\\\n", + "\t8 & -0.319818 & 1.35774 & 0.81869 & -1.15566 & 0.634484 & 0.291462 & 0.933324 & -0.741083 & 0.647478 & -0.970878 & 0.220861 & 0.852512 & -0.225905 \\\\\n", + "\t9 & -0.288334 & 0.566083 & 0.254958 & -0.652578 & 0.668922 & 0.978309 & 0.122863 & 1.47909 & 0.0672132 & 0.0795904 & 0.167532 & 0.246916 & 0.539933 \\\\\n", + "\t10 & -1.1576 & -0.781199 & -0.595808 & -1.00555 & 0.789829 & 0.571058 & 0.951304 & -0.295963 & 0.99042 & 0.561309 & 0.7331 & -1.73468 & -1.35278 \\\\\n", + "\t11 & 0.740569 & 1.40874 & 0.73469 & 0.0208323 & -0.337441 & -0.458304 & -0.142583 & -0.580392 & -0.684685 & -0.00785381 & -0.712244 & -0.313346 & -0.345419 \\\\\n", + "\t12 & -0.675892 & 0.279893 & 0.267916 & -1.04104 & 0.910742 & 0.866028 & 1.07414 & 0.0381751 & 0.766355 & -0.340118 & -0.809014 & 0.548522 & -0.0201829 \\\\\n", + "\t13 & -0.79541 & -0.69999 & 0.39913 & -0.510476 & 1.51552 & 1.28743 & 1.53772 & 0.133989 & 1.02026 & 0.499019 & -0.369483 & -1.10153 & -0.598132 \\\\\n", + "\t14 & -0.193483 & -0.286021 & -0.691494 & 0.0131582 & 1.52337 & 1.40106 & 1.53115 & 0.333066 & 1.04372 & 0.163207 & -0.422884 & -0.383528 & -0.489222 \\\\\n", + "\t15 & 0.151246 & 2.09185 & 2.038 & -1.12475 & 1.66557 & 1.62536 & 1.58751 & 0.635852 & 0.842578 & 0.450762 & -1.39479 & -0.560984 & 0.28935 \\\\\n", + "\t16 & -0.464609 & 0.361277 & 1.23277 & -0.826034 & 1.43475 & 1.74452 & 0.211097 & 2.64816 & 1.02511 & 0.119757 & 0.0596832 & -0.631232 & -0.207879 \\\\\n", + "\t17 & -0.732977 & -0.526223 & 0.616579 & -0.55448 & 0.947485 & 0.936833 & 0.972517 & 0.290251 & 1.01285 & 0.516207 & -0.0300689 & 0.878732 & 0.450255 \\\\\n", + "\t18 & -0.167326 & 0.175327 & 0.287468 & -0.402653 & 0.551182 & 0.522205 & 0.436838 & 0.299565 & 0.58311 & -0.704416 & -0.73081 & -1.95141 & -0.933505 \\\\\n", + "\t19 & 1.41159 & 1.78722 & 0.843976 & 0.481278 & -0.0887674 & -0.499578 & 0.304196 & -1.23884 & -0.153476 & -0.870486 & 0.0955473 & -0.983708 & -0.356345 \\\\\n", + "\t20 & -1.42997 & -0.490147 & 0.27273 & -1.6103 & 0.990788 & 0.711688 & 1.18858 & -0.371229 & 1.24703 & -0.0389162 & 0.883496 & 2.58988 & 3.3354 \\\\\n", + "\t21 & -0.147247 & 0.123284 & 0.617549 & -0.187131 & 0.256438 & 0.17795 & 0.412612 & -0.244809 & 0.0947625 & 0.723017 & -0.683948 & 0.0873751 & -0.26221 \\\\\n", + "\t22 & -0.187113 & -0.270777 & -1.01557 & 0.0602851 & 0.27242 & 0.869133 & -0.657519 & 2.32389 & -0.999936 & 1.44672 & 0.971158 & -0.358748 & -0.439658 \\\\\n", + "\t23 & -1.82434 & -0.93348 & 1.29474 & -1.94545 & 0.335847 & 0.359202 & 0.513653 & -0.0731977 & 1.57139 & 1.53329 & 1.82077 & 2.2274 & 1.50063 \\\\\n", + "\t24 & -2.29344 & -2.49162 & 0.40384 & -2.36488 & 1.41053 & 1.42244 & 1.17024 & 0.844767 & 1.79027 & 0.648182 & -0.0857231 & -1.0279 & 0.491288 \\\\\n", + "\t25 & -0.434136 & 0.740882 & 0.699576 & -1.02406 & 0.759529 & 0.956656 & 0.6333 & 0.770734 & 0.824989 & 1.84287 & 1.91046 & -0.502317 & 0.13267 \\\\\n", + "\t26 & -2.1921 & -2.49466 & 0.354855 & -1.93156 & 0.941979 & 0.978917 & 0.89486 & 0.463239 & 1.12537 & 1.70528 & 0.717793 & 0.645888 & 0.783968 \\\\\n", + "\t27 & -1.46602 & -1.24922 & 0.307978 & -1.55097 & 0.618908 & 0.662508 & 0.475957 & 0.484719 & 0.401565 & 0.55988 & -0.376938 & -0.933983 & 0.390013 \\\\\n", + "\t28 & -1.83318 & -1.53269 & 2.55674 & -1.51828 & 0.78941 & 0.908748 & 0.649972 & 0.668374 & 1.20058 & 0.277963 & 1.2505 & 3.3137 & 2.22036 \\\\\n", + "\t29 & -0.784547 & 0.276583 & 3.01105 & -1.11979 & 0.920824 & 0.750218 & 1.26154 & -0.403364 & 0.400667 & -0.217598 & -0.72467 & -0.391945 & -0.650024 \\\\\n", + "\t30 & 0.464456 & 1.33264 & -1.2306 & -0.357976 & 1.1825 & 1.54316 & -0.60339 & 3.38309 & 0.823741 & -0.129951 & -0.65798 & -0.499535 & -0.414477 \\\\\n", + "\t$\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ \\\\\n", + "\\end{tabular}\n" ], "text/plain": [ - "6670×13 DataFrames.DataFrame\n", + "6670×13 DataFrame. Omitted printing of 8 columns\n", "│ Row │ Trait1 │ Trait2 │ Trait3 │ Trait4 │ Trait5 │\n", + "│ │ \u001b[90mFloat64⍰\u001b[39m │ \u001b[90mFloat64⍰\u001b[39m │ \u001b[90mFloat64⍰\u001b[39m │ \u001b[90mFloat64⍰\u001b[39m │ \u001b[90mFloat64⍰\u001b[39m │\n", "├──────┼───────────┼───────────┼────────────┼────────────┼────────────┤\n", "│ 1 │ -1.81573 │ -0.94615 │ 1.11363 │ -2.09867 │ 0.744417 │\n", "│ 2 │ -1.2444 │ 0.10966 │ 0.467119 │ -1.62131 │ 1.05668 │\n", @@ -459,9 +537,7 @@ "│ 8 │ -0.319818 │ 1.35774 │ 0.81869 │ -1.15566 │ 0.634484 │\n", "│ 9 │ -0.288334 │ 0.566083 │ 0.254958 │ -0.652578 │ 0.668922 │\n", "│ 10 │ -1.1576 │ -0.781199 │ -0.595808 │ -1.00555 │ 0.789829 │\n", - "│ 11 │ 0.740569 │ 1.40874 │ 0.73469 │ 0.0208323 │ -0.337441 │\n", "⋮\n", - "│ 6659 │ -0.131005 │ 0.425378 │ -1.09015 │ -0.42093 │ 0.310943 │\n", "│ 6660 │ -0.52427 │ 1.04173 │ 1.13749 │ -1.27262 │ -0.607382 │\n", "│ 6661 │ 1.32516 │ 0.905899 │ 0.84261 │ 0.869143 │ -0.691461 │\n", "│ 6662 │ -1.44368 │ -2.55708 │ -0.868193 │ 0.390457 │ -0.0331364 │\n", @@ -472,267 +548,81 @@ "│ 6667 │ 0.475008 │ 1.46697 │ 0.497403 │ -0.46861 │ 0.496553 │\n", "│ 6668 │ -0.408154 │ -0.325323 │ 0.0850869 │ -0.182984 │ -0.46577 │\n", "│ 6669 │ 0.886626 │ 0.487408 │ -0.0977307 │ 0.830857 │ -0.168597 │\n", - "│ 6670 │ -1.24394 │ 0.213697 │ 2.74965 │ -1.80285 │ 1.11401 │\n", - "\n", - "│ Row │ Trait6 │ Trait7 │ Trait8 │ Trait9 │ Trait10 │\n", - "├──────┼────────────┼────────────┼───────────┼───────────┼─────────────┤\n", - "│ 1 │ 0.00139172 │ 0.934732 │ -1.22677 │ 1.11608 │ -0.443628 │\n", - "│ 2 │ 0.978947 │ 1.00015 │ 0.324874 │ 1.16232 │ 2.69227 │\n", - "│ 3 │ -0.303377 │ -0.0334355 │ -0.464463 │ -0.33194 │ -0.486839 │\n", - "│ 4 │ 1.17344 │ 1.08737 │ 0.536023 │ 0.802759 │ 0.234159 │\n", - "│ 5 │ 1.20389 │ 1.29801 │ 0.310114 │ 0.62616 │ 0.899289 │\n", - "│ 6 │ -0.260573 │ -0.165372 │ -0.219257 │ 1.04702 │ -0.0985816 │\n", - "│ 7 │ -0.608811 │ 0.189081 │ -1.27078 │ -0.452476 │ 0.702563 │\n", - "│ 8 │ 0.291462 │ 0.933324 │ -0.741083 │ 0.647478 │ -0.970878 │\n", - "│ 9 │ 0.978309 │ 0.122863 │ 1.47909 │ 0.0672132 │ 0.0795904 │\n", - "│ 10 │ 0.571058 │ 0.951304 │ -0.295963 │ 0.99042 │ 0.561309 │\n", - "│ 11 │ -0.458304 │ -0.142583 │ -0.580392 │ -0.684685 │ -0.00785381 │\n", - "⋮\n", - "│ 6659 │ 0.735302 │ -0.0923875 │ 1.35506 │ 1.0652 │ 0.643122 │\n", - "│ 6660 │ -0.613654 │ -0.233233 │ -0.72178 │ 0.948663 │ -1.19573 │\n", - "│ 6661 │ -0.433578 │ -0.890664 │ 0.4462 │ -1.71787 │ -0.457925 │\n", - "│ 6662 │ -2.6614 │ 0.750482 │ -5.62549 │ 0.86061 │ 0.506262 │\n", - "│ 6663 │ -0.351239 │ -0.16338 │ -0.376714 │ 0.781865 │ 1.62379 │\n", - "│ 6664 │ 1.30857 │ 1.00127 │ 0.876074 │ 1.37296 │ -0.727299 │\n", - "│ 6665 │ -1.74132 │ -4.1342 │ 2.52424 │ -1.4819 │ 1.32056 │\n", - "│ 6666 │ -1.10252 │ -0.717352 │ -0.905295 │ -1.07487 │ 0.505224 │\n", - "│ 6667 │ 0.195008 │ 0.661188 │ -0.544159 │ 1.54089 │ -0.290803 │\n", - "│ 6668 │ 0.251995 │ -0.915712 │ 1.62863 │ -2.2893 │ -0.813678 │\n", - "│ 6669 │ -0.586483 │ 0.256554 │ -1.32053 │ -1.10432 │ -1.7194 │\n", - "│ 6670 │ 1.16462 │ 1.1404 │ 0.451617 │ 1.40992 │ -0.482502 │\n", - "\n", - "│ Row │ Trait11 │ Trait12 │ Trait13 │\n", - "├──────┼───────────┼────────────┼───────────┤\n", - "│ 1 │ 0.824466 │ -1.02853 │ -0.394049 │\n", - "│ 2 │ 3.08264 │ 1.09065 │ 0.0256616 │\n", - "│ 3 │ -1.10649 │ -1.42016 │ -0.687463 │\n", - "│ 4 │ 0.394175 │ -0.767366 │ 0.0635386 │\n", - "│ 5 │ 0.549968 │ 0.540688 │ 0.179675 │\n", - "│ 6 │ 0.947393 │ 0.594015 │ 0.245407 │\n", - "│ 7 │ 0.332636 │ 0.00269165 │ 0.317117 │\n", - "│ 8 │ 0.220861 │ 0.852512 │ -0.225905 │\n", - "│ 9 │ 0.167532 │ 0.246916 │ 0.539933 │\n", - "│ 10 │ 0.7331 │ -1.73468 │ -1.35278 │\n", - "│ 11 │ -0.712244 │ -0.313346 │ -0.345419 │\n", - "⋮\n", - "│ 6659 │ 0.35674 │ 0.456428 │ 0.882577 │\n", - "│ 6660 │ 0.366737 │ 1.78286 │ 1.90764 │\n", - "│ 6661 │ -0.418756 │ -0.275519 │ -0.912778 │\n", - "│ 6662 │ 1.31914 │ -1.44981 │ -1.77373 │\n", - "│ 6663 │ 0.770329 │ -0.0470789 │ 1.50496 │\n", - "│ 6664 │ 0.757479 │ 1.10001 │ 1.29115 │\n", - "│ 6665 │ 1.29209 │ 0.697478 │ 0.228819 │\n", - "│ 6666 │ 1.00973 │ -0.362158 │ -1.55022 │\n", - "│ 6667 │ 0.141684 │ 0.183218 │ 0.122664 │\n", - "│ 6668 │ -0.2214 │ -0.575183 │ 0.399583 │\n", - "│ 6669 │ -0.985545 │ -0.636874 │ -0.439825 │\n", - "│ 6670 │ 1.39201 │ 0.299931 │ 0.392809 │" + "│ 6670 │ -1.24394 │ 0.213697 │ 2.74965 │ -1.80285 │ 1.11401 │" ] }, - "execution_count": 42, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "# Pkg.add(\"DataFrames\")\n", - "using DataFrames\n", + "#using Pkg\n", + "#pkg\"add CSV DataFrames\"\n", + "#using CSV, DataFrames\n", "\n", - "cg10k_trait = readtable(\n", - " \"cg10k_traits.txt\"; \n", - " separator = ' ',\n", - " names = [:FID; :IID; :Trait1; :Trait2; :Trait3; :Trait4; :Trait5; :Trait6; \n", - " :Trait7; :Trait8; :Trait9; :Trait10; :Trait11; :Trait12; :Trait13], \n", - " eltypes = [String; String; Float64; Float64; Float64; Float64; Float64; \n", - " Float64; Float64; Float64; Float64; Float64; Float64; Float64; Float64]\n", - " )\n", + "cg10k_trait = CSV.File(\"cg10k_traits.txt\"; \n", + " delim = ' ') |> DataFrame\n", + "names!(cg10k_trait, [:FID; :IID; :Trait1; :Trait2; :Trait3; :Trait4; :Trait5; :Trait6; \n", + " :Trait7; :Trait8; :Trait9; :Trait10; :Trait11; :Trait12; :Trait13])\n", "# do not display FID and IID for privacy\n", "cg10k_trait[:, 3:end]" ] }, { "cell_type": "code", - "execution_count": 43, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 21, + "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Trait1\n", - "Summary Stats:\n", - "Mean: 0.002211\n", - "Minimum: -3.204128\n", - "1st Quartile: -0.645771\n", - "Median: 0.125010\n", - "3rd Quartile: 0.723315\n", - "Maximum: 3.479398\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait2\n", - "Summary Stats:\n", - "Mean: 0.001353\n", - "Minimum: -3.511659\n", - "1st Quartile: -0.642621\n", - "Median: 0.033517\n", - "3rd Quartile: 0.657467\n", - "Maximum: 4.913423\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait3\n", - "Summary Stats:\n", - "Mean: -0.001296\n", - "Minimum: -3.938436\n", - "1st Quartile: -0.640907\n", - "Median: -0.000782\n", - "3rd Quartile: 0.637108\n", - "Maximum: 7.916299\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait4\n", - "Summary Stats:\n", - "Mean: 0.002309\n", - "Minimum: -3.608403\n", - "1st Quartile: -0.546086\n", - "Median: 0.228165\n", - "3rd Quartile: 0.715291\n", - "Maximum: 3.127688\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait5\n", - "Summary Stats:\n", - "Mean: -0.001790\n", - "Minimum: -4.148749\n", - "1st Quartile: -0.690765\n", - "Median: 0.031034\n", - "3rd Quartile: 0.734916\n", - "Maximum: 2.717184\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait6\n", - "Summary Stats:\n", - "Mean: -0.001196\n", - "Minimum: -3.824792\n", - "1st Quartile: -0.662796\n", - "Median: 0.036242\n", - "3rd Quartile: 0.741176\n", - "Maximum: 2.589728\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait7\n", - "Summary Stats:\n", - "Mean: -0.001989\n", - "Minimum: -4.272455\n", - "1st Quartile: -0.638923\n", - "Median: 0.069801\n", - "3rd Quartile: 0.710423\n", - "Maximum: 2.653779\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait8\n", - "Summary Stats:\n", - "Mean: 0.000614\n", - "Minimum: -5.625488\n", - "1st Quartile: -0.601575\n", - "Median: -0.038630\n", - "3rd Quartile: 0.527342\n", - "Maximum: 5.805702\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait9\n", - "Summary Stats:\n", - "Mean: -0.001810\n", - "Minimum: -5.381968\n", - "1st Quartile: -0.601429\n", - "Median: 0.106571\n", - "3rd Quartile: 0.698567\n", - "Maximum: 2.571936\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait10\n", - "Summary Stats:\n", - "Mean: -0.000437\n", - "Minimum: -3.548506\n", - "1st Quartile: -0.633641\n", - "Median: -0.096651\n", - "3rd Quartile: 0.498610\n", - "Maximum: 6.537820\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait11\n", - "Summary Stats:\n", - "Mean: -0.000616\n", - "Minimum: -3.264910\n", - "1st Quartile: -0.673685\n", - "Median: -0.068044\n", - "3rd Quartile: 0.655486\n", - "Maximum: 4.262410\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait12\n", - "Summary Stats:\n", - "Mean: -0.000589\n", - "Minimum: -8.851909\n", - "1st Quartile: -0.539686\n", - "Median: -0.141099\n", - "3rd Quartile: 0.350779\n", - "Maximum: 13.211402\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n", - "Trait13\n", - "Summary Stats:\n", - "Mean: -0.000151\n", - "Minimum: -5.592104\n", - "1st Quartile: -0.492289\n", - "Median: -0.141022\n", - "3rd Quartile: 0.324804\n", - "Maximum: 24.174436\n", - "Length: 6670\n", - "Type: Float64\n", - "Number Missing: 0\n", - "% Missing: 0.000000\n", - "\n" - ] + "data": { + "text/html": [ + "

13 rows × 8 columns

variablemeanminmedianmaxnuniquenmissingeltype
SymbolFloat64Float64Float64Float64NothingInt64DataType
1Trait10.00221138-3.204130.125013.47940Float64
2Trait20.00135253-3.511660.03351734.913420Float64
3Trait3-0.00129591-3.93844-0.0007821627.91630Float64
4Trait40.00230893-3.60840.2281653.127690Float64
5Trait5-0.00179039-4.148750.03103432.717180Float64
6Trait6-0.00119598-3.824790.0362422.589730Float64
7Trait7-0.00198906-4.272460.0698012.653780Float64
8Trait80.000614075-5.62549-0.03863015.80570Float64
9Trait9-0.00180965-5.381970.1065712.571940Float64
10Trait10-0.000437029-3.54851-0.09665076.537820Float64
11Trait11-0.000615918-3.26491-0.06804374.262410Float64
12Trait12-0.000588783-8.85191-0.14109913.21140Float64
13Trait13-0.000151238-5.5921-0.14102224.17440Float64
" + ], + "text/latex": [ + "\\begin{tabular}{r|cccccccc}\n", + "\t& variable & mean & min & median & max & nunique & nmissing & eltype\\\\\n", + "\t\\hline\n", + "\t& Symbol & Float64 & Float64 & Float64 & Float64 & Nothing & Int64 & DataType\\\\\n", + "\t\\hline\n", + "\t1 & Trait1 & 0.00221138 & -3.20413 & 0.12501 & 3.4794 & & 0 & Float64 \\\\\n", + "\t2 & Trait2 & 0.00135253 & -3.51166 & 0.0335173 & 4.91342 & & 0 & Float64 \\\\\n", + "\t3 & Trait3 & -0.00129591 & -3.93844 & -0.000782162 & 7.9163 & & 0 & Float64 \\\\\n", + "\t4 & Trait4 & 0.00230893 & -3.6084 & 0.228165 & 3.12769 & & 0 & Float64 \\\\\n", + "\t5 & Trait5 & -0.00179039 & -4.14875 & 0.0310343 & 2.71718 & & 0 & Float64 \\\\\n", + "\t6 & Trait6 & -0.00119598 & -3.82479 & 0.036242 & 2.58973 & & 0 & Float64 \\\\\n", + "\t7 & Trait7 & -0.00198906 & -4.27246 & 0.069801 & 2.65378 & & 0 & Float64 \\\\\n", + "\t8 & Trait8 & 0.000614075 & -5.62549 & -0.0386301 & 5.8057 & & 0 & Float64 \\\\\n", + "\t9 & Trait9 & -0.00180965 & -5.38197 & 0.106571 & 2.57194 & & 0 & Float64 \\\\\n", + "\t10 & Trait10 & -0.000437029 & -3.54851 & -0.0966507 & 6.53782 & & 0 & Float64 \\\\\n", + "\t11 & Trait11 & -0.000615918 & -3.26491 & -0.0680437 & 4.26241 & & 0 & Float64 \\\\\n", + "\t12 & Trait12 & -0.000588783 & -8.85191 & -0.141099 & 13.2114 & & 0 & Float64 \\\\\n", + "\t13 & Trait13 & -0.000151238 & -5.5921 & -0.141022 & 24.1744 & & 0 & Float64 \\\\\n", + "\\end{tabular}\n" + ], + "text/plain": [ + "13×8 DataFrame. Omitted printing of 2 columns\n", + "│ Row │ variable │ mean │ min │ median │ max │ nunique │\n", + "│ │ \u001b[90mSymbol\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mNothing\u001b[39m │\n", + "├─────┼──────────┼──────────────┼──────────┼──────────────┼─────────┼─────────┤\n", + "│ 1 │ Trait1 │ 0.00221138 │ -3.20413 │ 0.12501 │ 3.4794 │ │\n", + "│ 2 │ Trait2 │ 0.00135253 │ -3.51166 │ 0.0335173 │ 4.91342 │ │\n", + "│ 3 │ Trait3 │ -0.00129591 │ -3.93844 │ -0.000782162 │ 7.9163 │ │\n", + "│ 4 │ Trait4 │ 0.00230893 │ -3.6084 │ 0.228165 │ 3.12769 │ │\n", + "│ 5 │ Trait5 │ -0.00179039 │ -4.14875 │ 0.0310343 │ 2.71718 │ │\n", + "│ 6 │ Trait6 │ -0.00119598 │ -3.82479 │ 0.036242 │ 2.58973 │ │\n", + "│ 7 │ Trait7 │ -0.00198906 │ -4.27246 │ 0.069801 │ 2.65378 │ │\n", + "│ 8 │ Trait8 │ 0.000614075 │ -5.62549 │ -0.0386301 │ 5.8057 │ │\n", + "│ 9 │ Trait9 │ -0.00180965 │ -5.38197 │ 0.106571 │ 2.57194 │ │\n", + "│ 10 │ Trait10 │ -0.000437029 │ -3.54851 │ -0.0966507 │ 6.53782 │ │\n", + "│ 11 │ Trait11 │ -0.000615918 │ -3.26491 │ -0.0680437 │ 4.26241 │ │\n", + "│ 12 │ Trait12 │ -0.000588783 │ -8.85191 │ -0.141099 │ 13.2114 │ │\n", + "│ 13 │ Trait13 │ -0.000151238 │ -5.5921 │ -0.141022 │ 24.1744 │ │" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ @@ -741,20 +631,14 @@ }, { "cell_type": "code", - "execution_count": 44, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 22, + "metadata": {}, "outputs": [ { "data": { - "text/html": [ - "" - ] + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAH0CAIAAABuMsSDAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOydeWAURfb4X/fck8mdyQUBBULCIQIiqyEgh6IiEF1RUVjQVVYEdgFRyQIGD+SUa/2xX5dL1FXDIRhcBbkRiHiAHAmQcEguyDn3Pd1dvz8axjgzmcz0TKZnJvX5AyZvuqte95uq19VV9R6BEAIMBoPBYNorJN8KYDAYDAbDJ9gRYjAYDKZdgx0hBoPBYNo12BFiMBgMpl2DHSEGg8Fg2jXYEWIwGAymXYMdIQaDwWDaNdgRYjAYDKZdgx0hBhOuUBQ1bdo0pVI5aNCgmpoaVqhWq8eMGZOQkDB27Fi1Wu1BiMFgWLg7QtwIMRh+WbNmjU6nq6ioyMnJWbhwIStctmxZ586db9682alTp+XLl3sQYjAYFoJziLX333//zJkz69evX7hwoVqt3rhxIwDk5+cbDIaVK1fOmTMnOjp6yZIlLQkdlJeXZ2RkyGQyDjrQNE0QBElyd+eRUQK/lJeXp6eny+VyXy+BoiiBQEAQRMieZbfbRSKRT6cEk/79+3/00Ud33323Xq8vLy+/5557ACArK6uoqCg7O/vSpUt5eXllZWUtCR0cO3bs+vXr48ePb7VGhmEQQgKBwEsNfT0eIUTTtFAoDP7xAoEgrNtgRkaGVCr16W644uv9DM0SOMDdEQaqEfbv33/jxo39+/fnoINGo5FIJNycKItWqxWJRHK5nHMJOp1OIBBERUX5U4JYLJZKpZxL4Jf+/fuvXr164MCBvhpCrVbLZDJfL7ypqSk6OlosFvt0VkNDQ3x8vK+tq66uTqlUhmz/mJiYOGXKlPXr13fp0uWjjz666667AEChUDQ0NMhkMrPZnJKSotPpWhI6yMrKSk9P37lzZ6s1ms1mu90eExPjpYYWi8VqtcbGxnp5vNVqNZlM8fHxXh5vs9kMBkNCQoKXx1MUpdFokpKSXL9SKBSh/NDjGbYX7d27t16vT0xM5FyO3W7XarVu70/QSqAoSq1WK5VKziVwgLvXraio+OKLL4YNG8Y2QlZYU1PTuXNnAGDfw3gQOtDpdCNGjLh06RIHHbRarUQi8ceF6HQ6kUgkk8nOnTtntVqjoqJ69uzpUwl6vV4gEPjjSvV6vVKpDF9HGGR0Ol1JSYlIJLr//vt9HeFFGDqdDiFUWlq6bt26KVOmnDx5EgAQQuxtYZ+s2SPdCh1ERUWtXLnSG/cjkUjsdruXjs1qtZ45cyY6OvqOO+7w8oqsVqtYLPbJEQqFQu+PpygKALw/HuMnGo3mwoULANClS5fU1FS+1WkR7o4wUI0wOjr6ww8/5Ob/RSKRnyNCsVgsEom+//77p154hYyKTZOTl86d9qkEiUTi54hQIpFIJBLOp7c3Xn71nz/8WmKsvGCxWHwdF0YYSqVy1qxZaWlpM2bMWLNmDStMT0+vqqrKzMysqanp0KGDB2Fb8+aiZatXr35wxPA9RV8Gp0ZMqPHMi9OKT5+ndI3vv7Ng+vTpfKvTItzf+TRvhCUlJayQbW8A4NoInYQOCIJISEggeaWmpgZ6PWiYuIlCBC8KcLZCO6Sy4rph/DqCFGzdunXv3r18q8MnDz/88JYtW6xW6/r16wcMGMAKx4wZs3nzZoTQ5s2b8/LyPAjbmqtXr1J3/onGed7aMRXXrhqe+xD6P863Iq3AvQsO8UaIiWwYgXjq+59Of3Uu34rwyZIlSw4dOpSSknLw4EF2tRoAFBQUnDt3LiMjo7S0dMGCBR6EGAyGhfur0SVLlvzlL39ZtmzZ3XffvWnTJlZYUFAwYcKEjIyM/v37f/rppx6EGIxf2C2mMW/TX07lWw8+SU1N3b9/v5MwLi7um2++8UaIwWBYuDtC3AgxQeZ86YXJL8+oqKzkWxEMBhNR4NmpZlj0Nxua0u7s3rlrd4Zh+NYG48y/P1z/qyjLRjmvt8JgMBh/wI6wGZobNoaofemrqutXOW+vDCMGDRpE3Gbq1FvvGEM9NlBKNyC83Z2NwWAw3oAd4R8RiiElk28lggFC6NKlS9XV1Xq9Xq/XOxbf4wBdGAwm4NA0bbPZXLfPhQjYEbZT6urqbDZbXl5eWlraxIkTHaFGdu3aNWPGDIlEMmPGDEeoEbfCYFJZWdl74OBPv9ga/KoxflJ3o2b58uV/f2N+5+w+//73v/lWB8MDdP31WbNnS2XyyS9O4VsX9wQ1nlsIYjab//Of/5w6dYqmuYeGCUdqa2vvvffeVatWderUafbs2TNnzvziiy+AU2wgs9n873//u0ePHmzYDu8xGAwURdnt9laP3L9//290jEXmHLcJIUav13tZl1Ao9DXEmsFgkEql3mz0lMlkQY6OCACDBg0qLi5mP7/88ssffvghAKjV6kmTJp04cSI3N/fjjz9mo6i4FbYper3eaDKCvr6kpi7/rfeAFKK07BsuPx5Me4DR1qLxq0Eks9M/8K2Le7iPCMNyhsmFuW++PX/Lnh3HztE+duLhTt++fQ8dOtS3b9+EhISlS5d+9913rJxDbCCCIBQKBdGWAAApiyFEf4y/Y9LcqK1PSkmb+eprbVq7lxoGmRB/uT1s9JP79h8AhiaUXRBlI0QSkEYHoV4MhgMcn2EdjZCNOuh4Fmbb244dO+bMmbN8+XI20YRbIe/877sD6zd9dPr0r9T9M+BqcXt7R3z69GmLxZKTkwMAYrHYEeONQ4AuqVQ6adIkhULha6w7u93uZdBtqVTqxtnoGxmxnHpsvtl+TqFQeC7BbDYrFApfB21Go1GhUIRm6B/Hy+2ysrIRI0asX7+evZO7du0qKipi32Pn5eWxzc2t0IHNZjt48KA3UXYtFovdbvcmImDZ+V9JRRINAPD7ujOKoiwWi4ezrFar1Wr1fExzbDabT8dTFNXS8SKRyPssGZgIg2MLD+AME8MwNTU1dNCZ9dobX9uzbuqcmsStRnv16tUrV66YTKbgKBP8RapGo/GJJ564ePGizWZ79913H3/8VgykcIsNRIKwnYZpZV9ub9y4saKiIjY2dubMmaycw8ttu92+e/dusxdYLBaLxeLNkQiguQtkP9vtlDfn8gLeMdWe4TgiDOAMk8FgePnll8+cOcNBDX+yT9A0A70fJk7dCgf8hyYrie4/5CGrqu7r3V+1mh+KzT7h/TOp2xKSkpL8CR3Ogdzc3IULF44ZM0ar1Y4aNeqDDz5g5Tg2ULjAvtxmPy9durRXr17sZ27ZJ9auXRvY7BMEQQCwg/jf/5XJpJ5raW/ZJyiK+sc//rF9+/bu3btv27aNfd0SIrO87QqOjjCAjTAmJmbbtm3Jyckc1GDf6XFzIbdeg9x+3/aH/y0645ra2FVD4uPjW1VMKpX6mX1CKpUGP4sCQRDTpk2bNm2akzwcYwPdrKn+8ssv+/fvf+edd/KtS/AI4MvtgHP48GHKbm/3S/FaZ82aNTqdrqKiYuHChQsXLmQDxobRBFPEwPHV6OnTpx3L1VwbIfiSfSJkYWyW/6xfX1BQ8NNPP/GtC6ZlKn89fubixOmv7du3j29VgkrIvtwuKSkZM36yxWr943sWjBs+//zz119/XS6XFxQUvPLKK6wwNLcwRTYcn9mMRuO4ceOOHDnStWtX10a4ePFi10boJAx9jKr6j66QwqN7U1JSBg4cyLc6mBawGqH7YFIa1BfLoUDIvty2Wq3CmCTQNLVpLZFBoNKb6/X6p59++vvvvzcYDP4sOLDb7V7uR/KyBPYVoEMhq83a2NjouQSKorRaLYeV2DExMZxfrXF0hCHbCANM7vPCn/7LtxIYjBsi6eV2uyVQ6c3lcnl+fn5sbCxJkt7M4LYEu6k3ICVcvXq1vLzcbDbD7xNPIBKJWi2coiiGYTjo4M+iX4KXoJqPPPJIXV0d+7mmpiYpKYlbinaGYTjv4qqqqaHkSWBoBGk02K0AAGIZmDQQ3wEar0PSHaCqgphkwmpIkItjYmI86AAAjhX2WVlZhYWFPmmi0+nEYjG3JT984WTBxMREiUTiqyG8NB9N00ajUWW0IMoOEgWYtRCdDNqbEK0EkxqEUgAgSCIxShId3eJONZqmSZL0RsPmFqyrq1MqlaG5fcJPuLVBhBBCyPMNsVqttQ2NDGUHUgAECQQJdiuQJAjFcXKJ51Ue3pTf6vEe2iBFUWq1WqlUell+W5Oenn7q1Km0tLTa2tpu3boZDAYAyMzM/PbbbzMzMy9fvvzYY4+Vl5e7FbpaUCwW+3T3XEEIMQzjj0dxlFBRWQUCEWO3QmwKGJpAGg1ARIElubWbjxDq0qVLkF//8jOdXVJSsmnTJm4LZEKZ8vLyxYsX861FMMAWDHewBUMBNr35q6++6prevNUJpgi24LvvvhvkSnlb19WrV6+OHTvyVXsbEZFDh5bAFgx3sAV5x8/05tiCgQIvcG7vlJSU/OlPfzIajeyfeA8TBhM0cHrzECGcnp4wAUer1T7//PMmk8khCZFIlRgMBhM0sCNsvyCEnn/++fz8/OZCvIcJABjf4VHbkpKS5vEcwi7wPQbDO/46QtwIw5dly5Z17dp13LhxzYUc9jBFGDRNN9ymqampwTusVisv2uIxPQbjP345QtwIw5fDhw/v3bvXNVAThz1MEYZAIEi5jVKpTPEObvt//ASP6d2CELK2DJuwwhUcdLs949eOE9wIXUEIWXwn+N7l4MGDR48eFYvFrIcjCOL48eMQWUHyIh48pncLQsjoO76mlcZEEtxXjQa2ET7+1Piq2gbOyviEmISvv9yalOSc7jwgIITYYAreY7FYgpx6AgAWLVq0aNEi9jNB/B5XgXOQvIixYLjAjuld1xxyHtNHjAVJkkxISHD7FUVRBEG09G24EzEWDD4cHWHAG+HRY8c1ectBkchNH5+QbproT9Ykz5Ak6evuAoFAIBKJ2kgfX+EcJC9iLBguOMb07J8EQRw7diw3N5dz9glswXAHW5AzHB1hwBshAEC3HIgPxms3UtxOU7m2RPMwe37tYcIWDCIBH9MDYAuGP9iCnOA4R7ho0SJ0GwBACOXm5kIIpIAJODRNZ2dn860FhiNFRUW9e/eOi4sbMmQIG7Mx4ikoKDh37lxGRkZpaemCBQs8CMOCvXv39uzZMy4urmfPnu0t01Yk4bS/INQIcGSZCMs+sXbt2s8//7ysrIxvRTBcqKysnDhx4r59+/r27btu3boXXnjhxIkTfCvVJgRsTB9iMAwzYcKEbdu2DRs2bOfOnS+88EJNTQ3fSmF8xnV/QagRgA31ro2wurp69+7djjwaboWhxpQpU1avXs1+fvHFF1etWgUAffr0efPNN3nVC+Mtrha8du3a+PHj77//fplMNnnyZPxAE+K4WpCiqE8//XT48OFGo1EikcTFxfGrIcYzbntRt/sLQg0cWeYWTz755K5duwDAarUWFRWNHz8eAIYNGzZ69Gi+VcN4hasFhw4dumHDBgCgabqgoOCZZ57hW0eMJ1wtKBaLR40aZTQaY2Ji8vLyWGtiQha3vajb/QWhBnaEtxg+fPiFCxfq6ur27dvXr1+/9PR0vjXC+EZLFjxw4MC9994bGxu7du1afjXEeKYlCyoUCoPBsGjRopkzZ/KrIcYzrhZsKXBHqIEd4S3EYvFjjz22e/fuwsLCiRMn8q1OMHC7DCF8g+S5WhAh9M9//vOdd94pLCxcunSpUIhzrYQ0rha8fv3666+/DgBRUVEvvvjixYsX+daxTYiYQJWuFmwpcEeogR3h74wbN+6zzz7bv3//E088wbcubQ67DOGDDz5QqVTvvPPOCy+8wMrDOkiekwWLi4t37dq1e/fu9PR0g8HApv/GhDJOFkxPT9+0adPRo0cRQlu3bu3Xrx/fCgaeCAtU6WTBlvYXhBrcHWGEjScA4KGHHjp16tTw4cNjYmL41qXNaWkZQlgHyXOy4JEjR8rKyuLj46Nvw7eCASbi26BYLN61a9err76amJhYWFgYeXOEkReoMkx7UY6OMCLHE1KptGfPnq7vRZsvi40YWlqGENaRKp0sOH/+fPRHvCmEYRjtbXQ6ndY77HZ7W16Zez3bQxt84IEHTp06pVKpiouLI29Hb+RFi22pFwVfOlKGYep8x58MMBxnTRzjCYPB4DSeKCoqYh9b8vLy2DlSt0InYuIS7DtmE2I55yvxHqvezROx3W4/f/58VVXVyJEjg6BDiMAuQ1i7du3MmTN//vln8CNIXsRYkCAIR8Q7kUgkEolIsvXnRW+OCSy4DbYEm33C7VcURbHZJ1y/8tLQASTggSojxoIkSSqVSg5nca6RoyNkxxMGgyEmJqb5/CfnZ5lD3+z64YcfuCnjMy8/6hp19+uvv37llVc+/PBDR9w4bjAMo9frfTqFvY3+VMqB69evr1u3bsWKFewyhMWLF7NyzkHyIsaCBEHI5be6EplMJpfLg+/kvAG3wZZgs0+4/YqiKKPR6DbGvUKh8LNeXwl4oMqIsSAE/8kS+YfBYHjvvfcGDBjA/imXy81mM/tDlMvlLQk7dOhQVVXlZ9UhyJkzZ+666y69j9y4cUOv1wdZVavVGh8ff+TIEYZh1q5dm5uby8pnz56dn5/PMEx+fv5rr73WkjCCLdinTx/Hn7W1tTRN86iPN+A22BwnCzpht9vr6+uDqY+XNO+KcRvs1atXkCvl6HVbWtaMs9kRBKHwkaioqOCv7G9pGUKERaqMYHAbjFRwGww+HB1hS8uaIy/odgTjdhlC+AbJa2/gNhhJoIgIVBm+cHSEeDyBwfALboMYTKDg/kaOHU84CSMp8j0GE+LgNojBBIRQXA6HwWAwGEzQwI4Qg8FgMO0afsIQy2SygoIChUIBADabTSQSsXtFA4jdTjEMDQBACgExAAAkCQwNAjFQVhBKgLKCQAQMBYSghQNsIBDePgABMAAEAcjDFpnGxsbAXkXI4mRBoVAYqH0/FEXRiACGBoEQaDsIhEDTQJAACAABIXD5iv3lsF9RIBABbZdIJNxqb7cW9L8NWq1WAAIcZSD0+5/sZ4Df/yRIoYAUCAT+1OgWbMHgY6coBhFe9KK3D6BpEIqBsrptp7xYkEB8xA+7cOHCgQMH2M8rVqx49tlnO3bsyKEcs9ksFAodcUCa8/bi5ap+48mTnzOZg6H+CikUMspuxIUD6NG5sHUOjF9FfFWAHvgb+VMh03UQNP5GEsCk9iBK9qDH5sHWOfD0CmL32yj3r+SpHcydfwJVNYloJnt4wunPFs6f27wii8VCkqTDOyqVymeffdanq9DpdGKxWCqVcrgDfOFkwSeeeKJz585uDeEBk8nEhm5pLtz99dcH68Xk1WJm4Hg4uhFyJhKndqEOvUmrDkwaptM9xKVDaNBkOPj/4L4JxPk9KOlOkgDQ3GC6DiJK9qAhU+DIhx07dsxISx3/9O+RqwwGg5db45tbsK6uTqlUhuaGej/h1gZtNhtN064b0i0Wy7w3CxiBBAnFIFYQdiMyakl5NCOQkGIp01RFiGVIKAaJgrCZkFEtSu36zLAB9957r1M5bOQXR0CDVqFp2mKxNE/dAB7bIEVRarWaQ9SSEMTVgmlpaa53wyfc3k8PfL7jqx+PHQaBGPrleduLdriLOPs1Gj2f2Pb6yvdXuD4MMQwjk8lefvllzlfBhSDvW3SlX79+p06d4nauWq02mUxOwsOHD7/zzjtxaZ2h4GdBRm+Y9CEMHC8YPBn+uplM7Q7/sQAAbLCRMUpYcFLQqQ9MXAf3TxDmTIQpn5Ip3WCDDQgC/mMmY1Ng3nFB577w3FrImSy871lYeKpz1l1O1Wm1WoPBwE1/RwnsZucg89VXX/Xq1Ss2Nnbw4MFlZWWsUKVSjR49Oj4+fsyYMSqVyoPQQb9+/Y4cOeJqiFZRqVSuF/763Hx44l1BYga8ugeEEnilkIhLh6FThfc8LrjzHnjmfTIuDeYeAYKElz4hlV3g/omCnAnCTn1g4joyRgnzjgNBwHNrs/r9qXmx9fX1drvdVw3DYkO9/3jfBo1Go0ajcZVrtVqxPJqMS4OUbtDlPjI1E8RyQXIXyOgj6J4LIikZlwbJ3aDrfWRqdxDLZHeP/OSTT1zLsVgsTU1N3mtutVobGxu9Pz5kN9T7CWtBX++GKzabraGhwfvju/buDwtOCp170a6eetGphaTyTthgI4Uiq9XqWiYvNuL+qFtUVNS7d++4uLghQ4aUl5ezQt4j39M0PXzEgwt/smh1ujatKNyprKycOHHihg0bbt68OXbs2MgI2fw7nfvzrUEwCM02iMGEHRwdYch2o4gNVpT3FkEGfvohkrh27dr48ePvv/9+mUw2efLksrIyVs4hBQzDMDdv3qQDBMME6l09CpRK3oCCPsUQsm3QO5Ber8cZIjEhAsfFMo5uFAAmT568dOlSVs458j0myAwdOnTo0KEAQNN0QUHBM888w8o5hGw2GAyzZs3KycnxdZpTo9HIZDKnCXOLxQyggFtuBTX71y2I/cf1CJqmm5qaHH+q1WqGYXwNZadWq0mS9GaOMDY2lvMKHW4EsA3SNP3LL7/06tWr1UqtVqvdbnfN3uAxA44b81Gaxhn/mLn/8PeF//3YqRyWVjVhsdlsLWWTcEtIZZ/AhA4cHWFgu9EXXnjh8OHDHNTQarUSicRsNrN/njhxoqamxn2/2VJfilr4382BCABohlGpVM3ler1eIBD4kwpLr9fHx8fzsljmwIEDb7zxxsiRIxctWsRKkO8pYGJiYjZt2qRUKt0G9feASCSSyWSOCy8vL79/2EOahjoYveD24kOi2b9uubUW0fUIgUCYnJz8+3EEER8f76sjRAiF7GKZALZBs9m8bt06p6x4bjGbzXa73ek23rx585kXptopO+Ep5cAfWhZj1qIeI0x2yilNhNVqNZlM3j9S2Gw2NrO0l8eHVPYJzC+//CISiXr06MGufeURv7ZPBKQblclk06dP57bSyW63SyQS9md97ty5Z1+aTnS9z/07qpb6UqKV750OJQnCSVWapgUCgZ8rtYIfdBshNG/evBMnThQWFnbv3t0h55wCxn9KSkqojn2RrL6Nyo9IAtIGFQrFxo0bXbPquGIymex2e/NYl7/8enbSCy9e0QFCqIVG1OLTjFgkdqrUarVKJBJvNGFhtw14fzxFUQRBeH88pu1gRPJHJ//dcqP88P7vcnJy+FWGY/8bwG5UIBAMGDCA22slyW2AdSfxadpJm+Dkl9wuCgA8OMRbrZkgnFSVSCQCgcCf12ISiST4Y47i4uJdu3adPHlSKBSyUzXsQxkbnXnx4sWuIZudhG0CQQJPe6HCjhB5lFn6/uqLNU2CRC7bnzDhS0lJyfnz5w0G33Kv/gGLXjfn+9h/PRg4pbjDsf9lu9Hdu3enp6cbDAbHpDeOfB8uHDlypKysLD4+Pvo2rByHbA4XQqgNynAyBI6E77rf+wYPfXldUX1DhIQv4OgI22k3ajM1aPW9B+YOuD+Xoii+tfGL+fPnO+2kYeU4BUy40E7bYAQRvut+EUJGTZP++U8JAT+xyQIOx8uYP3/+/PnzXeURHvleXWO2WEuHLyLff4hhGL61iSiqq6srKioQvqte007bYAQRwHW/CCGtVsvchrNKrZZgNBqnvvrPK5fLOVfhDAKnGjlfhT8TTOHtz2vr6lb/6/8JBMIRQwd3SE8PQo2ESIq6DcLzWIGlqamp/6BhOp2WzugXmBJNmuuV1fGpHadN+et7774TmDIxmIASwHW/Op0uLy/vwoULBoPBH0dot9v1er2HTbFnz5796tu9pqfXwNH9DiFy+uBydrMDnDdEIYQ0Gk1DQ4NDQlGUVqvloHxcXBzntRrh7Qjnvvn27goGKKvNbn9p8kS+1cFwxGQymcwW658mCW+cD0yJ6hq7SG7903MNf9zrggkpbtZUffbZZxMmTOBbET4JyLrf2NjY7du3Jycny2SyxMREzsrY7XapVJqUlNRcaDQaN2/eDADV1dUMw4BYDj0ean4A4fTBZZjQ7ADnJcQESSQkJKSkpDgkFEWJxeIgx4MNb0cIANDrIVBVF+39/PDRY2YzDwHEMSGKQASyWADsCEMVXf2vNXV/mzaj3TrCEFn364EPNmz5cP2Gmzdv6mI6gbYWgCQT0mk/9kyHLKG4U9hnqs+r4rOu3fEw5fKghAl9Rj4xvlOnTiaziW9FMD6z7+ChM2fPcjyZoYnU7nwkvwkVQmjdbwusWr3mwl0v6mgBfe+zdMd+0HWgfeAET4GewpaIcIQAREwKKLvyrUX4QdN0dnZ2c0nwl27/eOwI/H1XRLauiGfc089eq6nlW4twJZTX/TY1NV2+fJmi7JDRB0ThlCGOG345wlDoRjGcWbt2bU5OjiPcNkvoL93GNIf3NkiKucdUAoTspPi+B0ZcvHgxcBqFDaG5hen8+fP79u3rec999wwbVV1T03YVhRTcHSG/3ahKpTp58qRarQlIae2TPn36vPnmm05CDtknQpn6utrjx483X5MWSYT9owxDU0ZtyW83uK0SxAScCxcu5D37/NOvL62vvKZ/6wIp8C3VdvjCfbFMnz59unbtOmbMmObCoGWfeHzii2cuXTU21EDGk/6X5isIiH/OXyAUCKZPeyUuLi74CgSEYcOGuQo5LN02Go0FBQWFhYUWi8UnBTQaDZuZ0yFpYd01l+wTAADV578pObb3wOE1S98dMWIEAPga01Wr1QqFQm/2JykUCpEo2L1GoNogTdPnzp276667Wq3RfhvnLzy92fb41pskhfIYiqLYMlss31d9WoCtyO3xAoEgNKOrBxO9Xk/EpmhfKYJp4dqtcYO7IwxUN6rRaO67776Kigqfar9WXqaf/JlgfWvrzRydbIvZJ5y2vjgf53YHDKKpVVeiJCc25Q7KycrKEggEcrncB+3/iF6vT05O5iX7hCsclm6LRKLBgwdLpVJfs09oNJrS0lLa07Ynb7NPuMekQf3yRHa9WCyWyWQymcxXR8helDf9Iy99aKDaoNlsXr58+WsWOJUAACAASURBVOjRo1utkc0+4fgTIdSKn/udlvO6IGQwGNhBIZtNQiDwNpmozWYzGAzem5WiKL1e7zbLRExMDM4+EXwYu23Hjh2nTp164IEH+vTpw5caAd4+waEbjYuLKywsTEtL86kitqkQrW5sdxzQYvYJp60vzse1uAPm4dekZXsTExNTU1P9zD4RFRUVOi2Qw9JtsVj80EMPSaVSX335gsXvf/XtPpPp90Q8nOMUeDqRIEQikUQikUqlvjpC9qzwGihwzj7htHvMLc2zT5SUlDAM08qzyO/ftRzOniDi4uLY2q1Wq1Qq9Sn7hEQi8X7nHEVRAoHAmyvFBAeTqv7/HbtOfHlgBUI8OsIAt3C2xwQA127USdic4CchwrREMJdu/3btmunPK3CYnsDCrQ36ypUrV3JHjjaZTV6PCDEYNyCGtj84i8gawq8aAXaEobMDBsONUFi6jfGH4LRBvV4P8jg/xvC/Y9Zrc3IGvf322/4XheFMY2PjY+Oe+8fr/7RYfZvpjwwCPBQrKCiYMGFCRkZG//79P/30Uw9CTIjgFFcwaCGbV61adePmDegZ2FJbpLa29vjx4xKJZNSoUUGqkifCrg1SJi3qhaPY88ae/YemzXxVp27UJvWilXeR1FG+NeIBfx1hkLtRhNDs+W8dO3a8tp7/POaIoa9cuQIAHTt29GeOsB2ybNmy+Rt2oca6YFTGUGfPnl2wZGUTKOiKs2ajH6lEQxJeHmWuXLnicZUTJmzYtOWT65l5cKVYkNgJMu6G8vboCMNpFQAA2Gy2D1YsOT3wDbud/3SAZq1q6ty3ho8au23bNr51CRsoilq0dMWOnV/RPR4kxL6tMuWG9cblD7btrbz+m/HPK/B0VkD48ssvn5v8V5PJHMAyjx8//tFHHwWwQIwPJHZq59mVw8wRArtStMfw1teLtj20xWR+fgsMfIZvRcKJ/fv3L163+ZfrjRCsKJPIZrY99BpBersiH9MqZWVldIfWNx36QMPVQ2evTn1tfkp6R40GR8kIEqd/PfPa3Pzz5wOU8sU/dDpdbW0tX6EVws8RYsKXL7Zt/8+GTRCXBvEd+dYFE0oggLg0Sqyw2Gk8WRg0nhj/l5WnzVdq+J9momuvvPXe0s7dsv827e+8KBA2jrC+vr5f7oj0O7vTdMi1E6vVqtfrfY2r0t5obGycNPn5onKdzeZtHJAAgxhKIBk5dtyChXiBInfWf/LFijUftEnRpADvpQkCCKFF768dOXZcQ2Mj5D5PymL41giQvoF+/F3bM6vtPPXuYeMIL1++fK1Oo3q5KNT2LTH1vxW89XZCUvLkF//Gty6hjlAig245vFVvt9Am7X7o+e3B9rgcwH8oinr2penzCxZqGHFbtUKEqqqqHAmJMG2BzWZbmP/a/g5PWSIxsyA3guEIAxX5nhBJIDEjgIoFBEZbx4xbRk1YV3Lh4mtz81etWs23RoHHHws2NDQsXLhw6KN5A4aMsNlsbaShtxAC6J4LAGq1+s/PTX7gkTGDHxr152eea2xs5FmxNiYgbbC8vPzbnVtpIEEoCax6DmwgGPnIY2P+/NTHH3/SRlWEKYHqRauqqsrKyoAg4Z4/E0TIDYRomjIYDNevX7906dKlS5dQsFYSBONG+Bn5vr6+/qG8p6bNfiOk3z1e/eFSo2Xlz7p5S1Z36Jp9Z2Z2SGvrI/5Y8C9/m/7Ot6VHjxyu6PtiSEz/6BtLL1xMSe/4VWnj8V/OHEfd9ny332g0tn5iOOOPBWmaLr98ZVTen/vnPEDRdJv2nshiYOLSjpRc//Kb7w4fPrxy5craWpzvECBA+UPKy8vvvm9I7qhxDBOSOcwbr3/zv/8pU9PvH/7IwJGP9+zZ02wO5MpkDwQjtlmrke8vXrzIRr5nGObIkSMajcZoNHbt2rWhSbVz9zdlF0vPNCFGoSTNl6CmFACg6iwCBLXliLJBUyWYtYhAoKoGuwWqzjc/AOw2UFeDSQN2MaiqgLJCzTn2AEAI6i4juxXUNWBUI5KEpkqgrFB1FgCg+hwgBHVXkN0KmhtgUiHGDo0ViLLdPuA8IAbqroDdApobYGwi4tIgJdP2s+nGn5eQm55ftmxZbW1tbGwsQig2NjY1NbWxsTE9PX3voaMMg4Qk/PtfayQSCQDY7fYQDzLXqgXLysr69esnFAoZhtmxY8elS5dMJlNWVlZpaenVq1ch6ymidD8SCAAh0NaC1QiEAPT1wNDQ+BsggMbrwNCgb0BmPVhNoK0Fhob6awAIGiuAtoOhEZm1yGYBzQ1ADNRfBQBQVSDaBiY1ALplR4Sg7jIA3PoxmLVIIADKBqpKQAjqrwIguP4zJY1l4jJIRSJBCiG+A0MKps6cI2CoJ5/IO3fuXEJCQmpqalpamsFgqKmpGTBgQG1tLRuiMzs7GyF0uPinLZ9+JhXA4X17HHHGQzx3ga9t8Ny5c3FxcXK5/NT5i198uau2qgJEUkFiZzCcR4gByopsRoKyAgCiKbCbwUICQoAYsNvAbgLKBoAQQ4HdgkjBra8oG9hMQFkBsWdZwCYAhIBhgLKC1QR2KwAAQ4Esbv/BQ//btYMUisxm8x133KFSqUpKSrp169bU1JSRkTFo0CBWVZ1Od/XqVYTQ3XffbTAYGhoaDAZD//79GYZBCJEkSRAEQohhGIIgSJKkKEqr1Uokkps3byKExo4d62h6EWDBrKwsu91utVqPHz+u1+uvXbuWnp5+s7Z+/9HjNGX79VwpY7fqQUZN2gDLh7ddL4rsNo69aMNV6Hof1ftRYs9y/Qv/JVeMePb5KQatuusdnUQiUUbHDtnZ2Xa7vUOHDrW1tayxHnnkEUeIdn8sSARh7KlQKBoaGmQymdlsTklJ0el0zb/NzMwEgB9++AEArFbrgyMfphFhNhrkUrGNRoxQYtRpaCClErFer09ITKq7UZ3aIaOxvi4uIdGg04qlMqDtFAMyqUSn0ycqk2pr2APqY+MTTAadSMIegGQyqU6rTVQm11ZXpnbs1NRQHxMXb9TrRBIZwVB2BsllUp1Gm5icXFtTldIhQ9VQHxMXbzbohRIpwVA2CkXJZRqNOik5pbamKiU9Q93UoIiJMxt0ArFUAIyNoqOi5Bq1Oik5pa6mKiW9o6qxITo2zmLUS8QikiAsFmtMbGx9Y1NSSqqYhEPffct2o3q9PjExMSaG/ynrlmjVggzDHD16VCqVWq3WoQ89YjSZjXqtTCq1A0nZ7VJ5lF6jjktMVDc2xMbFm0xmkYAAgdhmtcgVCp1aFZ+YpFE1RcfEWKxWAQGEUGK3mOXRMZqmhoQkpUatUiiirDaKBCQQS80mQ0xsnKqhLjEpWavVyOUyO42AoUUSqdloiImLb6q7mahM1un0MomYAgJRdrFMbtBp4xISVA318QmJBqNRIhQwpJC22aRRt3TT1NcmJibpjOb4xES7XiWTK0iRWK9VK+Qys42SRyloqzkuLtZqtZrsNE0Im1sQQj53ga9t8GZtbWxcvM1kICRRao1aLJIQBIEArGajUCwRkgQiBbSdomlKLJEQBDCIsFlMIrFEQBCIJGmKoilKLJEAAIPAbjWLxBKSIAhSQFF2mqLEYjFBEAwQNotZKBILSQBSSFF2irKLRWKSFDAAdotZKCATExPMeq1IHtPUUCeTR4lkUXIB6t+vr81mq6ioMBqNNpphKKprlzsrKiqEEpndZr37rt6VlZU0Td9xxx0SiUStVtfX18fHxycnJ9tstt9++81isdgZkMuk+/buiSQLHj161GAwREVFPTTyYQagpqYmOTlFo9UJpHLGbm2pFzXqtGKZHFE2igGpRKzT65OUSp96UZNBL5JIgabsDJJLJVqNNiklxade1GLUC8RSEtE2ipbL5Rq1SpmSWnejOjmtg6qpMTom1mrUS8WimJhou8UsFArNVps8KooE+C5QFkRtj1wuN5vNCCGj0SiXy52+7dev36lTp7iV/NFHH504ccIf3f773/8ePXrUnxIKCwsPHDjgTwk7duzYv3+/PyW0Na1a8K233iouLva12HXr1p05c8bXs95///3Lly/7etbbb79948YNX8+aO3euTqfz9awQJFBt8MiRI5999pn39f70008bN270/vjz589/8MEH3h9/5cqVFStWeH/8jRs33n77be+PDx28sWBZWdmqVav8qeXatWvLli3zp4SKioolS5b4U0JNTc0777zjTwkcCMYbOc9JfNLT0//xj39wy+d39erVmJgYpVLJWbdr165FRUWlpKRwLuH69etSqTQ1NZVzCRUVFU8//fSDDz7IuYS2plULbty48auvvvLVEGVlZUql0vucOyylpaU7duyIjo726ayzZ8/u37/f14yJp0+fPnnypDePmW+99VZODn8LYlsjUG2wrq7OZDJt2bLFy3obGxvVavXWrVu9PF6tVtfV1e3evdvL4/V6fWVl5b59+7w83mw2l5eXHz9+3PWrCLAgRVHV1dV79uzhXIvBYKioqDhw4ADnEoxG47Vr1w4dOsS5BJPJdOXKlWPHjvl6oj8WDMar0VdffVUikSxevHjevHkURa1YsaL5tw0NDb/++mtb6xDi9O3bNzk5mW8tWgRbsFWwBcMdbMFwxx8LBsMRajSaCRMmnD17lo18z2b1xIQR2ILhDrZguIMt2KYEwxFiMBgMBhOyhO5yYQwGg8FgggB2hBgMBoNp12BHiMFgMJh2Tdg7wqKiot69e8fFxQ0ZMqS8vNync/0P3+dP7Q5KSkoiI8E9hwvx9QZyMxlnM0WMafzB890bNGgQcZsXXnjBg3Vcbed9yVOnTnUr8Vw+4YLn8iMGf363EdCjciHI+xYDS0VFhUKhKC4uNplMK1asyMnJ8en0uXPnTp8+3WKxTJ8+PT8/P8i1s2g0mnvuuSfcDYE4XQiHG8jBZJzNFDGm8QfPd49hmISEhOrqar1er9fr58yZ48E6TrbzqWSz2ewq8Vw+QkjfjDfffHPu3Lkeyg/kXeMVf363EdCjciO8G/nhw4dfeukl9nN9fX1iYqJPp3fv3v3ixYsIoYsXL3bv3j3ItSOEGIZ5/PHHt2/fHu69LbcL4XADOZiMm5kixjR+4vnu3bx5U6FQ3HPPPQqFIi8vr2vXrh6s42Q7n0quq6tzlXguv/lX586dGzFihN1u91A+1zsUWvj5uw33HpUzEdLIKYqaOnXqtGnTfDorKirKZDIhhEwmU3R0dJBrRwgtWbJkzpw5CKFw7239vBDvb6A/JvPJTBFjmkDh9u79+uuvw4YN+/XXX5uamiZNmiQUCj1YpyXbeVPy+PHjXSVelm+1WgcOHFhaWuq5fM53JqQI1O82THtUzoRfI8/KynJ6qbt///5+/frNnTu3+ROfN3gO3+clnGs/dOjQAw88YLPZUBj2ts2t4NOF+Gk+zibzqZawNo3/cLPRjRs3CILwYB23tvOy5Pj4eM+SlspHCC1evHjGjBkertdtaeEC55botgSWMO1R/SG8GznDMPn5+YMHDy4rK+Nwerdu3crLyxFC5eXlmZmZQa59/vz5TvO1x44d41AO73C+EA43kIPJONQSMabxH89379SpU46o942NjQKBwIN1nGznU8mpqamuEs/ls0KKojp16uQapb3V0sIR/3+3Yd2j+kN4O8Ljx49nZWWp1WrHlLhPp8+ePTs/P5+9+6+99lqQa29OxAw7fLoQDjeQg8n8NFPEmIYbLd29w4cPI4S+//775OTkCxcuWK3WmTNn9unTx9U67JHIxXY+lTx16lRXiefyWeG+ffsGDRrU/IpaKr8t7h6PcPvdRkyP6ivh3cgXLVrkzyJYtVo9atSoDh06jBkzRqPRBLn25kRMb+vThXC4gRxM5qeZIsY03Gjp7rEfGIZZt25d165dk5KSJk2aVFlZ6WodxylOtvOpZK1W6yrxXD4rfO6555zyLrVUfpvdQn7g9ruNmB7VV3CsUQwGg8G0a8J+Qz0Gg8FgMP6AHSEGg8Fg2jXYEWIwGAymXYMdIQaDwWDaNdgRYjAYDKZdgx0hBoPBYNo12BFiMBgMpl2DHSEGg8Fg2jXYEWIwGAymXYMdIQaDwWDaNdgRYjAYDKZdgx0hBoPBYNo12BFiMBgMpl2DHSEGg8Fg2jXYEWIwGAymXYMdIQaDwWDaNdgRYjAYDKZdgx0hBoPBYNo12BFiMBgMpl2DHSEGg8Fg2jXYEWIwGAymXYMdIQaDwWDaNdwd4aBBg4jbTJ06lRWq1eoxY8YkJCSMHTtWrVZ7EDooLy83m81OQoqiEEKcdYuYEsICJwva7Xb/y6RpmmEYPwthGIamaf+VCcgVhTJu2yCLn79h/03g580PyA8p9PFgQfDdiL5aDSFEUZT3xwdBJV/h6AgRQpcuXaqurtbr9Xq9fs2aNax82bJlnTt3vnnzZqdOnZYvX+5B6GD8+PEXL150EqrVaj8vW6PR+GqbgJeg1Wojvg+FP1oQIdTU1OR/mTqdzmaz+VmIzWbT6XT+K9PU1BTZDzRu2yCLn79hq9Wq1+s5nw4AKpXKH09mMplMJpM/CoQFHiwIvndlvt40mqZdRzieaWuVfIWjI6yrq7PZbHl5eWlpaRMnTnR0N7t27ZoxY4ZEIpkxY8bOnTs9CB0ghHQ6HRNKVFZWHj9+/McffwyaYpzt5w+BGtOHIGazubi4+OTJk1euXOFbF4xf1NfXFxcXFxcXGwwGvnXBtMi1a9eKi4s9jEpDHCG302pra++9995Vq1Z16tRp9uzZM2fO/OKLLwCgpqamc+fOAMAOAdmD3Qod6HS6MWPGOHVYTU1NFEUJhRzVAwCVSmWz2UQiEYdz7x083CJLMleWfLn1i4EDB3LWQa1WW61Wb3SIi4uTSCScK+KAY0wfGxsLAI5bzQ7fd+zYMWfOnOXLly9ZsqQlYSjzxsJFmz/fDgRxX+/uB/d8zbc6GO7kjZ/URAkt1ZeOHT7oT2PEtB01NTWPPPGMUVV/7szp7OxsvtXhAkdP07dv30OHDrGfly5d2qtXL/YzQoggCPaD492mW6GD2NjY7du3p6SkNBeSJBkfH++PIxQIBLGxsb46wnPnzjU0NDRU/YY+LIl5f3BCQoJSqayrqwMAuVzO+gzvEQqF0dHRYrHYp7OCg2NMX1ZWNmLEiPXr10ulUgDYtWtXUVERO3zPy8tjfZ5bYShz9epV02MLQSCmrn3Cty4Yv6itum5+91Lsv0fzrQimRUwmE0WKpMmd+FaEOxw9zenTpy0WS05ODgCIxWLHaCY9Pb2qqiozM7OmpqZDhw4ehCHI5cuXhzwyllB2QfD7hND6zR/PnDULgBj96COrVy4Xi8Wpqak8KhkoAjim12g09957b3V1NQAghBoaGvyfUdNqtVKplPMo2WqxAABoak4cO0qQJAFQWVkpEAi4lVZfX88wDPsw1xLx8fHskwQG037QarWlpaURMAvL0REajcZx48YdOXKka9eu77777uOPP87Kx4wZs3nz5sWLF2/evDkvL8+DMAQxGAwgj9P8/Tv42+/970+//GLLexfUNUV71u85dLRDivLyhXM8KhkoAjimj4uL2759e1paGnsASZL+PytIpVKZTMbZtUjYE1XV0PV++PtumBqVmprK+QUDQRApKSmeHSEG0w75y9SZe4p2IsqGFIlRimi+1eEOx8Uyubm5CxcuHDNmTIcOHdRq9bJly1h5QUHBuXPnMjIySktLFyxY4EEYZmjriLseNf/9G3ukrMQ+ffp0cXEx+9l1TA8ArmN6J2FzQsdJzJq3sM99Q04Un/xdRODNsmGP2aAbNGSoRBa1bds2vnUJJBRFTZs2TalUDho0qKamhhWG0YK1365dpeI6Ml3uC/dl1Rz7CIIgpk2bduXKlYaGho8//jgmJoaVx8XFffPNN9XV1bt373bMqLkVhj4Wg3bw4CEffbSFb0XaBKPR+MQTT1y8eNFms7mO6RFCrmN6J2Fo8q+l754fstBIu3HMBoNh586dO3fuPHnypOu34UgEr/t1gjJqqX/8T9Tn4ePHj2/btu3ChQt8axQY1qxZo9PpKioqcnJyFi5cyAo5bELjmZB5DuYMflhuEcqkQ3OPCNK6861ImxDJY/rMXELoZoHSzPw3J85d8pd/Ll/w7tLgKxVwAriXNzwQSW11V//zTfGkOW9t376db20Cw+eff/7666/L5fKCgoJXXnmFFXLYhEbT9JkzZ+xB548DQURRVPB1cODPPjTuyzLbBWTEPiiwY/pp06Y5ydnhuzfCcEIaff+IR69evWZ+9C2wmYq/fTchLUNEQtVvV0NzTa83RPa6XwBQqVQ6na55V4tsVvtjr0FdOY9aBZaKioovvvhi2LBhXbp0+eijj1ghhwVrZrN51apVY8eOdVsLG9PA+yX0RqMRIeSNX2GarRhACOn1eq1W600VbaFSTEwM5+aMHaFvIIZhLR1G73gxyKz9pd9MwcW/AwA0XLMl3Gl+8RPBguywntgI4LpfvV4/bty4n376ybUWlUpltVo59y8Wi8VqtXKI0GQyme4b/rBOb7JYLQ4havZtY2OjN+UYDAaCICwWS6tH+tONcob19KWlpevWrZsyZQr70p7DgjWFQrFx48akpKSWKvJpL5lUKkUIRUe3vvhF0GwBGkGQ8fHxHnQIjkrcwI7QFwyNNTdvJnfoZDPq/OkdMDzQcwQhvLUgiBBJIaEjv+r4TwDX/crl8nnz5rl9tqMoyp+9sGKx2Gq1cnhqpGlaVV9nXVUH02IcQsdMlEQi8bJMkiQJglAoFK0eyXl3jT8olcpZs2alpaXNmDHD8XI7rDehhSkR++rPV06cOPHzzz+3EuDUqEayWNuaRlLIJWANpk0pKirasmVLGA/xfCSA634FAkFmZqYoxPBw7QKBIODVkXzMgzz88MNbtmyxWq3r168fMGAAKwz3BWvhCHaEAAD/+9//Rj45cfZ7a03m1l+hYEKQPXv2PDdtzt8/Ogzh/LbTJyJ13W+7YsmSJYcOHUpJSTl48ODGjRtZYZguWLNYLL3v6rNy5Uq+FeECfjUKAHDz5k3oOdzUbRi5+22+dcFwwWg0CjvdrfvLRvi+vcRUc6z71Wq1o0aN+uCDD1h5QUHBhAkTMjIy+vfv/+mnn3oQYngnNTV1//79TsIwXbBGmXREr4f8zNjDF/6OCEtKSqKiohx/RuQeJgwmBIngvbwXLlw4efIkQi0sEbQaP9n21UNjnly99l/B1QvTGmEbvMIvvbVa7fPPP9880FzE7mHCYDBBoaysLOfBx8bPfNNmayEPYsWpa1GZB8zpR3/4ObiqYSIW7o4QIfT888/n5+c3F3LYChqOIGlMl55339G955kzZ/jWxV/wmB4TUmi1WohO0v/1c08HpWVDXIfv9u2PSUwZ8chjwVINE7Fwd4TLli3r2rXruHHjmgs57GEyGo2zZs1S/xGNRqP2Dy9LuH79+szZrxZu20ZT7HrRFpdaNE9JgYzqmgn/baJEdXV1/uugVqvtPCWyx2N6TLjScI3qPEA/ccONBhXfqnCn/QTJC3E4OsLDhw/v3bvXNT4Fhz1MIpFo9OjRMp6Y99aifx29eqi8sZWNEwDNdjEBAEBatkAil0gkAVGDl6XbARzT22y2YGgcYNCZM2d+/fVX3K2EKyIpyOP5VoI74R4k78CBA40NDXxrERgIbsE1FixY8N577zWXHDt2LDc3NzMz89tvv83MzLx8+fJjjz1WXl4OAK7CRx55hM12CwA1NTVJSUlOmedommZ3wnK9Lm9LaGhsNCAx2K0AAGIZmDQQ3wEar0PSHaCqgphkMDSBNBrsVgAE4igwqSC+IzReh8TOpK42JSnBKVVQVlZWYWEh+7mpqSlkE/MCwNKlSxsbG99//32C+P1noFAoGhoaZDKZ2WxOSUnR6XRuhc0tWFVVRdN0RkYG+ydN0/7vTWbz/3n/AzCZTCq9iYlOvmU7dQ3I44CyAW2DmBT3wqZKUiBENJWQEN/85XCXLl3+7//+r3nh9fX1SqUywvIRttoGWXw1hBMIITYzl/enWK3W2oYmRqEETQ0kdobGCkjMAHUNxCSDvhHkcWDRg0gKDA2IAWmMyKzq2CG9eQnN26Ber/dyQ33wqa2tzczMzMrKcgTJS05OBoCsrKyioqLs7OxLly6xIfTcCr20IPjenbKRzDxbjWGYispKIEggSCAFQNsBIRBJExRSb1ZjeVapuQVZDAZDKEaWWbRo0aJFi9jPzbtRL/MRlpSUbNq0ibV6JFFeXr548WK+tfAKdkzvunTbyzF9ZFuQza3oICLzEUa2BfnWwiv8DJIXwRZctGiRSvWHN95srFHPU0gKhSJUYo16v4epV69eHTuGfZgrJ3h5w8mNgwcPHj161PG7IQiCHdN7H94JWzBEKCkp+dOf/mQ0Gtk/1Wr1pEmTTpw4kZub+/HHH8fHx7ckxBbkF/+D5EWqBQmCaP6SBm6/WnASOsE58zYEJLJM85erEbCHqZ2waNEidBsAQAjl5uYCjksSbuDlTuFLYJNjRxIEQUh8x59noLB5esIEhzAN79Q+ac9bmDxD+07wU5HgIHmhAw6xhnEzpnc6IPTDO7VPArWFKcKgabqpqYn9zKZhslqtrZ4VGxvb0mKTNgIHyQsdsCPEYMISP5c7RTACgcCxhEQmk4XsqtH2lRw7tMGvRjGYsMSx3In1cARBHD9+HPAMEwbjO9gRYjBhCV7uhMEEilB5Nfr4U+OraoMUpEBMwtdfbk1KSgpOde0EbMEQgfMME7ZguIMtyJlQcYRHjx3X5C0HRWIQ6pJummix4AS8AQZbkEcCstwJWzDcwRbkTKg4QgCAbjkQH4zZC1Ic1LVh7QhswXAHW5APOIdEcAO2ICe4zxHu3bu3Z8+ecXFxPXv23LdvHyuMsLjpbmPDY8IIiqKmTZumVCoHDRpUU1PDtzoYnyFc4FujABPxIRGOHj3at2/f6OjoDyNpVgAAIABJREFUvn37fv/993yr4x6OjpBhmAkTJnzwwQcqleqdd9554YUXWHkkmbCl2PARQ3t4lFmzZo1Op6uoqMjJyVm4cCHf6mB8Rt+MN998c+7cuXxrFEjaQ0iEiRMnzp8/X6VSzZs3b+LEid6cwoYV9RU2Vjg3ODpCiqI+/fTT4cOHG41GiUQSFxfHysPXhFOmTFm9ejX7+cUXX1y1alVdXZ3NZsvLy0tLS5s4cSKbh6FVEEKW21itVot3BH9rV+Q9yrhaEAA+//zz119/XS6XFxQUvPLKK7wqiGkFtxZU3Oa3334rLi52xPqPDCIsJIJbC8bExGi1WoPBoNfrvdzQyTCM9o/odDqdTqf1CEVRnDXn6AjFYvGoUaOMRmNMTExeXt6GDRtYefia8Mknn9y1axcAWK3WoqKi8ePHs7HhN27cWFFRERsbO3PmTG/KQQiZfcefZxluRN6jjKsFAaCiouKLL75ISEgYNmyY95HpGd9pwwtrN7i1IIvNZnvppZf+9a9/+RNYOdQIYFbXEMGtBT/++OMpU6YkJCS89NJLn3zyiTflCASCpD+SmJiYmJiY5BF/Et759atSKBQGg2Ht2rUzZ878+eefIZxNOHz48IkTJ9bV1f3000/9+vVLT09PT093GxveMyRJOuaxGYYJ2XyE7KOMwWCIiYlx7MWGcH6UcbUgAOh0OoRQaWnpunXrpkyZcvLkyVbLoSiq4Y/pRpuamlpN5xYXFxfkAF2Rh1sLsqxcuXLgwIE9e/bkUb2A438GmFDDrQXnzp37xhtvzJo1a/Xq1fn5+QcOHOBbTTdwHBFev3799ddfB4CoqKgXX3zx4sWLrDx8o1qIxeLHHnts9+7dhYWF7IvslmLDRxLso8yiRYsc493wfZRxtSAAKJXKWbNmpaWlzZgxo6SkxJtyBAJByh9RKpUprcHLzyPCZnndWhAAaJr+8MMPvXwlE0ZEXkgEtxb88ccfZ8+enZaWNnfu3B9//JFfDVuCoyNMT0/ftGnT0aNHEUJbt27t168fKw9fEwLAuHHjPvvss/379z/xxBPQcmz4yCDyHmXAxYIA8PDDD2/ZssVqta5fv37AgAH8qhdYIm+WF9xZEAAOHTqUkZHRrVs3HhULJmGdAcbVgn369Nm0aZPBYPjkk0/uvvtuftVrCe5zhLt27Xr11VcTExMLCwsdc4RhbcKHHnro1KlTw4cPj4mJgWax4Tt06KBWq5ctW8a3goEkIh9lnCwIAEuWLDl06FBKSsrBgwc3btzIr3qBJfJmecGdBQFgy5YtI0eO9L4QhJDpNmaz2eQd/L7qiJisrq4W3Lx587fffpuWlrZjx46QbYPc5wgfeOCBU6dOOQnDOm66VCrt2bOnY0TfUmz4yMDxKPPbb79lZ2dv3ryZlYd1ChgnCwJAamqqa36GyCDyZnnBnQUB4LPPPvOpEHbxPfvZbrcTBOH40wMikUggEPhUEcYVVwtmZ2efOHGCR5W8IVSWYMXEJdh3zCbE8iDUZdW7mSOx2+3nz5+vqqry6dkzrAnsowy2IC8EcMFaxFiQJEnHmIld5RSaaZgCTsRYMPiEiiM89M2uH374IUiVvfxoQkKCk+zrr79+5ZVXPvzww9Bc5Bn6YAsGmevXr69bt27FihXsLO/ixYtZOec1h9iC4Q62IHcQH3To0KGqqoqXqtuUM2fO9OnTx/FnY2Oj1WrlUZ+2I4IteNdddzkJb968yTAML/p4wGq1xsfHHzlyhGGYtWvX5ubmsvLZs2fn5+czDJOfn//aa6+1JIxgCzZvgzqdTq/X86iPZ/bs2dOjR4/Y2NgePXp89913rFClUo0ePTo+Pn7MmDEqlaolYTuxIIter2e3QrUROB8hBhOWROSCtXZFRK77DVNC5dUoBoPxlchbsNaucKz7NRgMTut+i4qK2CW+eXl5bOgZt0JMoMCOMMDU1NzgWwUMBhMGROS634DAMIxKpWouMRqNqNliYLcoFArOc5PYEQYY1PohGAwGc4tIClQZKAiCiIqKai5hZ/KchE74E4eW+xxhUVFR79694+LihgwZUl5ezgrDN7wTBoPBBJOIjO4UEAiCkPgOSXJ3ZxzPrKysnDhx4oYNG27evDl27Fg8zRuO4EcZDIZHIjK6U5hCIMTlZd6RI0c+++wzdqFaQ0NDjx49GhsbASArK6uoqCg7O/vSpUt5eXllZWVuhZmZmYMHD2Z3udpsNpFIFITE0wghm90OAjFQVhBKgLKCQAQMBYQAEAMAQJLA0M0OsIFA6HwATYPQ7QEIADWpNYcO7Fc1NbI1NjU1hWz2icrKyl69eu3bt69v377r1q3btWsXG/0hPz/fYDCsXLlyzpw50dHR7Jy8q9CDBR3vcAKO3W5nCBIQAoRAILplBdoGpCAQQgYE4qa6G+Xl5efOnWteb21tbUpKSoTlRuelDTbHZrMhUggMBaQAGBoIEhAAICBIQDQIREDZbrVToRgo+6221trBer2+tLT07NmzbC16vT6UN9QfPXq0eXSn7OxsANBoNBMmTDh79iwbyIkNDuAq5MWCVqsVgAC2HoSafQYABA4FEOIchr6xsbG5BVkMBgNCKDo6mqvircDRETqgaXrGjBkkSa5btw4AFApFQ0ODTCYzm80pKSlsMltX4YULFxzJON555x2DwfDmm282L9ZoNMpkMn+GuiaTSSKRNI+ZVFNT89H2ry0jZsPWOTB+FfFVAXrgb+RPhUzXQdD4G0kAk9qDKNmDHpsHW+fA0yuI3W+j3L8Sp3agO/8EqmoSUUyHPsTZ3Wj0Atg6B55aTnz9Lhr0PHl6J3PHvaCtJe1mptfDcb98cun8r2yNarVaoVCIRKJWtQ1+Eh8/H2WaW3D58uWjR49m2zAA6PV6/3+vZrNZJBI5vfT/cNOWsth+0PgbaWhkBr8Iha/C+FXkt0uYux6FpkpSV8cMmQKFr8L4leS3y5jeD4OqmtTeZB74G2tQcs9yptdI0N4kVdXM0Jdh62vw1FLyu5VMjxGgbyAbf2OGTSO3z9244T+TJk1qXq83jrDVPE1tQVFR0fz586urq/v06bNx48bu3bsDgFqtnjRp0okTJ3Jzcz/++GM2KZirsLkFV6xY8eyzz3bs2NG1CqPRKJVKOcces9vtFEXJZDLXrxavWF2XPZb8eSvTYwRc+wmSu4LFQFp1THov4tJhNPgl2LMCnloK2+dC3kJizzKU+yL54+dMr5Fw9QdI7gZmHWk3MSlZxOXvUc5k2L8Gnlwi2DnvvXfe6tSp07PPPsvWEuKO0B+8tCD4bkSr1QoAbnukWbNmEYoERiAlJXKmsYKUyBihlJBFg64eaAqkCiSOIiirwKx++sknMjMzXTfde6OSUql0WJClrR2hXxvq9+/f369fv7lz59rtdlYil8vNZjNCyGg0yuVyD0IHffv2PXz4MP1HamtrrVYr7Qd1dXVs5ncHP/74Y2xmf/iPBQBgg42MUcKCk8JOfWDiOrh/gjBnIkz5lEzpChtsQBDwHzMZmwLzjgs694Xn1kLOZOF9z8LUQlJ5J2ywAUHC/xnIuDTIPyq4oz+MXwVDXhLeOw7ePtuhaw9HjfX19Waz2Rtt/bGCn1AUNXXq1GnTprF/RkVFmUwmhJDJZIqOjvYgdNC1a1elUllXV1dXV1dbW3v+/Pk6vykrK6usrHQSPvDIGJhaCA/+Q3D3KNhgY+0oSO0GL38OI2cL+zxyS7jeIkjrDn/7Lzw8R9B75G2DmgRpWfDSJ/DoG4JeI2CDDUgB/J9ekJ4Nf/0IHpsn6DEUNtgEYmlFRYVTvefPn6+trfWssMViCaLREEKooqJCoVAUFxebTKYVK1bk5OSw8rlz506fPt1isUyfPj0/P9+D0EG/fv1OnTrlthY/g0KYTCa1Wu32q+z+90H+98IOPeCvH0HWAzBmAQydKrzncXh+A5meDYsvgVgOG2wgksKaWjI2BeadELIWzBoCY9+EIVOE9z4Jf/k/smMveOccSBWwwSaOitFoNM1rCfEN9YHCgwURQg0NDTabzfvSPOxeJ0hSkNwFMu4W9BgGYrkgPh3SehDZQ4m4NCIqnkjtDt1yyIy7CHmcOD5106ZNQVApIHBcZoMQmjdv3okTJwoLC9nnUBYO4Z0IgoiJiXEa/JG34aae2xK8K83vh3ri94r8v4q25sCBA2+88cbIkSMXLVrESpDvK9ZiYmK2bduWnJwMt5+r2M/+IBKJZDKZVCptLhSLfk9h6hC6HYcRbj79/pEAN6cTt/9VKpVOIxiGYZKTk0Pt1ei1a9fGjx9///33A8DkyZOXLl3KykN/F9rFixd/+OEHrUbDow6YNgVRVjJ7EN9a+ABHR1hcXLxr166TJ08KhUKDwQAA7MsHdkZ38eLFrtO8TkIMvwTwUQbDC0OHDh06dCgA0DRdUFDwzDPPsHIOu9BMJlN+fv7WrVtda9FoNBRFcZ7ntlgs7ICyuXDIyFGmTveZ6+sB2Hkmb3E+lD33D1Kk0WgYhnH8bTAYvMw+4eUsBsZLaF3Ty9Nm7N6z76vthXzr0jocHeGRI0fKysrY6QcW9rcemkl8VCrVC9Nm3aypslgswa89NMGPMm5hxFEDH3hIICCXvTX/4Ycf5lud1gnImF4oFObm5rqdyZPJZDKZzJ8FXyRJOpVs1utMT60iL/T3dXmCy5DcdYxOSKXS5tVRFEUQhNtLc9XTR3UwnmAMjXTWMI3BzLciXsHREc6fP3/+/Pmu8tAM77Rt27a9F2ptyb1IOw77covwepQJGoxJWzJ4gezg6traWr51aYUAjunFYvHo0aOdXkSzSCQSqVTK2RGyDti55OYvmf154Uw0+xcAAGih5NEnn5WKBB+v/3dmZibczkfo9tJCAX+WO/GtuxcQYfNsETaK+olAeQd0vodvLUKI+fPnO00Xs/Kwzo4dGLrlCGJT+Faiddgx/e7du9PT0w0GAzush/a9C4026n7tO+Pc5etarZZvXVoH78YOHdqLI8RgIgzHmD76Nqy8vWefyBoilMfwrYRXOJY7yWSyyZMns1uVAGDXrl0zZsxgVzbt3LnTg9ABTdOlpaX2tqS8vDztzu7I+zlddgIXMW2qVXOazw37Co41igkDho564ucTR62IhAlPBaM+hj537ty3336bnJw8YMCAYNToO+E1PYFxJbDLnd577/+zd+bxURTZA3/dcx+5EwIJoBASWFQOxQsQAY9V5FRUdmURXV1ZZEUOFQUBEcP1U0FlZZVl3XW9UEBAVuRGLkUDBIKBADnIncncV/f0Ub8/GsY4SSbTPZM5MvX9I5/Jm66q1/2m+nXX8d4b999/f4sN2e12AAh8KZAQ4drHr/z888/O5B5QcbGpsE2vyLJci2/nIVHJh8TERBx0G9Nh4Xn+4M6t8HatfFmYFmTTNSXvbTz1j20Hb+qReXAX9h+YdiQky50SEhLWr1+fnp7eWitJSUmBex21Wo2a7V5PTEwkZb7+wt8ELwEAoFAoWtMqeJVCCHaEIYbn2NLSUgDo0aNHpHXpWOhSg1pYIQZEu5h753j0aVzxB+FpEROH4C1M0QOeIwwpdkO9wdR/6N29evVyu2Nj3TAGg4kI8bDcyWxsbD6jGYVgRxhSKDtSah1LzsuUUbpcG4OJIBUVFT/88APf0dPpBUjHX+7kMJ69UPbQQw9FWo+2CcoRchznDbUsEG1JfIrOFt8z5qE1f/+A49hwtosJFe+9995rr70WaS0wIcDpdA66Y+R9U2e62zmuBeOhX1y45OHHHv/xxx/btaEg6fhbmDiGyPpdpJUICOmOcM2aNYMHD/Yu+RWIth0wb6x6c48j4xyl5YJYWduBifJHmc8//3zuyveXnOBEBeLCRCcMw9jNRuvcw+0910uZDfsVA7756VxZWVm7NoTpMEh3hP369fPJnQSSdsC0O90HQkq3cDcaC8TEo4yy2/UwpoVNAhiBKH+U8bJjx44PP/yw+XLH9uKGUYqMa8PUFib2ke4IR4wYMXr0aB+hhB0wbrd7xYoV9t/icDjswSHUwLQVbLfZi0arbx6oha/8vaYIs98BasuyERi5DdWjDM/zFRUVweTMao1gdsheIbg3SWGpeiCI2GgcOmLiUQYACgsLH5n6zKvbzrB4dhATlYR4+4SEHTAkSXbu3Nknxw1xFcmaBFs+FAR/Fu3KiBEjmgslPMo4nc6ZM2cOGTIEABBCZrM5+Cj+FotFo9EI2TibiK98bips0QO16JTQrx9aKI5aOpZlWaPRaDab5XK5fzsmJSWFObUyAPTr1y8nJ2fMmDFNhVGYholhGEVKpmv0Qvjxq3C2GxNwHHfdddedO3fOK4nCWKMURR09ejR8L/RhJ8SOUMIOGJVK9fjjj/ukkHa73Xq93idBuSgoitLr9XKFwv87QQDx7L1ftBDq3k/Ner1eJpPp9fpgIveHH2mbeb/44ovQ5iOsrql96533ykovcWxT73I1d2C75SP0OVahUHTq1Ck68xFC6B5lbDbbXXfd1fRe7MVsNrvdbslPNkIaJpPJhPiW+6GoN+lA0jA1+ZK3Wq0NDQ0EQTidzjYrT05ODv+jzJo1az799NMW3+m/+uqrOXPmrFy5UnhqaVEYNua/9saad9cR3ftJK37ixImcnJxoXuMTYkcYPUl8Xnpt2Sf/+pBFJEy8OTwtNgXJFH96ahrw/OsLX+nbt2/4FZBMlGzmfX7egu/tqWCwyfWxEGU/mpD2KLNu3bqMjIzmtZEkmZCQIPlhzu120zRtsVgIsuXHCFGPFy0/thItH0IQZGJiYnp6OkEQPs/ZLRKRNEyx8k5fX1/Pdeol0VuoEwbfOXLjJ/8ZO3ZsaLUKISF2hNGTxGfP7l30jO2yT54NT3M+cLRrs+xWzcHVz9bVxZYjjJ5HGeg/GkoOgulymJprCavZuHnzZovFMmXKlGDGJ8KJhEcZgiBSU1Nb9ATkVaQpc+7cuTvv/r3TYpR1yWv7aLG05UMJgiBJUvgb+tZDQQinJ5577rnt27e32IqQrFhsYM+mCxc8Hg+IS6J8Neg2AFB21aAHHA5H01VawavUnGBSKwfbt31GNqIr4K9KF7agXC1w+2RFwacRa10q0fMoE3lqfim+VPHEyo8dP21/5JFHAnmriAai6FEG4OzZs+TvRsLv7kf/Wx62RmMdCe/0SqVy/PjxreUfFrIrB+4khMVf3trKy8sF1yv5ZopYz65du5KSku66666QqNQiwTzuxMZDLqb9iLZHmS++2nzg4PcXLl2CnuFpsHU8LuicZ3tqo/xU9M5tNAc/ysQ6Et7pFQrFyJEjW8s/LGRXDtzrsCyLEPLW9vvxj1TV1EB6T3GesMm4tbuu7D/VFz0gf+CBB0KiUsiJ0hEDTHzCsuwfHn14naFbvTEGEqtGCS0+ykRDXJKzZ89WV1fzOBiCSKIt1qjZUE/oUoOqgudQVt9o/h10QEfo8XgcDgffyio1TNRDwN3PEUp/YyCY6GfTpk03DR25eNn/0TQdEQV42v2PDf9+9vk5u3fvjogCkomeWKNms3nEmIlOlzMEN1PKvnnz5m49coKvqT3oIEOjVquV53m5XK7T6frffuel4jNMFJwaQ7uefWF+UmLiS89NmzBhQqTVwWDCgd1u/+STT7799lvPkCcRZZdHaLkTXXP+gPp24vjla7NPRnnvi7bpCS8lJSU/F53jeRSCFyaXhet+k/HyT2+//fakSZOC32ocWjrCG+GhQ4cyu16Tmd29U48+KZnZ504eZ94xk/LIX2ja3HAuZ0KBXVNeXh5pXTDS4RWaO+8be+vwe3bt2hVpXWKAGXPmzXz3ix2HCiI/JnrdPXDNjS6Xq7Gx0ZvkCCMKmUobusqQm/K8snRVFN4PO4IjLCwsJG77I/Pw/3mUetvcQ5FW57f0uJlMuybSSkQ1BoNh1apV+fn5ox969LEnnw42Klo7wLvtJwY+d9pMVFdXR1qXGMDhcHhu/RPXuXekFQEAgPqLb7+7Nqt7j6f/OiPSqsQYCKEDBw6EMvojQiCTqTK62e12qp0zkIgl8uOHoYSUQ1JmpJXAiGPy08/uraRR42UgCH7YU4D+E2mNWqLPcNmZLZFWInopLy8/deoUAAwbNizSuvwGZK1Bo+bxKV0ZOsoekaObS5cuvbbk9W27DxCJIb6jum3m+0c9kJXT5/cjhq1ftza0lUumYzlCTExRWVlZVVXV2GjgbpsBxXvlthp+8J/gv3+LtF4YEZw/f76srOzxac+5U3tSpSdze+XU1dfDPXdGWi+MRHieX//Rx4veWG53OFCXvqTTGNr6Wbed79Sr2qPauWdfaGsOhnAMjbZHCpiampotW7YsXPJG9943LH5jRUSyNwQKQg0NDefPn28eDyJWCJUF7Xb73r179+7d++mnn+7atavfLUPuf3LWqcIzIVQV0yLt0QfNZvOlS5duGTZy0itvN9RW2Sev5wjZLzkPWj1RN7jdAQhbIq2ysrJ58xfYbTakT2u/VkCf1mhzXpvbZ9GiRSdOnNi0adPq1asjtcAYwuMIQ5ICxu12z3jhlfsffHTMhIlFRUX3jH9kyqJ3899cU9nnQWNq74gkwQkQvq7k/977YOCQEVOfeqaoqOjChQuR1kg0wVjwH//4x4oVK9566y0A+Oucl8c9NeuBPzz5+Iw5j8x9w9JQY51ziNTFxnZ11lTzl+kz1PrE0RMmRloX0YSkDyKEFi5a9Mr8BY9PnTrjb891733DDTcPsZmM1unf/Bp1pMfNhCL6tr44TXsOHup53cBbh9zxyvwFy5bHXpib9k6kRVFUY2Pj/v37+w+6lSGUBCkLeRO/wUN5PMzlOsPStf8acd/oR5/+29wXXyorK9u0adPx48fbt+mWCMfQqP9wsRzHFRcX33DDDQDA8/zevXuNRmN9ff3AgQNTUlJ4nr98+fKcBUvsFpPZSfO3PQaH/vnN15tAkwiPfyBrEPbTEMhpAlMVMBRUngEAqCxEgKCuBBgPmKvAZQFGCaZKYGmoPi0cAAhB/QXE0GCuBqcZkSQYLwNLQ2UhAEDVaUAI6i8ihgZLDbhMiGegsQKxnqsHnAHEQ/1FYCiw1IDTiGgHNJYBx/zmgIZLvLmKHzKVddv27Pnkhhu2J3TuPujGgUoSvvz0Y2+gBJlMFrXhEEGMBTmO27t3L0Jo87d7vv7sYwAEiZlEdl9Uefrtdf80GM30vS/AxSMyl9n6wGIoGg6VhYhjobEM7I3IbfOaDzgWjOVXhVdMhjgWGsvBYUBu65WLXFmIWBaMFWBrQJT9qvA04hgwVoDdAO5mQlv9r0dWnUGsB0yXwVaPaOevQk4QNgDluCKsLmINFdyIGaxMceCHT7vlXacioeDHo1rtlTV1HcaCPM/v37/fbDaXlJRce+21J04XlZSWO13uisuX3TaLgdcCzxEeB0rpRvIk/8RH8M64K5eo5hfhB3/l6lEOsNWDx4Xc1itdr7YYEA+VhYAAqs8C4qHh4pXrTzvAVg9OM3JbwVwFDA21564cDE0OFixIO8BaD04jooSDKag7f/VgBNVFV7r2lYOdYK0HS41dlWZN6l1+avtxe6K67LCHpgFg7ty53njiHcmC33//vcFgyMjIEELIms1mnU4nnGBFRYWQpZUkSZvNZnV7Vr/zXmZaaklZhdVmA5eF6N4fTFWAELAMeJzAUAAAHAMeNwIeAIBngaEQ5QDEI8QDSwPlAI4FhID1AO0Cxg0IEMsC40YUCQCI58BzpQgIRTxOAACeQ4TMRqpItQyxzO8G3Ezw7LU5uT26ZWsTk+oNjXcPufXhhydWVlbabDaE0PXXXy+coNPpNBgMADBq1ChvEOBgLEiE4V1Kr9cbDAaNRuN2uzMzM202W9Nvc3NzAeDYsWMAQNP03ffexyJw2a06jbpLly5Go9FqtRKaJIah7XZ7alp6fU1V5+xujQ31yalpTptVqdEi1sPyoFGrbDZ7WkZ6XXVV5+xuhvr65NRUl8OuUGmAY1geaTRqm9WaltGprupy567djYaGxOQUl8OuUKmBYxkeaTVqm8Wa1qlTXXVlZnY3Y0N9YnIK5XTIVWqCZz0s0mk1Fos5vVNmXXVlZlY3s9GgT0ymnHaZUk0izsNyOp3WYjand8qsr6nq1CXbZDDok5I9LgepVJPA0R5Wr9dbTMb0zM5KEvZ99z9v6LzExMRoztYUuAUpihpx1z0gkxsMjWp9As94PCyXmJTUUFOVmd3NbGzU6BMQ6/F4uKTkpLqaqs7Z3cwmo0arQxxDe9jklGTBfGZjo1qrJ3iW8jDJKSmCySxGo1qnA+63QpNRrdUBz1K0rxBxLEXTKalpgr1sZqNSoyMQ56bolNTUuurKzKyuNrNJqdGSwLvcdEpqan11Vaes7KtC5HK5U9LSfnvkFWFHtaDQBzkE1VWVnbtkGRqN2sRkymlXqLU8QyFSriAJysPodFqbxZqant5QV5OZ1bWhtiY9s7PFZNQnJrsdNqVGy3koRMrlBFAeRq/X2azWlLR0Q11tZla2cLDZaExIunow7Qa5UkYg2sPqtFqbzZqSlt5YV9spK7u+pjqjcxezsTExOcVltyk1Oo52CQd7GE6rUdts9pTU1MaGuk5dhIOzzEZDYnKK025Ta3WM2wVypZxAHpbTaNR2uz0l5crBHdqCv6comgQ+PT09Ozvb7XZXVVUhhHr27Hm66KxcqbSbTdqEJNrtVGn1ToqmPSztdiqUahkJiJAxNEWQpEIuA0LGsizHMkqViiAIHiHa7VKptSQJCAiP2y1TKuUkAYTMQ1NAEEq5HGQylmU5hlEqVQRJ8Ahol0Ol0ZIEgQjSQ7lkcqWcJBBJsh4PAhCKcCzHeGiVWk0QBAKgXE6VWqPVJ6iBuenGG0+dPkPKFS6bZdCgQSRJVlVVmUwmHki1WrVr57ehsSBqf7RardvtRgg5nU6tVuvz7cCBAwsKCnyEr732Wk1NTTCN5ufitb6BAAAgAElEQVTnV1RUBFPDihUrSktLg6nhzTffLCkpCaaGKCFwC9I0PWvWrOBbXLduXfNfhVgKCgrWrVsXvDKzZs2iaTr4eiKIhD4oEORv+Pjx4+vXr5dcHCH0wgsvCPmZpfH111/v2LEjGAWiBMkWRAi98cYbly9fDrytHTt2fP3114EfX1tbu3jx4sCPD4NKYgnHG2Fubu7//ve/3NzcCxcuPPDAAyUlJU2/HT16tMVi8Q4xCRQWFubl5fmPNe6fM2fO9OzZU6fTSa6hqKjo2muvDSbnwC+//NK1a9fExMQ2j1y8ePHgwYMlN9TeBG5BnudPnDgxaNCgIFssKSlJS0tLSwtqut5oNBqNxry8YBMA/fzzzzfeeKP/UZcOY0EfAv8Nt0hjY6PZbBbeV6RRUFDQv39/yTmwKisrCYLo2rVrm0d2VAuC+Juh8PrYrVu3AI93u90lJSX9+/cP8Ph2UikYC4bDEc6ePVulUuXn57/yyissy65atarptwaD4eTJk+2tQ5QzYMCA4LO6tx/Ygm2CLRjrYAvGOsFYMByO0GKxPPbYY4WFhUIKmLAFv8eECmzBWAdbMNbBFmxXwuEIMRgMBoOJWqJ3uTAGg8FgMGEAO0IMBoPBxDXYEWIwGAwmrolqR1hUVBTM/oetW7def/31ycnJw4YN81lt3CbBR/YLpvUOQJC2g6AvYKhiM8a5HQWCsWYEu2E8227IkCHEVaZNm+bzbfML6/9aNa3tiSee8GOUFk0WeOWCqmKVDwHtt0UxSCwWy0033SRZw4qKCr1ef/ToUZfLtWrVqsGDB4sq/tJLLz377LMURT377LPz5s0Lc+uxTpC2Q6G4gEFaMFRqdACCsWYEu2E8247n+dTU1KqqKrvdbrfbhZ34TfG5sP6vlU9tc+bM8WOU5iYTVbnb7RarfEiuWJQ6Qp7nx48f/+WXX0q+me7fv/+pp54SPjc0NKSlpYkqnpeXV1xcjBAqLi7Oy8sLc+sxTfC2Q6G4gEFaMFRqxDpBWjOC3TCebVdbW6vX62+66Sa9Xj9u3Lj6+nqfA3wurP9r5VNbTk6OH6M0N5moyuvr68UqL/0yNSFKHeGyZcvmzJmDEAr+nZVl2WnTpk2fPl1UKZ1O53K5EEIulyshISHMrcc0IbQdCuIChsqCQaoR64TKmhHshnFou5MnT44YMeLkyZNGo3HKlCmTJk3yOaC1C9vitfKpTS6X+zGKH5MFUvmkSZMkKx8M0eIIe/fu7R2q3bdv35133unxeJDI7te0EoHdu3cPHDjwpZdeYhhGlD7+I/sFiOTWY4uQ2A6F1HwoRBYMXo2YI3hrRlU3jB/bNb/sAjU1NSkpKT5CrVbrDT3ovbCBXKuamhqCIPwYpTWTBVi5j6qtKR+qru0lWhxhU+bPn+8zkXno0CGxlfA8P2/evDvuuOP8+fMSdOjVq5cQa7ikpCQ3NzfMrccuIbEdCsUFDNKCoVIjpgnemhHshvFsu4KCgiNHjgifGxsbO3fu7HOAz4X1f618apPJZH6M0txkoirv3LmzWOUDuyRtEI2OsCmSB2QOHz7cu3dvs9lsv4qo4rNmzZo3b55gwrlz54a59Y5BMINpwV/AIC0YKjU6DNKsGcFuGM+2+/777zt16vTLL7/QND1z5sxp06YJ8v379wsffC5sa9dKON6ntn79+jU3Sms1o9YN0WLl06ZNE6t8SK5Yh3WES5cuDWZ9rNlsHjVqVHZ29pgxYywWS5hb7xgEc9bBX8AgLRgqNToM0s49gt0wnm3H8/zatWtzcnLS09OnTJlitVoFufci+FzY1q6V8MGntsuXLzc3Sms1o9YN0WLlVqtVrPIhuWI41igGg8Fg4pqo3lCPwWAwGEx7gx0hBoPBYOIa7AgxGAwGE9dgR4jBYDCYuAY7QgwGg8HENdgRYjAYDCauwY4Qg8FgMHENdoQYDAaDiWuwI8RgMBhMXIMdIQaDwWDiGuwIMRgMBhPXYEeIwWAwmLgGO0IMBoPBxDXYEWIwGAwmrsGOEIPBYDBxDXaEGAwGg4lrsCPEYDAYTFyDHSEGg8Fg4hrsCDEYDAYT12BHiMFgMJi4BjtCDAaDwcQ12BFiMBgMJq6JvCMsKSlxu91iS3Ecx/O8hOYQQizLSigIAAzDSCvIsixCSFrZ6CdAC0q+CJILSraX5IIxirQ+GExX4nme4zhpZSX3fei4li0pKbHb7WK7CcdxYq0gzegSunAwvxAJSHeELMtOnz49IyNjyJAh1dXVgtBsNo8ZMyY1NXXs2LFms9mP0MukSZOKi4vFtu50OiV0XQDweDw2m01CQQBobGyUVtBms3k8Hmllo58ALSj5IpjNZmldwmQySbhjIoSMRqOE5sJPZPsgy7IWi0Wa5m632+l0SisbTG+S9pOIfiZNmvTjjz+KPTWXy+VyuUQV4XneZDKJKgIAVqtV7CMIRVEOh0NsQ5KR7ghXr15ts9kqKioGDx68aNEiQbhixYprrrmmtra2e/fuK1eu9COMRWpqao4ePfrTTz9FWpG4oLGx8ejRoyUlJZFWJHqJoT544sSJo0ePXrp0KfxNY0JIYWEhRVGR1qIdQFIZOHDgqVOnEEI2m+3nn38WhHl5ecXFxQih4uLivLw8P0Iv/fv337x5MysSk8lktVrFlmJZ1ul0NjQ0SCjIsmxa9jWJeYMSUtLEFmxoaHA6nX4O4HlesiEizsCBAwsKCto8zGg0UhQVeLU3D/+9OqPbmIcebWhoYBhGgmJ1dXUcx4ktxfN8bW2thObCT6j6YIAW9MHj8RgMhkCO3L17ty6zu6ZLz8em/lmQOBwOq9UqtkUBk8nkdrullZX2k4h+Bg4cuHv3bpZlRZWy2Ww2my3w48vKyjQ6/cGDB0VqhxobG2maFlXE6XRaLBaxDUlGLtmDVlRUfPbZZyNGjOjZs+e//vUvQVhdXX3NNdcAgPD46Ufoxel0zpw5c8iQIaJat9vtMplMwrOJx+NxuVxI/JwTQshYXQFvH9Muvl7s0JnFYqFpWqlUtnZAUlKSSqUSq1LH5nLZJarf2J3ffZrVtetzM2YMGzZs3LhxkVYqughVH7TZbKNGjTp9+rSo1hmGsdvtgQzHnTt3DnKHunsOc7oPNTQ0AIDL5eI4Ttq7hcVi0Wg00vpLY2MjQogkWxgJw32wTex2u9sZvuHKcCLdEQqPEmfPnl27du3TTz/9ww8/AABCiCAI4YN3XqdFoZeEhIQvvviiU6dOolpXq9UymUyn04lVm6Zpp9OZmpoqtqDXdxIEIVZbuVyu0+lwNxONtQ7l3M6XHl+97VhhSRl2hD6Eqg/q9fo1a9akpaWJap1hGLlcHkgpvV5PECQAqFRK4Xi1Ws1xXGJioqgWBUiS1Gg0arVaQlmWZdPS0lp0hC0KMU0J5+qVMCPd9hkZGc8//3yXLl1mzJhRVFQkCLOysiorKwGguro6OzvbjzCmkbxUJ6pgQ7TUon0hZQjx/HX3dtxVt9IJVR8kSbJLly6ydsPrYwiCbL9WgkR4UMD44cnpz0VahfZCuiP8/e9//9FHH9E0/cEHHwwaNEgQjhkzZsOGDQihDRs2eJ/fWxTGKh63m0XJaRl9ru8faVWCJYaWWmBaJE77YIejqKio6eBW1D2MXqXs4sWItBsGpDvCZcuW7du3LzMzc+/evevXrxeECxcuPH36dLdu3c6ePbtgwQI/wliFcfMMzb76syf212B/+umnL7zwglarXbhw4V//+ldBuGXLlhkzZqhUqhkzZmzevNmPsCkURfGhplkjKORNhJB2tFPrxGkf7FhYrdapU6c23cYQtQ+jNOUGTaJ3NrojIX2OsHPnzrt37/YRJicn79ixIxAhJuKEaqmF1WodOXJkRUWF/+YsFovb7fazYqgpW7ZscTmdkPHr1KzHwxgMhoBPDgBAWNMkdu4HIWQ0GmUymahSycnJ4Z8Dxn0w1kEITZ06dd68eQ8//LBXuGXLlq1btwrPnePGjVu2bFlrQi8sy/7www+DBw8W9bulaRoAAuySAMAjHgBoDyMUDByPxyOsGhWlG8OIa0ihUEie6JXuCDGxTqiWWiQlJX355ZeZmZn+m1MoFAGuGPrqq69mv7GGcjoBwDtzo1Qq22yiORkZGRIcIQBIaAuDEcuKFStycnImTpzYVCjhYZSiqC1btjzzzDOiHKHwGiq2gwib0EQVcTqdBEGI2lPvdruF1ViBF9Hr9YE7dR+wI5QE7TTZXZndc24ZdOP2zV9GWhuJNF1qsXr1akEorKrIzc1tvtTCR9h+uFwuWY9BYPqmvRvChBkPTRsMBo1Gg1emAMD+/ft37tzZ/J1e2rrfFStWpKaminKECoUCABISEgI8ngACEL9j53elpaXeOelAQAglJCSI8lIul4thmKSkpMCLBANeMSwJu4FhmIYH8i9WVEVaFengpRaYsGK4tHXr1uxres6a+2KkVYkK9u7de/DgQaVSKXg4giAOHz4M0b72HrlslkhNircf2BFKhZRDclaklQgKvNQCE1YstTBwLPNgPtvR7qISWbp0qTeyCQAghIYOHQpR/jDa4VyggHRHOGTIEOIq06ZNE4RRu/AX0xxhqYXFYjl48GCvXr0EobCqoqqqatu2bd5xiRaFGIwUEDp16uSCVxd99913kVYlSonqh1FeYr6RKEeiI0QInTt3rqqqym632+127wxT1C78xWA6HjH5MHrh8GkDs3pv8c69+yOpRpTRdEUlfhgNPxIdYX19vcfjGTduXJcuXSZPnuxNbCRtFxoGgxFL7D6MEt37w+9GRlYHDKYpEleN1tXV3XzzzW+99Vb37t1nzZo1c+bMzz77DKQG3X7xxRe//FLc2ksh6LaEtGQ0TbtcLrGL1hwOR7PnaAQAPMcF8nxtsVg8Ho+fnQN6vV5YwYXBBIj3YfT8+fN33XXXBx98IITflLALjabp//znP3l5eaIUYBjG6XQGEvOToigECOA3ne6XM2dWrlx544033nbbbaLadTqdHMexknICO51OjUbT4oYBtVotarE+piMh0fADBgzYt2+f8Hn58uXXXXed8FnCwl+FQjF8+HCNRiNKAYZhZDKZ2FIAQJIkQkhsweEPTLh44cJvZQQAECQRSFUURfmPl48D/mLEEsKHUf5q5ilRCjRd6CGaS8eO11l/Wr912n01t956a9jaDUpnTMdFoiM8ceIERVGDBw8GAKVS6b3FS9iFplQqR40aJTaWvMfjkclkEiLQEwTBcZzYgheKCt1LimFWl2a1kYFUpVKp1Gp1tGWfGDJkyNGjR4XPzzzzzLp16wDAbDZPmTLlyJEjQ4cO/fe//52SktKaMMzwPO92u9VqNd6CJhDCh1GNRvPSSy8Fvp9MgGEYnucDKaVWq4nfvg4CxxJ5Q7mUbgqlWWy7LMtKzj7hcrkSEhLwc6cEGhoaaE9HTMkLAJLnCJ1O54QJE4qLiz0ez+uvvz5+/HhBHtULf0MO4s1G499mzVm//p+RVkU0MTbDZDccOnhAp9OdOnUqAq1HJULOd+Fz84dRiNJdaJhYpbS0FHXMrRMAAITkEYb333//rbfeslqto0aNevfdd4XUYhaL5bHHHissLLzxxhs//vhjYXVTc+F9991XX18vVFVdXZ2eni72bUnY0SnhyU4YGBFbsLyiAqV0BeNlSMkGSw0kdAJHI2iTwWUBpUZN8F06X4nI1bt3788//9ynuMlkirZ8hHV1dbm5ub179/bOMAlJFnv37r1169Y+ffqcO3dOmH9avnz5m2++mZmZqVAoGIapr6/v2rVr06qqqqq0Wm2br4kcx5EkGcj7nMPhMDooRLtAqQGEgKFAqQHEkzzXuVN64PEpAm+xKbm5uStXrhQbYk2lUokNTxokhw4dmjhx4oEDB3Jycl588UWapt9//30AmD17tkqlys/Pf+WVV1iWXbVq1fLly4WvUlNTTSaT8KFpVdL6IEKI5/lAztrhcDTa3VfvNAgIEngOZDIg5YlyPk1kclCe54WFsq0d0GIfFKivr5cQdS/iLF++/IsvvvBzQHV1dWpqqtgZH1F3UZqma2qEQXWUlZUl6tfSpsl86N2794YNG8IZWUaiIwySrl27/vOf/xSb3jb6KSkpyc/PLyws9JFHoSM8derU7NmzvTNMHo9HmGHS6/VCECy3252ZmWmz2Z599tnExMRHHnkk0iqHg5KSkqVLl27atEmsI9RqtWFe7hT4w2i8WbDFPigQo44wDi147NixcDrCiK2Suu6663xeLDoAMdTBRM0wde3adeDAgRHRM8yQJEmSZGJiYvRv1SIIYvr06dOnT/eRt5hoIq4sGGkV2gVswfZtNPxNYqIBUTNMGAwG04HBjjBOEbXcCYPBtAc7d+7s27dvcnJy3759d+3aJQijKAxQ3IAdYZwydOjQRYsWjRkzJjs722w2r1ixQpBHS0hDDKajw/P8Y4899u6775pMpiVLljzxxBOCPPKLtOMPHEkhThE1w4TBYEIOy7Iff/zxyJEjHQ6HSqVKTk4W5P7DAGHag2DfCIuKinQ6nfffaA/4i8F0OALpg5goRIgl4nQ6ExMTx40b9+GHHwpy/2GA4gGe5x0Oh8PhsItBWtQ9gaAcodVqnTp1qsvl8kqidDs2BtNBCbAPYqIWvV7vcDiWLl06c+ZMQeI/DFA84M1ULIpgWpQ+NIoQmjp16rx58x5++GGvUELAX4HxD0+qrDNIVkYUShK2b/oiPT29nepv/tvlrtJaEQn7viPI22ve+e9XW8PTFgnw6gszx44dG57mYovA+6AP2ILRQHl5+dq1a1etWqXT6f785z/n5+cLcv8xKQU6tgUJgtDr9QzD6PX68LQo3RGuWLEiJydn4sSJTYUSAv4KHDx02DJuJejTJOsTOOp/Tqao9gqax3Gc0Wj0EVosFpqm/YRESUpKitR2+6KioltvvdXpdAr/BhJW9Ezx+ROqvtB/dBjUUx54T9jOgWlO4H3Qh45tQY7jWjtxg8HAsmyLO9VSUlKkxS+VTFZW1j//+c/Ro0cPGzbsiy++8G4TFFZu5+fn+1m53bEtGH4kOsL9+/fv3Llz9+7dPnIJAX9/pddgSAnHxjVS2Y4uRyaTNY+YI5fLoy2yjEBrA2tfffXVnDlzVq5c2epEfWZueFLKkae/DkMrsYioPtgCHdeCMpmsSxff+PhXlCHJ6Ikso1Qqt2zZMnv27LKysj59+mzYsEGQL1y48LHHHuvWrZsQG6jV8h3XguFH4g9i7969Bw8eVCqV3sHcw4cPQ8cK+NvQ0DB58uQuXbp07dr1L3/5i91uj7RGIcY7sNZUGNNZlDmO69OnT1NJB145IqoPxgrNLdiasGNw5513FhQUmEymo0ePes8xppPRNzfW1q1br7/++uTk5GHDhpWUlERKMf9IdIRLly5tmtkLITR06FDoWNknnnzyyR49elRUVFy8eDElJWXx4sWBlLJYLO2sV8iQPLAWnaxZs2bw4MHnz59vKpSwcoTneYvFYhKJhBzRQSKqD8YELVqwRSEmOmlurMuXL0+ePPnDDz+sra0dO3asd69ktBHiIYIWt2PHxB7tp59++u233xY+//nPf37rrbcOHjw4d+5cpVKpVqvnzZu3adOmQOoRArpHP8LAWvORz5hYsdbcWADQr1+/V1991edICS+4BEFotVqdSKInuXmMdjdoxYItCjERJ0ALlpaWTpo06fbbb9doNI8//njUPtCEwBE2zV/R4kt9TLzpP/TQQ1u2bAEAmqa3bt06adKkQYMGLV++3GKxNDQ0LF68OLZej9okpgfWmhsLAEaMGDF6tO/aAQkvuARBCMFXRRHZaac2+2C0EbgFWxRiIk6AFhw+fLiwP5LjuIULFz766KMR0bZNomLSOBoYOXLkL7/8Ul9fv2vXroEDB2ZlZX300UfCk/Xtt9/es2fPVJFZ06KcmB5Ya26s1o6MiRfcOCRwC2KiE1EW3LNnz80335yUlLRmzZqwaSiKaBnPiThKpfKBBx7Ytm3bgQMHJk+eDAA6nW7Lli3CnoeDBw/27t070jqGg0BXrEWU5sZqjUC2ZGHCT+AWxEQnAVoQIfTKK68cOXLk888/z8vLC6eGosBvhL8yceLETz75ZPfu3RMmTACAF1988ZlnnrHZbLW1tfPmzXvuuecirWB7EXMDa9DMWK0REy+48UmAFsRELYFY8OjRo1u2bNm2bVtWVpYQNS2cGgYOdoS/cs899xQUFIwcOVLI9L1q1SqTydStW7e77rrrySeffPDBByOtIOZXfIzVGjGxciQ+CdCCmKglEAseOHDg/PnzKSkpCVcJp4aBI90RBp5JK1b2cqnV6r59+3pf89PS0rZu3Wq1Wn/55Zenn346srphfPAxlpemb7cQOy+40ojpbHYBWtCPEBNZArHg/Pnz0W8Jr46BInGOUMiktXHjxhEjRmzevPmJJ56orq6GVuKSBBKsJDE5lflqFqHUBnMyAULbW7g7MAxz5syZysrKe++9Nww6xDSpqamqPR8rLv8YhrbY0p9g/PU+QmwsENkHfcAWjHWwBUOLREcoKpNWIFGA9+3YcuzYMcmnIY5n7m++BHT79u1//etf161b5yciKEbgjUULBvQN29Kh+2+55RYfETYWBJfNDlsw1sEWDC0SHaGQScvhcCQmJnq3oEEQQbdzcnJycnKkKRMSHnzwwZDMAiIEzcejLBaLx+PxE2tUr9crFIrgWxfFzp07Z8+eXVNTk5WVtXr1auHJLpCg2yqVKrIr/UJlrJhGVB/0oWNbkOf51gIiOhwOtVrd4qZPjUYT/qgIW7dunT9/flVVVb9+/davXy+sq8R9MPwEtVgmwExa8bWXiwCNeMK/HVsYWHv33XdNJtOSJUu8oY9wNrvYAmeza47YPHbBZ7OTRmvhx3AfDD8S77/l5eUvvPACAAiZtIqLiwV5Rwq6LQ0CQN0MlUrVXNgUmUwWZj29A2tOp9NnYC12g26HBMF/iCX8qwBE9cG4Qshm1yI6na61r8L/Otha+DHcB8OPRNuLyqQVSHotTJgJZmCtY8PzvNlsFjtSHf6MksFks8NEA8OHDx8+fDg0Cz+G+6DQBxmGETWkodfrJU9YSp8jDDyTVkwEK4lPhIG1NWvWzJw586effgI8sAYgk8nS09ObJ5WMNoLNZoeJDvbs2fPiiy/ee++9S5cuFSS4DwqB7xmG0el0gZcK5p1eekkhk5aPUNi2FYiwo8Jy3O7du3v16tWjR49I6+KP8vLytWvXrlq1ShhYy8/PF+Q4JlkMEXgfxEQhrYUfw32QIAghkH3YRllwZJkQ46bosX/6y3//+99IK9IGwsDawYMHEULNB9ZwTDIMpr1pLfwY7oPhJzJBtzUazcKFC/V6PQB4PB6FQhGRVVvNYRiGJ0jgOJDJgWNBJgeOAbkSWM9v/soUwLFAygBxQJCAeAASABmNRqTQ0Df/MdLn0TaBD6xpNJqvvvqqqKhIGLFpPnkWEgvStAeEGhAPBOn3AwIAhVze5lJb7xBT4DQ2NorXPdoRLOgnFVxo+yDP8wzLAUEAQgAEgPevlyZCgrhi1l//Ct1KBoDkJCF2HVkMWdAbfswrEZZcSeuD8gB6hGQQQs1TT7fT61pELEhEJObNL7/8smfPHuHzqlWr/vCHP3Tt2lVUDRRFkSQpYWqUZVmPx6PV+oawcTqd2//37fkLl0w9RpBnvuUHPUQc/xwNfZI4+AG6dxZ8uxLGLoItr8IDL8N3b8LQJ4gfP0fX/5648D3q2o80lPL6DJJx8xxLpnRB19782i3qpgkqTSaTTqcL82KKUOF2u4WMYq0RoAVdLpdSqWxtHP+lV16lkruTSjWqOoMy80hShurOo855JM8hQynq3Jtk3MhUhTJ7k5RVhTx/nTxRWFDgB7vdrtfrxd7f09PTR44c2blzZ1GloplQWdAHjuMoimo+i3P27NmPvz3s1mWStlpen0G6zLwqgWTcPCknCeA5llSoedpJapN4h5FIzETmGkjrDoYyyMwl6s6h7OvIqjN89wEKl2ncDZ3vuOMOn/pdLpdCofCzmikjI+MPf/hDi1/V19dnZGRENnmkBAKx4NixY3Nzc0WdGk3TEJg/KykpWbvuA0DoyrMLkARwq996K8CGnE6nqLXxGRkZ48aNYxgmbGERI/NG2Ldv3759+wqfP/roo0mTJt14442iarDZbDKZTNRUqgBN006ns3lkmWkzZh5zpZNMGVx3D3HxMAwYQ5zahm5+GI58BLf/Cb57E4Y9BVtehdsfgz3vwICxcGo79BlB1J5FPW4mOZrPyCXdFp6hibRuURpNTyoajcZ/5o0ALej/aeDV15dRGT0IbSKqOw+dckiZnGssh069SM7DWWogM5d0W3iHEXXOI201St71yCOP3Hrrrf5blHbXQwjV19eLKhLlhMqCPjAMY7Va09PTfeTffPPNFwVV7vTfkTVn+E65pOkyn5BJui28XE0QCBgPoUkAu5FIyQJDGZHVBxEkXDsIGBp63gKUBXJuJ+wNkDtUZiq7886+M2bM8KnfbDZrNBq1Wi1K25gmEAtOmDBhxIgRol6ghbADgQTC3rt379oP/vmrIyRkBILAE/IYjcaEhARR7y0ulyvwg4MH5yNsQt5Qsq6Yj7QWGEyMYrVa//y3uUWFJyg+OdK6YDAikD5EsHXr1uuvvz45OXnYsGElJSWCMKazT2AiAkLojfxlNE1FWpHYI9r64I8//rjzaMF5TV5oFv0jZLFYqqurw/xygIlDJDpCUcGBcMQgjB/ef//9xRu20LTvVLwfWA81b/HS5Sv/r/20in6isw/KE9KgU2iCBnMNlxbnr+jRKy/612BjYh2JjlBUcCD/EYM4jjt//jwTaXhe0phoK/OBHMeFo/XgiJ73CcW1N4pa0uI2NRxokG/+366QaxJDhLAPIoSMRiMfNKFdecfbGrnxSxW3/wEhFFQwvx4AACAASURBVLxubRJCzTExh8Q5QlHBgfxHDHK5XAsWLLjnnntEKWC322UyGcuyYjX3eDxOp7PplHJhYeGERx+jaBpGvyqmH/96bNPuL3ykadpqtXqFNpuN4zg/c8WJiYlhzmYivE/s2rVrwIABa9eufeKJJ44cOQJBZJQMN936g+VopJWIJCHsg3a7/dFHH/VGKw0QhmHsdntT52exWIR/BdnVvwi8vaVZT/GlSZGrAmS32w0Gg1ditVqF+L2itBUwGo0A0OL6qeTk5Bhd1x0hUGFhYf/+/SOtRmgIarFMgMGB/EcMSkhIWL9+ffO1Z/5RKpWSV42q1eqmq0YPHz5sv+mPsnP7AEDMQnvi1z/Eb6QIQKvVNj0jkiSjbfuE930CAB5//PHly5cLcgkZJSmK+vDDD1etWuW/RafTiRBiGMZHTtM0avXV+le8d9MrN0mGvlBWcfvwu9eszPeuQG7eooTMHgghp9Pp3d0cIGq1OvxRmyFEfTAxMXHjxo2ZmZmimmYYRq1WN/2dp6SkCK0QTfrFFYlwxG97Sgs0KXJVQCQmJjbVTalUBrNqNBa3T0Sc5uNASKFbnL9yyxefRESfkCOx64oKDhQDEYNUOiBDeRf7/vvv8/Pz77vvPrFL0sNGCN8nBM/R5rAYukpzuZQTMFfZWNmZ8jqz2dxaDa21KE3PaKOj9UFMFPPvTz73FRHARXsXEYHEu78QHOiHH36Qy+XCs7MQJgZnnwAAVHJoL6CD679KTk6OWkcoEJL3CY1G8/zzz7e5G4lhmBZfi9VqNdEs9EhzCAB09T1DOJLQJitUpFarba1pl8uVkJAg4Y1QKCiqVPjBfRATNpwdfeGuREcYeHCg1oQdGYRQ37vkjrpI6+EP/D4R68RLH0TIbDZXVVWlpqY2DwiFwYQEiWPl8+fPR79FkAuR76uqqrZt2+aNjtOiEBNZRAX8xVGAo5Co6oMGg+HWkff/cepT7lC/OjD1pQuWvNEzt/fGjRtDW3OUwHFcnz59mkqicDe2xWLxkSAETqe4efRoBk8axyne94mEqwjyhQsXnj59ulu3bmfPnl2wYIEfIQbj5ezZs8VVjcbrH+T4EE8cIYeRfWiF6paJoa02SlizZs3gwYN94qFH4W7sM2eKfEU8t+/b7WFWo/3AjjBOiar3iWBgGEbCLhpMyJGptKAXt/Yb069fv6bR+QUk7ARtbzjeZ2UAAl9JbBPXsUbLyspGPzK5tqIUbnsq0rpgpOC2Gu++595bBg/94dCBSOuCwYhmxIgRzYUSVm5brdbRo0eXlpaKCrotTIgEE8EuwPD0ZrPZ7Xb7SRjSHLfbzTAMRYmIvBjMTtC4doQ7duy4KMv26OhIKxK/FBcXHzt2jOc10oqzThsas4BqOBBSpTCYSCJtJ+iaNWsyMjJEOUJhL2YwC6QzMjICOYwkSQnZJxiRaZiC2R4a1NBoTEzz+odMSAeVvj1q5h2m11at6Xvjrd7Mi5jm3Pn70Z9/e4ANSYzmuCQa+uChQ4cOHjzIcniAOjQIi7QBoPnKbR+hF4Ig1Go12T7s2Lmrtb1Nf//739upUQkEc82lF46Vad7W8Hg8zaOchBC29kJD7zGXuQQhqlN0EvHbqNNmYbP7hbDCuCIa+uCpU6dGPfKnZf/a7Ha3Y/4Qzm6at2T57wbe8t1337VfK1FCtK3c/uBf/2l5l68mcd2/PwunJu2HdEcYK9O8LVJYWJiamT1n7tx29YWQfg2p980AHD1Ew200JJgbG5YtW1ZRURFpRcJNNPRBl8slS+lC3zYlhHU2h2korb/+4csoJZofK0NFzKzcjvroS4EjfY4wVNO8Tqfz+eef37p1q6jWhaDbHo+I3D0CNE27XK7jx48T192FqKaDOWKNin7903ooYSH8mMVi8Xg8fiZy9Xq9qJnkkNCvX7+cnJwxY8Y0FUqINRphjBVVBvOS1f+45ZZbhJ9Z/BCqPuhyuV5++eXPP28WRssvDMPYbDafuNsC6NfO4fNFix+bHfWboNtXSb+WNF0QehNN09KWRVitVrlc3uIwWkT6oIDPBRQWafsc06IwbLRmLA9NeScvY5oQL5aRMM2rUChGjx6t0YhbLsEwjEwmE1sKAEiSRAgplcqgTUe08KlZKGHiaoxgjUbjp+sGOcAtjVDdRu12+8SJE48fP+6/ObPZTFGU74T51VtA8+wEVwXo1wNbfOxAiEjOUiYkWq3WxsbGpmVNJhNBEGKvLULIZDKJjaAd/vwhrSGhD8rlciGdk6iG5HI5wzAqlYpoFkC71c7VZtBt4Sui5c7F2RrnLX5Du+LNd1Ytu//++0VpK6BWq1uLwx6RPhgTWC3mlj0hz10oOklRlIT7cLQRYkcoIUCXUqm8++67xcaS93g8MplMQgR6giA4jlMoFC35LPEQbRUlCIVCoVKpJCeOCTMSbqNarfbll19uc30Xx3FNY40aDIap02fSPu/0za7krzdZ4tdcH6jZ4Sztem3l2yOO/LBq+RveshRFJSUlSXCEQkFRpUSt1mtXpPXBsWPHiu1NMpmMpmmlUtnC7//X1CyhhGmscg+ZRpfsoShKWvYJoSdinycKyt3K/gqEQK7++OOP//KXv4RXo9AT4h9EtE3zNsdqtT4xY/ai/JUShlXjAQkr1mQyWV5enkIkGzduPHDZ7XWrbd40/R9AG2tOQ9ddBw+LVSNURM+9Nfr7YFAkZ5OaxEgrgbkK20H2noW490bzNK/H45m3aOm4hyd9e7jgorxrs1gJGIDw3kbJjJ6hqgoAICOnztA4asLDp06dCmW1sUbY+qDRaLxn/KOjxk90OsObmoDnysrKCgoKmua+xrQfjYZGP9/OnvuCyWQKmzLtRLBDo9E/zevl9OnT7/xjvbvrIJmagtTuYKsJT7scx7EsG/357QRiOHeBtdbqpA4WXqitrR0wYECktQkfkeqD5eXlF6vqHUOnkz99Edqa/UMZLi9ZfWrJspUbP/nP2LFjw9l0HLJt27YyP+uxZQqnw7Fs5f+tWp4fRqVCT7xEltm+ffvOnTtBnQDd+0NRmFwgAHgaqyZP/hMA+uabb0aNGhW2dgMnhh5l2kaTpEjpDACFhYUGgyE7OzvAyBcYaZAKJYR9gxCiHPRDK5MK/hvmduMTl8sFiG/1a5IEQvXZV18vf+P16Jkml0C0TGy0K2VlZQ9Pnrru6GWaDve8IGdvRE9/nHBDC+sz45kjR458++23PN96B5MK4tg9e/YMuXf0mIl/aHMhKwaD8U9RUbO8Ez4QUF1d9fe//z0s6rQXceEIEUIKXRJ/44ORVgRzhbvuvW/XqdL2cITuxup3//WpW6ZV9hx0+PDhb775JuRNYKxW6/DRD419+A8ulztSOiDWs3379jVr1sRhIIWwQVHU8jYDaCAEPHfu3LmysrKwKNUudPyh0bKysp9//pnH0SyjhtraWoQQdMppj8p52s31G0OeP+iuL/vou+rd3x+76aab0tLSpK22xzTnwoULP/zww08nC123PU0e/Xek1HDVlv7rJ7n8650///xzbm7ulClTrr322kgp01Gpra3lCEUAwUbQp1//r6Ki4sMPP+zSpUs4NAs1HfmN0Gg0njx58qYhw//80utuMek82gOWZT0eT6wsmWk/vvvuux55fcOwdwWxDNvnrqqa2mt79sLvhcHj8Xhmz1805K77brj1jr++/DrF8JAc0Vsez3F3PMUSsv9eYJet3XDp0qVIKtNB2blzZ4BHWs3GHTt21NSEb/lFaAmHIwxP9gmbzXb8+PEjR45UV1e/tPiN/rffmd2r7x0PTDQ3Njgmrm6nRgOEsjSMf/BBtVp98uTJyGoijVBZ8IkZsx/545+YIU9AeGIyMW6kSVb1GTp/6coHxk7YuHHj5cuXw9Fu9BG8BSsrK99//x9HNQM9hMI55vWQaygRBHDbH1Wdrjl//vxPP/0kJNjrkIQ/h8/OnTuffX5uQIciHnEcAKxbt06nT4jF3RThcITtF7KZZdnS0tLS0lKXyzX5mefuevjxO+4f37P3dSvz3zh900zaanDODPSJpl3hnFb03PaEnjdUV1eXlpbGXEb14C3odrtnvfjy5//6wJaaKz6sa1DQDeUlZNa3+w4+PmPOyy+/vGDBgq+++urUqVMWiyWcakSWYCzIsuzLS/KnPjODI+TQ87Z20jAYXKb6OUvfuv2eB9Iys7QJifPnz3/vvfdOnDhRVlbWvlH1w0jYAt+bTCabzbZq1aqnp01HgXdVjgWldv1/PnFxkNGtlzIxZcGCBXa7vf30DC3hmCP0H7KZ47ji4uIbbrgBAHie37JlS3l5Ocdxw4cPP3nypMPhUKvVAwYMaGhosNvtxcXFarVaqVQqlKpPNm2z2h1V50+DSp+UlOSkPeyDy4mNcz2Pr4f1U0CdAABQcw4QgoZLwLFgqgSWBmsdol1gb0RuG4AMEQhoB9jqgaHAXIM4BkyXgefAUAYIQf1FQAC15wAAas4CANSeA4SgsQw4BsxVwFBgrUeUA5xGRDuA9SB7I3icYK1HLA3masSxYKxAPA+GUpfVMnbsWNAmpyUnqRVk0akTOp1OuAgymSx6QpM0R5QFv//++/Lycoqizp4vcTicRpMZsR7gue9+qQOEgOeRzQAIgLIjRyMAgNsOCiUCALcNEAcIwGVDchkAApcV8QwgBE4rou2AeHBZEOUAhMBpBtqJEA9OM/K4gefBYQKGAp4DhwlYGgEChxFxHgRyUKhBrmRI9WffHQLnNkKm5J0meVInGUd375nbJS3p4YcmHD52vLSqlkRcnx7dSJK8/fbbKYoym819+/YVwvIJBsrJyTEYDGfPnuU4bvr06YqrYZo7kgUPHDhgNptLSkpyc3MLCgoUCsWq1e9y/ccRHgoay4FjoLECsR4wVSKWBlPVlb7jcYOtDmjnlR5hNyC3/UpHI+WIdoDMDYCAZYFnwG1FCiVQdmQzAOUAWz14nGCrB9oN1jrkcYOlFhgKLDVX+hHrAeNlxDFgrEAcC43liOegsUzoqpzbzoxeQG5f4hm3hPx25YqPNvMOI7IZQKYYPW6CwWylKYqjXX+b9nRRUVF6enppaWlOTo5cLn/uuec6jAVLSkqGDh3K8zzP85s2bUpLS+vcuTP8dn+UcIJCxESEUEFBQVVVVV5e3u9+9zsAKCoqeunl+WabAyHkpmgATlyYPJ4DkgCe43mO58hlb65Z897fc3JyLC5P7+6drTa71enO7pw5buzoCyUl6enpW3d8+/qiV7Va7alTp0aOHOnVk+d5giAoimJZ1ps0+PTp0zqdrkePHl4buVyuM2fO/OlPf/KGBQ7GgkQYZq30er3BYNBoNG63OzMz02azNf02NzcXAI4dOwYANE2PuOc+hgeedirlMheLlCoNwVI8y5BKNeVyUxwvA0QSBMexSKHmPLRSo6VcTo0uweWw6ROTbRZTcmqaydCQntm5oba6U5dsQ11NeqdMU2NjcmqqzWrV63VOp1ujUdEeViknWR5IxIFcyTMemVLN0m6lRke5HFp9gsNmTUxOsZiMKWkZRkNdRmZWfXVlZnZ2Y31danons8mYnJxss9n1eq3LTamVSg/LyUnggCR5VqhQrlJ73G61Tuey23WJiVcqNDamZnTSKGT7vvufN1ht9IRsbhFRFrz73t+bTCZCJrdYbTqdDuRKh9mYkJjIETKX3arX6xEhdzmsep2eI0jK6dDptBwiKLdTq9VxCDxul1arZXnkoSmtRstwHOOhtRotw7Ecw6o1ag/D8Byn0ahpmuERr1apaNoDBKiUKoqmSIJQqlSU203KZHK5gqbccplcoVK6nU6FQiFXqlwOu1KpvPJBpZIrlC6rSafTIYXKYbVqtWqZXGk3NyYlJhIyudVs6n5ND4fVxAORmJTitJlvu+22H374QaFPVpDE3o5rQYZDNdWVXbK61tbVpWdkNjYaUtPTTY2NKWlpFpMpOSXVYjEnJiXZrNbEpCS73Z6g1zscTp1O63K5tVq1y01p1GqKptVKJeVhVAq5h+UUcpLhkIIElgcSEAeEHBBPyAjEIlJOcAzIlYj1EHIVYmhSqeI8tFylZmhKoVIzlFup0XrcLqF7qrV6yuXQ6BLcTrtWn+B02HX6RIfdmpCYZLdaEpNTbGZzYkqq1WxMTk0zGxtT0jLMDbVdunSuratPz+xsqKvr1LmLSk52JAtyHHfs2DEhAOyddw7PyemZmZkJAAghb6xgb8RgwREeO/6z3W6/pmt2nz69EULHfy64ePEiAhKAB4K4Gt73qoNAcPXz1QDAV/8DAECo2XwHAoKUkzK5Si0H3uFyA0GqFWR2dle7w84zjMPlvvfukWazubS0fMSIO+FqIGjhA8/zHMcplUpB1cPHfuickd6zZ0/viTQ0NFy4VHpg397QWBC1P1qt1u12I4ScTqdWq/X5duDAgQUFBWLr/PLLL3ft2iVBmaKionfeeUdCQYTQjBkzpBVcs2bN2bNnpZWNBkJiwXfeeaeoqEhC60uWLKmurpZQcO7cuU6nU2wpmqZnzZoloblopj36YHl5+bJly6Tps2fPni+++EJa2X/84x8///yztLIvv/yyxWKRVjaytGnBp59+uqGhQVSdW7du3bp1q6giDQ0NCxcuFFUEIbRq1aqLFy+KKnLgwIFPPvlEbEOSCcfQqP/I91lZWc8995xWqxVVZ0VFhVKplLBU12Kx1NXVbd++XWxBADh+/LhPGtsAKS4u/uSTT/wkNFi8ePHgwYMl1BweQmLBc+fO/fe//xWb1QEATp8+/d1334n9hQBAQUHBiRMnxAa84Hn+xIkTbe8j/i3xYEEfHA5HeXn5vn37JOhTW1tL0/T69esllC0pKUlLS0tLS5NQ9sSJE0ePHm3xvSHWLbhp06Zz586J2iYkRNJ/7733Ai9CUdS5c+eEwYPAOXv27KZNm7zjnIFQX1/vcrk++uijwIsEY8FwDI3Onj1bpVLl5+e/8sorLMuuWrWq6bcGgyFG11KGkAEDBnTq1CnSWrQKtmCbYAvGOtiCsU4wFgyHI7RYLI899lhhYaEQslnCOwEmsmALxjrYgrEOtmC7Eg5HiMFgMBhM1BK9y4UxGAwGgwkD2BFiMBgMJq7BjhCDwWAwcU1sO8KioiJvcJbA2bp16/XXX5+cnDxs2LCSkpJAigQT6E9Ccx0G/+c+ZMgQ4irTpk0ThP4vtZ9vJbTlRx6IMhhpv+0I9iZpd4woh+O4Pn36NJUEfoX9//59kGY4UU0Ecy5BEbYdiyHHYrHcdNNNYk+hoqJCr9cfPXrU5XKtWrVq8ODBgZR66aWXnn32WYqinn322Xnz5rV3cx0D/+fO83xqampVVZXdbrfb7cJmYdTWpW7tW2lttSZvszkMCuK3HaneJO2OEeWsXr36lltu8TmpAK9wm79/HyQYTlQTwZxLkMTqb4Ln+fHjx3/55Zdif9b79+9/6qmnhM8NDQ1paWmBlMrLyysuLkYIFRcX5+XltXdzHQP/515bW6vX62+66Sa9Xj9u3Lj6+npB7v9St/attLZak7fZHAYF8duOSG+SfMeIcvbt2yeEB2kqDPAKt/n790GC4UQ1Ecy5BEms/iaWLVs2Z84chJDknzXLstOmTZs+fXogB+t0OpfLhRByuVwJCQnt3VwHo8VzP3ny5IgRI06ePGk0GqdMmTJp0iRB7v9St2kIUW21Jg+8OQwS/9uOSG8K/o4RzficVIBXuM3fvw8SDCe2CcnnEiQx85vo3bu3dyx33759d955pzfPraiyArt37x44cOBLL73EMEwgrfsP9NcmYpuLXaRd6pqampSUFOGz/0vt/1uxbbUpD9LuHY/guxKKRG8Se8eIZpqbADU7KT9XuMXiqPV+EWC1gRBIE0jMuYSQmPxNzJ8/32em89ChQwGW5Xl+3rx5d9xxx/nz5wNvsVevXiUlJQghITdN4AWlNdcx8H/uBQUFR44cET43NjZ27txZ+Oz/Urf2rbS2WpO32RwGBfHbDn9vCuaOERP4OI8Ar3Cbv38fJBhObBNI6rkESUyuGl26dKnPVRs6dGiAZY8ePbply5Zt27ZlZWU5HI4AU1qPGTNmw4YNCKENGzaMGzcucFWlNdcxaO3cDxw4AABOp3PChAnFxcUej+f1118fP3688G2Ll1oo0tq3kttqTd5mcxgI4rcd/t4UzB0jFgnwCrf2+w+y2mCaCEmjUmgnBxs2xJ7C0qVLJVwBs9k8atSo7OzsMWPGiErjIq25jkFr5y584Hl+7dq1OTk56enpU6ZMsVqtwrctXmpv2dYMIa2t1uRtNodBQfy2I9ubOmQf9DmpAK9wa7//1pBgOLFNSD6XIMGxRjEYDAYT18Tk0CgGg8FgMKECO0IMBoPBxDXYEWIwGAwmrsGOEIPBYDBxDXaEGAwGg4lrsCPEYDAYTFyDHSEGg8Fg4hrsCDEYDAYT12BHiMFgMJi4BjtCDAaDwcQ12BFiMBgMJq7BjhCDwWAwcQ12hBgMBoOJa7AjxGAwGExcgx0hBoPBYOIa7AgxGAwGE9dgR4jBYDCYuAY7QgwGg8HENdgRYjAYDCauwY4Qg8FgMHENdoQYDAaDiWuwI8RgMBhMXBN5R1hSUuJ2u5tKeJ7nOK7NghzH8Twf5sNYlkUIhfkwDAaDwbQf0h3hkCFDiKtMmzZNEJrN5jFjxqSmpo4dO9ZsNvsRepk0aVJxcXFTicvlcrlcbSrgdDp9PGiLOBwOiqLaPMxms3k8njYPs1qtDMO0eZjFYmFZts3DTCZTIN4Xg8FgMO2HREeIEDp37lxVVZXdbrfb7atXrxbkK1asuOaaa2pra7t3775y5Uo/wiBhWfbEiRMhqQqDwWAw8YxER1hfX+/xeMaNG9elS5fJkyfbbDZBvmXLlhkzZqhUqhkzZmzevNmPMEi+//77seMnhKQqDAaDwcQzcmnF6urqbr755rfeeqt79+6zZs2aOXPmZ599BgDV1dXXXHMNAAivgMLBLQq9uFyuV1999dNPP/VKnE4nQsj/mGFlZSWPwGaztTkCabPZFApFm+OZwtAoTdNtHsbzvFKpbPMwAFAoFG0eJpfLZTKZVqtt82AMBoPBtAcSHeGAAQP27dsnfF6+fPl1110nfEYIEQQhfPAueGlR+KsGcvnAgQObugG5XA5teRGZTCYc2ab/UCgUAR4mEP7DZDIZSUZ+1RIGg8HEJxId4YkTJyiKGjx4MAAolUqVSiXIs7KyKisrc3Nzq6urs7Oz/Qi9KJXKBx98UKvVeiU8zyOEmkqao1KpCAK0Wq3/wwCAYRiFQtHmYTRNazQatVrt/zC3263Vatt8I3S5XIG85Gk0Gq1WKzh1DAaDwUQEiS8iTqdzwoQJxcXFHo/n9ddfHz9+vCAfM2bMhg0bEEIbNmwYN26cHyEGg8FgMNGAREc4dOjQRYsWjRkzJjs722w2r1ixQpAvXLjw9OnT3bp1O3v27IIFC/wIMRgMBoOJBiQOjRIEMX369OnTp/vIk5OTd+zYEYgwSMxmM+Px2Gw2nU4X2poxGAwGE1fE6hqNl199jaao8vLySCuCwWAwmNgmVh2h0+kEhGOyYDAYDCZYYtURYjAYDAYTErAjxGAwGExcgx0hBoPBYOKa2HaEgWRrwmAwGAzGD7HsCJXajRs3RloJDAaDwcQ2wTrCoqKipjv5JOQjlA4Ry14cg8FgMNFBUL7EarVOnTq1aRLdsOUjxGAwGAwmJEh3hAihqVOnzps3r6kwbPkIMRgMBoMJCRJDrAHAihUrcnJyJk6c2FQoIR+h2+1evXr12rVrvRIhH2Fb7SMAYBjGbrf7P87hcCgUijaX1TgcDpZl20xb6HA4AKDN7BMOh4MkyTazTzgcDqVSKZPJNBqNkHwKg8FgMGFG4s13//79O3fu3L17t4+8eerB5cuXKxSK22+/nSAIhJBMJhs4cGDTIo2Njfv37x86dGjTSgBAqKc1NAqZWkF+8803R48e9a9qILUBAM/zBEEEcpj/3IG5ubkfffQRcRX/tQV4GAaDwWDaD4mOcO/evQcPHvS+GBEEcejQoaFDhzZPPVhZWTlt2rRHHnkkZCpHMSUlJfn5+Xq9nqIovV7f5huhy+XS6/U4HyEGg8FEEImOcOnSpUuXLhU+C696wmch9WB+fn7T1INdu3b1eQvsqOBE8xgMBhNzhPjGjVMPYjAYDCa2CMECjaYLW9oj9SAGg8FgMO0HHsrDYDAYTFyDHSEGg8Fg4hrsCDEYDAYT12BHiMFgMJi4JgLRTN5e885/v9oanrZIgFdfmDl27NjwNIfBYDCYmCMCjvBM8fkTqr7Qf3QY2lIeeK+ysjIMDWEwGAwmRolQfMvMXPjdyDC0Q57+OgytYDAYDCZ2wXOEv8JxXJ8+fZpKdu7c2bdv3+Tk5L59+/5/e/caE1e57gH8XTPMDAMzAwMou9MibikwrWxaMPqBHmugSnKodKQaooE0Ia1JNc2u1lh3sO32WA7Emlg5hqSmlOyIzUFrwhma6j6tW7ECzdZdiaUtt9gqFJBrBzr3y3rPh2XnjHNZMw5zdf6/T/adh2e9wwf/mVmL9zl//ny0NgYAAOETfBB6DYlwzeANv9bW1rKystHRUecKy7J1dXXvvffe0tLSm2++2dDQEMXtAQBAmAQZhL5CIi5m8D7//PPHjx/n/nv37t3vvPMOIaS4uPjw4cOuZXa7vbOzs6KiwmAwSCSS9PT0KOwVAADCLMh7hM6Q0Ov1riHR3d2t1Wq5GbwajaalpSV0Ww2Zp59+urm5+eWXX7ZYLFqt9ujRo4SQ8vJytzKxWFxVVaXX6xUKBcMwfX19gTSnlOr1eoPBIBQK/U6f4CJWKBQmJydjHiEAQFQE+T9fXyHBP4M3RlRUVNTX18/Ozn7zzTclJSUqlYqnWCaT6fX61tbW/fv3f/vtt36b018LsPi3vQEAAAidVT0sw4VEU1PTFy2LqgAADIdJREFU/v37uRXPwbwxSCwWb9++vaenp6urq76+3lfZjz/++OqrrxJCUlNTd+/ePTw8HEhzgUAgl8tlMpk8AM4yfBwEAIiWIIPQV0hwg3kJIc7BvLHpmWeeOX369IULF2pqanzVqFSqU6dOffXVV5TSjz76KEFGKgIAJJogg9BXSHCDeSmlroN5Y9ATTzxx+fLliooKhULhq0YsFnd3dx84cCAzM7Orq+vkyZOR3CEAAERG8PcIuZC4efOmWq3u6Ojg1o8cOVJXV5eTk1NaWtrZ2Rm6fYZYcnLyxo0bPb8Xdbtd99hjj12+fDmC+wIAgEgL/taU15AIZDBvRkaG5PNO0cQ/g7504Ow3viVPFbkt2my2oaGhycnJysrKCOwBAABiWRSe0fjPvx7avLEwUlf790ceecRt6ezZsy+88MKJEyfEYnGktgEAADEqCkEokUh4ntWMgJ07d+7cuTOKGwAAgNiBs0YBACChIQgBACChIQgBACChIQgBACChIQgBACChBR+EWq22qKgoPT1969atY2Nj3GL8ziMEAIDEFOSfT0xMTNTX158/f37z5s1tbW0NDQ39/f3k7jzCTz755JVXXjl27FhLS4tUKv3444+vXbuWlJRkt9sJIW4HTFutVpFIxB3VHSCr1eo8AoZhmNj5c8CFhYVobwEAAH4bJrgZQL29vadPn+aO35yfn9+wYQOXAYWFhVqtVq1Wj4yMaDSa0dFRk8l0//3379mzJzs7e3Z2tr29/fXXX3dt1dLSUlJSsm3bNucKl3MSicTX1d9597+m7854SpWn/cehv/Bs1Ww2BzIa0GQyiUQiv1MgjEYjN0HQV8E999xTW1u7uLioUCj8XnR+fj4jI4PbHk9PAAAInyCD0MnhcOzbt08gELS1tRFCZDLZ/Py8VCo1mUzZ2dkrKyu+Fp3UanVubm5XV5dzxWAwUEplMpmvi26tePzqteuEEMIwmfeuGb/yL54drqysiEQiqVTK/0Z0Op1UKuVJX87S0pJMJvP7GfS3BqFMJvNbDAAA4bCqk2U+//zzgwcPVlZWNjU1cSte5xHyDylMSUlpaWlRKpXOFZFIRCmVy+W+risQ/v+2GYHA9We9FAsEIpEoJSXF79uRSqXJycn8NSzLyuVyv0HocDjS0tL8ZpvNZlMqlfgsCAAQRUEGIaW0sbGxv7+/q6uroKDAuc7NI8zPz3edR+h1MVSsJkNoGwIAQEIJ8qnRgYGB7u7unp4elUql1+v1ej237nUeYRiHFFJ2ZX4mlA0BACDBBBmEvb29o6OjSqVSfhe3fuTIkStXruTk5Fy7du3QoUM8i6GxqvubAAAAq35YZvVKS0vb29tLS0udK3q9nv8eYVrW3SduKCV2C/9bWF5eDuQe4e3btwO5R7i4uBjIPcKFhYVA7hHOzc1lZmbiHiEAQBTF5ckyRqMx2lsAAIDfibgMQgAAgFCJvyC0Wq2sg3VdcT6qAwAA8FvFXxBOTEyw1CUIBUl9fX3R2w4AAMS3+AtCd2LpqVMd0d4EAADEq/gPQtbxj6/7o70JAACIV/EfhJRlmVUdFAcAAIlsVUHocDjUarXritd5hBhSCAAAMSv4IGxtbS0rKxsdHXVd5OYRzszM3HfffceOHeNZBAAAiAXBf6lYXFycl5dXXV3tutjd3a3VaiUSyb59+zQaTUtLi69FJ7vd3tfX9+CDDzpXLBYLpdTX6S1Wq9VthVJqsVh87dNisbAs6/f0FovFIhAI/M4Htlqt3PYCKWNZNpAybh6hQBD/X1MDAMSh4IOwvLzcc3Fqaio3N5cQwn0E5Fl0MpvNH374YX19vXPFaDRSSn0Fg8lk8lw0GHzOoDAajdxcJ/63w13Uc0qU54UYhrHZbH7LApkG7BzzG8iMQwAACIcQP2YSxDxCmUx24sSJjIwM54pYLOY5a/TmzZtuKwzDuP64Gy6Q/J41yjBMIGeNchvzG1osywZy1qjdbucG8/KXAQBA+IT46zhu9CAhxHMeodti0D777LNVdgAAAHAKcRBGYB7h3Nyc24r5zrLdbl9lWwAASEwhDsIIzCO80HvxV/+mrMWw7PbwKgAAQIBWe4/Q7SGU9PT0c+fOudV4XQza8vKK+5JU8ZfGQ2e13aG6BAAAJI44e2R/bm5uZtr9uVPCCPz8mQIAAIAPcRaEs7OzXtctFgvPX1AAAAD4EmdB6J3d9o/er1/Y9+do7wMAAOLP7yIIWTsRMH//3/MLCwvR3goAAMSZ30UQEkIomZ+55Xn6GgAAAL84C8J3333X+wsOG0kSX7p0KbLbAQCAuBdnQfi30//t/QVKCCHT09OR3AwAAPwORCIIQzWP0GazUZ55DozgpYONtbW1er0+6EsAAECiiUQQhmQe4eTk5P1/fIB/hARrt575n7P3/XG9NCV1ZGQkuAsBAEBCYfzOJ1q9wsJCrVarVqtHRkY0Go3bcWibNm06ePBgbW0tIYRl2TNnzmRlZSkUCoPBMDQ0JJVKBwYGvu4fmJi8RQkhDEMoQ5xDAyklhBLCEMItUsIICOsgQpFQKExLTsr+wxq90cjabfnr11dWVl69erWmpmb9+vUXL17U6/W5ubnj4+NlZWVarXbbtm3r1q2TSCQTExMrKytFRUXEZW6Gc3qGTqeTyWRCodBt3flrZBiGYZilpaXl5eWVlZXi4mLXJlyZxWK5evXqc889t7S0lJmZKRQKhUIh5hECAERFJIJQJpPNz89LpVKTyZSdnb2y8qsz0vLz8wkh3HMuFovl3x59dOOGDUqlcmpqanR8XCIWT/88a7dYKCO4GzWEkF/yh/xyc/CXdYYwhFBCnYtUyDAOlkokIrudvSdTuXh7+aGSTWq1+tPP/s4wTFpa+s8/z5SWbB64dGnro4+qVCqBQHB9eNhmtW/a9CdngLkym80ikSgpKYl7lfvtMQzDzeB1hpnZbL5x44bJbC4tKSF3I9D5Izqd7vsrQxe/6jUYDEqlUigUKhQKzCMEAIiKEM8j9Ip/HqFcLm9vb8/KyuL+efPGjXPnzjkcjh07dvC3PXPmjFKpfPzxx/nLPvjgg7y8vC1btvCXvf/++w8//HBpaSl/2fHjx6uqqgoLC/nLmpub33rrrZycHP6yw4cPv/TSS5mZmfxlAAAQPpH4RJifn//pp5/m5+ePj49v3759bGzM9dUnn3xSp9O5Ds69desWpdRvivz0009isXjNmjX8ZT/88INcLr/33nv5y8bGxrKysngG/HKuX7++bt06hULBXzY0NPTAAw+kpqbylw0ODm7cuFEikbzxxhtlZWX8xQAAEA6RCMIDBw5IJJLm5ubGxka73f7222+7vjo/Pz84OBjuPcS4zZs3+41qAAAIh0gEoU6nq6ur+/7770tLSzs7O9PS0sJ9RQAAgABFIggBAABiFh7ZBwCAhIYgBACAhIYgBACAhBajQbhlyxbmrr1793oWBHh+KX8fh8OhVqv9tvUs82yr1WqLiorS09O3bt3q/PsQz25ey/y+WQAACJ9YDEJK6cjIyK1bt+7cuXPnzh2vo5cCOb+Uv09ra2tZWZnbeW+ebT3LPNtOTEzU19efPHlyZmZmx44dDQ0NXrt5LQvkzQIAQBjR2DMzMyOTyR566CGZTKbRaGZnZz1rCgoKhoeHKaXDw8MFBQVB9Pniiy/Onj3r9hvwbOtZ5tn2yy+/3LNnD/fq3NxcZmam125eywJ5swAAED6xGISDg4Pl5eWDg4OLi4u7du169tlnPWtSU1ONRiOl1Gg0yuXyoPu4BaGvtq5lPG3tdvvevXtffPFF/m6uZYFsEgAAwidWgtB5eqfb+vT0tFKp9KxPSUkxmUyUUoPBkJKS4re/rz5uV/TV1tdHZ9e2Fy5cKCkpee2112w2G083zzK/mwQAgPCJxKHbgXAdH/jdd9+ZzWbu7E2xWCyRSDzrVSrV5ORkfn7+1NTU2rVrvfYMpE9I2lJKGxsb+/v7u7q6CgoKfHXzWhbEJgEAIIRi8WEZg8FQU1MzPDxstVqPHj361FNPedZUV1d3dHRQSjs6OjQaTdB9QtJ2YGCgu7u7p6dHpVLp9Xq9Xu+1m9eyIDYJAAChFNXPo96xLNvW1paXl5eVlbVr167l5WXPmtu3b1dVVa1du7a6ulqn0wXdx+034Kuta5ln26amJq+/VbduXssC2SQAAIQPzhoFAICEFotfjQIAAEQMghAAABIaghAAABIaghAAABIaghAAABIaghAAABIaghAAABIaghAAABIaghAAABIaghAAABIaghAAABLa/wG2jEm26H1ojAAAAABJRU5ErkJggg==" }, - "execution_count": 44, + "execution_count": 22, "metadata": {}, "output_type": "execute_result" } @@ -766,71 +650,54 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Pre-processing data for heritability analysis" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "To prepare variance component model fitting, we form an instance of `VarianceComponentVariate`. The two variance components are $(2\\Phi, I)$." ] }, { "cell_type": "code", - "execution_count": 45, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 24, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "3-element Array{Symbol,1}:\n", - " :Y\n", - " :X\n", - " :V" + "(:Y, :X, :V)" ] }, - "execution_count": 45, + "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "using VarianceComponentModels\n", + "using VarianceComponentModels, LinearAlgebra\n", "\n", "# form data as VarianceComponentVariate\n", - "cg10kdata = VarianceComponentVariate(Y, (2Φgrm, eye(size(Y, 1))))\n", - "fieldnames(cg10kdata)" + "cg10kdata = VarianceComponentVariate(Y, (2Φgrm, Matrix(1.0I, size(Y, 1), size(Y, 1))))\n", + "fieldnames(typeof(cg10kdata))" ] }, { "cell_type": "code", - "execution_count": 46, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 25, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}}([-1.81573 -0.94615 … -1.02853 -0.394049; -1.2444 0.10966 … 1.09065 0.0256616; … ; 0.886626 0.487408 … -0.636874 -0.439825; -1.24394 0.213697 … 0.299931 0.392809], Array{Float64}(6670,0), ([1.00605 0.00671009 … -0.000109037 -0.00556144; 0.00671009 0.997916 … 0.00173694 0.00685701; … ; -0.000109037 0.00173694 … 1.00166 0.000938955; -0.00556144 0.00685701 … 0.000938955 1.00125], [1.0 0.0 … 0.0 0.0; 0.0 1.0 … 0.0 0.0; … ; 0.0 0.0 … 1.0 0.0; 0.0 0.0 … 0.0 1.0]))" + "VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}}([-1.81573 -0.94615 … -1.02853 -0.394049; -1.2444 0.10966 … 1.09065 0.0256616; … ; 0.886626 0.487408 … -0.636874 -0.439825; -1.24394 0.213697 … 0.299931 0.392809], Array{Float64}(6670,0), ([1.00547 0.00656224 … -0.000121918 -0.00555374; 0.00656224 0.99614 … 0.00176999 0.00683183; … ; -0.000121918 0.00176999 … 0.998578 0.000962983; -0.00555374 0.00683183 … 0.000962983 0.999599], [1.0 0.0 … 0.0 0.0; 0.0 1.0 … 0.0 0.0; … ; 0.0 0.0 … 1.0 0.0; 0.0 0.0 … 0.0 1.0]))" ] }, - "execution_count": 46, + "execution_count": 25, "metadata": {}, "output_type": "execute_result" } @@ -841,42 +708,30 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "Before fitting the variance component model, we pre-compute the eigen-decomposition of $2\\Phi_{\\text{GRM}}$, the rotated responses, and the constant part in log-likelihood, and store them as a `TwoVarCompVariateRotate` instance, which is re-used in various variane component estimation procedures." ] }, { "cell_type": "code", - "execution_count": 47, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 26, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - " 48.837361 seconds (39 allocations: 1021.427 MiB, 0.57% gc time)\n" + " 49.812646 seconds (1.74 M allocations: 1.080 GiB, 0.33% gc time)\n" ] }, { "data": { "text/plain": [ - "5-element Array{Symbol,1}:\n", - " :Yrot \n", - " :Xrot \n", - " :eigval \n", - " :eigvec \n", - " :logdetV2" + "(:Yrot, :Xrot, :eigval, :eigvec, :logdetV2)" ] }, - "execution_count": 47, + "execution_count": 26, "metadata": {}, "output_type": "execute_result" } @@ -884,15 +739,12 @@ "source": [ "# pre-compute eigen-decomposition (~50 secs on my laptop)\n", "@time cg10kdata_rotated = TwoVarCompVariateRotate(cg10kdata)\n", - "fieldnames(cg10kdata_rotated)" + "fieldnames(typeof(cg10kdata_rotated))" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Save intermediate results\n", "\n", @@ -900,198 +752,252 @@ ] }, { - "cell_type": "code", - "execution_count": 48, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true, - "scrolled": true - }, - "outputs": [], + "cell_type": "markdown", + "metadata": {}, "source": [ - "# # Pkg.add(\"JLD\")\n", - "# using JLD\n", - "# @save \"cg10k.jld\"\n", - "# whos()" + "To load workspace" ] }, { - "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "data": { + "text/latex": [ + "\\begin{tabular}\n", + "{l | r | l}\n", + "name & size & summary \\\\\n", + "\\hline\n", + "Base & & Module \\\\\n", + "Core & & Module \\\\\n", + "Main & & Module \\\\\n", + "Plots & 2.988 MiB & Module \\\\\n", + "PyPlot & 782.799 KiB & Module \\\\\n", + "Y & 677.461 KiB & 6670×13 Array\\{Float64,2\\} \\\\\n", + "cg10k & 1022.983 MiB & 6670×630860 SnpArray \\\\\n", + "cg10k\\_trait & 1.605 MiB & 6670×15 DataFrame \\\\\n", + "cg10kdata & 679.508 MiB & VarianceComponentVariate\\{Float64,2,Array\\{Float64,2\\},Array\\{Float64,2\\},Array\\{Float64,2\\}\\} \\\\\n", + "cg10kdata\\_rotated & 340.136 MiB & TwoVarCompVariateRotate\\{Float64,Array\\{Float64,2\\},Array\\{Float64,2\\}\\} \\\\\n", + "maf\\_cg10k & 4.813 MiB & 630860-element Array\\{Float64,1\\} \\\\\n", + "missings\\_by\\_snp & 4.813 MiB & 1×630860 Array\\{Int64,2\\} \\\\\n", + "mp & 28.748 MiB & 6670×630860 SparseArrays.SparseMatrixCSC\\{Bool,Int32\\} \\\\\n", + "people & 8 bytes & Int64 \\\\\n", + "snps & 8 bytes & Int64 \\\\\n", + "startupfile & 57 bytes & String \\\\\n", + "Φgrm & 339.423 MiB & 6670×6670 Array\\{Float64,2\\} \\\\\n", + "\\end{tabular}\n" + ], + "text/markdown": [ + "| name | size | summary |\n", + "|:----------------- | ------------:|:-------------------------------------------------------------------------------------- |\n", + "| Base | | Module |\n", + "| Core | | Module |\n", + "| Main | | Module |\n", + "| Plots | 2.988 MiB | Module |\n", + "| PyPlot | 782.799 KiB | Module |\n", + "| Y | 677.461 KiB | 6670×13 Array{Float64,2} |\n", + "| cg10k | 1022.983 MiB | 6670×630860 SnpArray |\n", + "| cg10k_trait | 1.605 MiB | 6670×15 DataFrame |\n", + "| cg10kdata | 679.508 MiB | VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}} |\n", + "| cg10kdata_rotated | 340.136 MiB | TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}} |\n", + "| maf_cg10k | 4.813 MiB | 630860-element Array{Float64,1} |\n", + "| missings_by_snp | 4.813 MiB | 1×630860 Array{Int64,2} |\n", + "| mp | 28.748 MiB | 6670×630860 SparseArrays.SparseMatrixCSC{Bool,Int32} |\n", + "| people | 8 bytes | Int64 |\n", + "| snps | 8 bytes | Int64 |\n", + "| startupfile | 57 bytes | String |\n", + "| Φgrm | 339.423 MiB | 6670×6670 Array{Float64,2} |\n" + ], + "text/plain": [ + "name size summary \n", + "––––––––––––––––– –––––––––––– ––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––\n", + "Base Module \n", + "Core Module \n", + "Main Module \n", + "Plots 2.988 MiB Module \n", + "PyPlot 782.799 KiB Module \n", + "Y 677.461 KiB 6670×13 Array{Float64,2} \n", + "cg10k 1022.983 MiB 6670×630860 SnpArray \n", + "cg10k_trait 1.605 MiB 6670×15 DataFrame \n", + "cg10kdata 679.508 MiB VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}}\n", + "cg10kdata_rotated 340.136 MiB TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}} \n", + "maf_cg10k 4.813 MiB 630860-element Array{Float64,1} \n", + "missings_by_snp 4.813 MiB 1×630860 Array{Int64,2} \n", + "mp 28.748 MiB 6670×630860 SparseArrays.SparseMatrixCSC{Bool,Int32} \n", + "people 8 bytes Int64 \n", + "snps 8 bytes Int64 \n", + "startupfile 57 bytes String \n", + "Φgrm 339.423 MiB 6670×6670 Array{Float64,2} " + ] + }, + "execution_count": 40, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "To load workspace" + "#pkg\"add FileIO JLD2\"\n", + "\n", + "using JLD2\n", + "@save \"cg10k.jld2\"\n", + "varinfo()" ] }, { "cell_type": "code", - "execution_count": 49, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true, - "scrolled": true - }, + "execution_count": 45, + "metadata": {}, "outputs": [ { - "name": "stdout", + "name": "stderr", "output_type": "stream", "text": [ - " Base Module\n", - " BinDeps 41348 KB Module\n", - " Blosc 41202 KB Module\n", - " ColorTypes 41457 KB Module\n", - " Colors 41480 KB Module\n", - " Compat 41196 KB Module\n", - " Conda 41205 KB Module\n", - " Core Module\n", - " DataArrays 41456 KB Module\n", - " DataFrames 41684 KB Module\n", - " DataStructures 41356 KB Module\n", - " FileIO 41310 KB Module\n", - " FixedPointNumbers 41695 KB Module\n", - " GZip 41181 KB Module\n", - " HDF5 41403 KB Module\n", - " IJulia 4185781 KB Module\n", - " Ipopt 41172 KB Module\n", - " JLD 41376 KB Module\n", - " JSON 41245 KB Module\n", - " LaTeXStrings 4058 bytes Module\n", - " LegacyStrings 41212 KB Module\n", - " LinearMaps 22 KB Module\n", - " MacroTools 41606 KB Module\n", - " Main Module\n", - " MathProgBase 41353 KB Module\n", - " MbedTLS 41269 KB Module\n", - " Measures 41175 KB Module\n", - " NaNMath 41200 KB Module\n", - " PlotThemes 41167 KB Module\n", - " PlotUtils 41332 KB Module\n", - " Plots 42960 KB Module\n", - " PyCall 41711 KB Module\n", - " PyPlot 41771 KB Module\n", - " RecipesBase 41283 KB Module\n", - " Reexport 41160 KB Module\n", - " Requires 41172 KB Module\n", - " SHA 62 KB Module\n", - " Showoff 41163 KB Module\n", - " SnpArrays 41218 KB Module\n", - " SortingAlgorithms 41178 KB Module\n", - " SpecialFunctions 41252 KB Module\n", - " StaticArrays 41744 KB Module\n", - " StatsBase 41810 KB Module\n", - " URIParser 41171 KB Module\n", - " VarianceComponentModels 41278 KB Module\n", - " Y 677 KB 6670×13 Array{Float64,2}\n", - " ZMQ 41223 KB Module\n", - " _ 77 KB 630860-element BitArray{1}\n", - " cg10k 1027303 KB 6670×630860 SnpArrays.SnpArray{2}\n", - " cg10k_trait 978 KB 6670×15 DataFrames.DataFrame\n", - " cg10kdata 695816 KB VarianceComponentModels.VarianceCo…\n", - " cg10kdata_rotated 348299 KB VarianceComponentModels.TwoVarComp…\n", - " h 24 bytes 3-element Array{Float64,1}\n", - " hST 104 bytes 13-element Array{Float64,1}\n", - " hST_se 104 bytes 13-element Array{Float64,1}\n", - " hse 24 bytes 3-element Array{Float64,1}\n", - " maf 4928 KB 630860-element Array{Float64,1}\n", - " missings_by_snp 4928 KB 630860-element Array{Int64,1}\n", - " people 8 bytes Int64\n", - " snps 8 bytes Int64\n", - " trait57_data 347778 KB VarianceComponentModels.TwoVarComp…\n", - " trait57_model 232 bytes VarianceComponentModels.VarianceCo…\n", - " traitall_model 2792 bytes VarianceComponentModels.VarianceCo…\n", - " traitidx 16 bytes 3-element UnitRange{Int64}\n", - " Σa 3848 bytes 13×13 Array{Array{Float64,2},2}\n", - " Σcov 2592 bytes 18×18 Array{Float64,2}\n", - " Σe 3848 bytes 13×13 Array{Array{Float64,2},2}\n", - " Φgrm 347569 KB 6670×6670 Array{Float64,2}\n", - " σ2a 104 bytes 13-element Array{Float64,1}\n", - " σ2e 104 bytes 13-element Array{Float64,1}\n" + "┌ Warning: type SparseArrays.SparseMatrixCSC{Bool,Int32} does not exist in workspace; reconstructing\n", + "└ @ JLD2 /Users/juhyun-kim/.julia/packages/JLD2/KjBIK/src/data.jl:1153\n" ] + }, + { + "data": { + "text/latex": [ + "\\begin{tabular}\n", + "{l | r | l}\n", + "name & size & summary \\\\\n", + "\\hline\n", + "Base & & Module \\\\\n", + "Core & & Module \\\\\n", + "Main & & Module \\\\\n", + "Plots & 2.988 MiB & Module \\\\\n", + "PyPlot & 782.799 KiB & Module \\\\\n", + "Y & 677.461 KiB & 6670×13 Array\\{Float64,2\\} \\\\\n", + "cg10k & 1022.983 MiB & 6670×630860 SnpArray \\\\\n", + "cg10k\\_trait & 1.605 MiB & 6670×15 DataFrame \\\\\n", + "cg10kdata & 679.508 MiB & VarianceComponentVariate\\{Float64,2,Array\\{Float64,2\\},Array\\{Float64,2\\},Array\\{Float64,2\\}\\} \\\\\n", + "cg10kdata\\_rotated & 340.136 MiB & TwoVarCompVariateRotate\\{Float64,Array\\{Float64,2\\},Array\\{Float64,2\\}\\} \\\\\n", + "maf\\_cg10k & 4.813 MiB & 630860-element Array\\{Float64,1\\} \\\\\n", + "missings\\_by\\_snp & 4.813 MiB & 1×630860 Array\\{Int64,2\\} \\\\\n", + "mp & 28.748 MiB & getfield(JLD2.ReconstructedTypes, Symbol(\"\\#\\#SparseArrays.SparseMatrixCSC\\{Bool,Int32\\}\\#386\")) \\\\\n", + "people & 8 bytes & Int64 \\\\\n", + "snps & 8 bytes & Int64 \\\\\n", + "startupfile & 57 bytes & String \\\\\n", + "Φgrm & 339.423 MiB & 6670×6670 Array\\{Float64,2\\} \\\\\n", + "\\end{tabular}\n" + ], + "text/markdown": [ + "| name | size | summary |\n", + "|:----------------- | ------------:|:------------------------------------------------------------------------------------------- |\n", + "| Base | | Module |\n", + "| Core | | Module |\n", + "| Main | | Module |\n", + "| Plots | 2.988 MiB | Module |\n", + "| PyPlot | 782.799 KiB | Module |\n", + "| Y | 677.461 KiB | 6670×13 Array{Float64,2} |\n", + "| cg10k | 1022.983 MiB | 6670×630860 SnpArray |\n", + "| cg10k_trait | 1.605 MiB | 6670×15 DataFrame |\n", + "| cg10kdata | 679.508 MiB | VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}} |\n", + "| cg10kdata_rotated | 340.136 MiB | TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}} |\n", + "| maf_cg10k | 4.813 MiB | 630860-element Array{Float64,1} |\n", + "| missings_by_snp | 4.813 MiB | 1×630860 Array{Int64,2} |\n", + "| mp | 28.748 MiB | getfield(JLD2.ReconstructedTypes, Symbol(\"##SparseArrays.SparseMatrixCSC{Bool,Int32}#386\")) |\n", + "| people | 8 bytes | Int64 |\n", + "| snps | 8 bytes | Int64 |\n", + "| startupfile | 57 bytes | String |\n", + "| Φgrm | 339.423 MiB | 6670×6670 Array{Float64,2} |\n" + ], + "text/plain": [ + "name size summary \n", + "––––––––––––––––– –––––––––––– –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––\n", + "Base Module \n", + "Core Module \n", + "Main Module \n", + "Plots 2.988 MiB Module \n", + "PyPlot 782.799 KiB Module \n", + "Y 677.461 KiB 6670×13 Array{Float64,2} \n", + "cg10k 1022.983 MiB 6670×630860 SnpArray \n", + "cg10k_trait 1.605 MiB 6670×15 DataFrame \n", + "cg10kdata 679.508 MiB VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}} \n", + "cg10kdata_rotated 340.136 MiB TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}} \n", + "maf_cg10k 4.813 MiB 630860-element Array{Float64,1} \n", + "missings_by_snp 4.813 MiB 1×630860 Array{Int64,2} \n", + "mp 28.748 MiB getfield(JLD2.ReconstructedTypes, Symbol(\"##SparseArrays.SparseMatrixCSC{Bool,Int32}#386\"))\n", + "people 8 bytes Int64 \n", + "snps 8 bytes Int64 \n", + "startupfile 57 bytes String \n", + "Φgrm 339.423 MiB 6670×6670 Array{Float64,2} " + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ - "using SnpArrays, JLD, DataFrames, VarianceComponentModels, Plots\n", + "using SnpArrays, JLD2, DataFrames, VarianceComponentModels, Plots\n", "pyplot()\n", - "@load \"cg10k.jld\"\n", - "whos()" + "\n", + "@load \"cg10k.jld2\"\n", + "varinfo()" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Heritability of single traits" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "We use Fisher scoring algorithm to fit variance component model for each single trait." ] }, { "cell_type": "code", - "execution_count": 50, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true, - "scrolled": true - }, + "execution_count": 46, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Trait1\n", - "(σ2a[trait], σ2e[trait]) = (0.25978160614793233, 0.7369535197912689)\n", + "\n", + "******************************************************************************\n", + "This program contains Ipopt, a library for large-scale nonlinear optimization.\n", + " Ipopt is released as open source code under the Eclipse Public License (EPL).\n", + " For more information visit http://projects.coin-or.org/Ipopt\n", + "******************************************************************************\n", + "\n", + "(σ2a[trait], σ2e[trait]) = (0.2594760477653529, 0.7375938878381831)\n", "Trait2\n", - "(σ2a[trait], σ2e[trait]) = (0.18647130348299173, 0.8129591079735827)\n", + "(σ2a[trait], σ2e[trait]) = (0.18588150448760332, 0.8137880281704619)\n", "Trait3\n", - "(σ2a[trait], σ2e[trait]) = (0.3188368159422607, 0.6798809726936244)\n", + "(σ2a[trait], σ2e[trait]) = (0.31960937565352526, 0.6795300238388766)\n", "Trait4\n", - "(σ2a[trait], σ2e[trait]) = (0.2651357653143703, 0.7308007669086968)\n", + "(σ2a[trait], σ2e[trait]) = (0.26578244697498915, 0.7304994606945259)\n", "Trait5\n", - "(σ2a[trait], σ2e[trait]) = (0.28083388108246, 0.7172036435586534)\n", + "(σ2a[trait], σ2e[trait]) = (0.28143198005767217, 0.7169747061764987)\n", "Trait6\n", - "(σ2a[trait], σ2e[trait]) = (0.2824159905728832, 0.7170988773569172)\n", + "(σ2a[trait], σ2e[trait]) = (0.2830055131241748, 0.7168800753377107)\n", "Trait7\n", - "(σ2a[trait], σ2e[trait]) = (0.2155274336968625, 0.7815346282986375)\n", + "(σ2a[trait], σ2e[trait]) = (0.2156542141311619, 0.7816910320891676)\n", "Trait8\n", - "(σ2a[trait], σ2e[trait]) = (0.194687807263945, 0.8049690651320599)\n", + "(σ2a[trait], σ2e[trait]) = (0.19408878271207824, 0.8058201577783562)\n", "Trait9\n", - "(σ2a[trait], σ2e[trait]) = (0.24706855916591713, 0.7512942998567308)\n", + "(σ2a[trait], σ2e[trait]) = (0.24746236011763145, 0.7512222977091793)\n", "Trait10\n", - "(σ2a[trait], σ2e[trait]) = (0.098712236297271, 0.9011756660217387)\n", + "(σ2a[trait], σ2e[trait]) = (0.0992417256213392, 0.9007769787053657)\n", "Trait11\n", - "(σ2a[trait], σ2e[trait]) = (0.1664264642608195, 0.8322427413046204)\n", + "(σ2a[trait], σ2e[trait]) = (0.1645726648140337, 0.8343110221526228)\n", "Trait12\n", - "(σ2a[trait], σ2e[trait]) = (0.0834296761650666, 0.9153609794266608)\n", + "(σ2a[trait], σ2e[trait]) = (0.0822495659186408, 0.9166483378364408)\n", "Trait13\n", - "(σ2a[trait], σ2e[trait]) = (0.05893968504298988, 0.940270012443928)\n", - "elapsed time: 0.160999612 seconds\n" + "(σ2a[trait], σ2e[trait]) = (0.05687679106195183, 0.9424058676223598)\n", + " 6.203222 seconds (12.80 M allocations: 621.516 MiB, 7.13% gc time)\n" ] - }, - { - "data": { - "text/plain": [ - "0.160999612" - ] - }, - "execution_count": 50, - "metadata": {}, - "output_type": "execute_result" } ], "source": [ @@ -1104,8 +1010,7 @@ "# enviromental effects\n", "σ2e = zeros(13)\n", "\n", - "tic()\n", - "for trait in 1:13\n", + "@time for trait in 1:13\n", " println(names(cg10k_trait)[trait + 2])\n", " # form data set for trait j\n", " traitj_data = TwoVarCompVariateRotate(cg10kdata_rotated.Yrot[:, trait], cg10kdata_rotated.Xrot, \n", @@ -1120,28 +1025,23 @@ " h, hse = heritability(traitj_model.Σ, Σcov)\n", " hST[trait] = h[1]\n", " hST_se[trait] = hse[1]\n", - "end\n", - "toc()" + "end" ] }, { "cell_type": "code", - "execution_count": 51, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 47, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2×13 Array{Float64,2}:\n", - " 0.260633 0.186578 0.319246 … 0.166648 0.0835307 0.0589863\n", - " 0.0799732 0.0869002 0.0741007 0.08862 0.0944407 0.0953238" + " 0.260239 0.185943 0.319885 … 0.164757 0.0823403 0.0569176\n", + " 0.0799434 0.08689 0.0739664 0.0887138 0.0944375 0.0953072" ] }, - "execution_count": 51, + "execution_count": 47, "metadata": {}, "output_type": "execute_result" } @@ -1153,10 +1053,7 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Pairwise traits\n", "\n", @@ -1165,195 +1062,187 @@ }, { "cell_type": "code", - "execution_count": 52, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 49, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Trait1Trait2\n", - "(Σa[i, j], Σe[i, j]) = ([0.258822 0.174358; 0.174358 0.185108], [0.737892 0.585751; 0.585751 0.814301])\n", + "(Σa[i, j], Σe[i, j]) = ([0.258529 0.173943; 0.173943 0.184525], [0.738519 0.58639; 0.58639 0.815122])\n", "Trait1Trait3\n", - "(Σa[i, j], Σe[i, j]) = ([0.260236 -0.0144726; -0.0144726 0.319245], [0.736512 -0.11979; -0.11979 0.679488])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259934 -0.0139395; -0.0139395 0.320025], [0.737149 -0.120342; -0.120342 0.67913])\n", "Trait1Trait4\n", - "(Σa[i, j], Σe[i, j]) = ([0.259615 0.222203; 0.222203 0.265149], [0.737116 0.599854; 0.599854 0.730788])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259307 0.222267; 0.222267 0.265791], [0.737759 0.600077; 0.600077 0.730492])\n", "Trait1Trait5\n", - "(Σa[i, j], Σe[i, j]) = ([0.259574 -0.146827; -0.146827 0.28153], [0.737153 -0.254777; -0.254777 0.71653])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259264 -0.147125; -0.147125 0.282102], [0.737798 -0.254669; -0.254669 0.716327])\n", "Trait1Trait6\n", - "(Σa[i, j], Σe[i, j]) = ([0.259476 -0.129115; -0.129115 0.282688], [0.73725 -0.23161; -0.23161 0.716837])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259171 -0.129519; -0.129519 0.283267], [0.73789 -0.231373; -0.231373 0.716628])\n", "Trait1Trait7\n", - "(Σa[i, j], Σe[i, j]) = ([0.259115 -0.140455; -0.140455 0.215297], [0.737606 -0.197616; -0.197616 0.781774])\n", + "(Σa[i, j], Σe[i, j]) = ([0.258784 -0.140357; -0.140357 0.21535], [0.738269 -0.197894; -0.197894 0.782002])\n", "Trait1Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.259778 -0.0327756; -0.0327756 0.194698], [0.736957 -0.127026; -0.127026 0.804959])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259467 -0.0336174; -0.0336174 0.194107], [0.737603 -0.12623; -0.12623 0.805802])\n", "Trait1Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.261858 -0.204589; -0.204589 0.246027], [0.734961 -0.307734; -0.307734 0.75232])\n", + "(Σa[i, j], Σe[i, j]) = ([0.261524 -0.204467; -0.204467 0.246378], [0.735632 -0.308118; -0.308118 0.752288])\n", "Trait1Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.259649 -0.0994858; -0.0994858 0.0956585], [0.737083 -0.303942; -0.303942 0.904218])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259326 -0.0987008; -0.0987008 0.0962217], [0.73774 -0.304854; -0.304854 0.90378])\n", "Trait1Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.25947 -0.138603; -0.138603 0.164709], [0.737257 -0.359557; -0.359557 0.83395])\n", + "(Σa[i, j], Σe[i, j]) = ([0.259194 -0.137578; -0.137578 0.162813], [0.737868 -0.36076; -0.36076 0.836058])\n", "Trait1Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.261779 -0.145414; -0.145414 0.0807748], [0.735076 -0.041823; -0.041823 0.9181])\n", + "(Σa[i, j], Σe[i, j]) = ([0.26168 -0.145305; -0.145305 0.0806851], [0.735514 -0.0421192; -0.0421192 0.918294])\n", "Trait1Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.261125 -0.108774; -0.108774 0.0538214], [0.735674 -0.114123; -0.114123 0.945416])\n", + "(Σa[i, j], Σe[i, j]) = ([0.260776 -0.108277; -0.108277 0.0513909], [0.736359 -0.114756; -0.114756 0.947913])\n", "Trait2Trait3\n", - "(Σa[i, j], Σe[i, j]) = ([0.186541 0.144056; 0.144056 0.320627], [0.812888 0.0995944; 0.0995944 0.678167])\n", + "(Σa[i, j], Σe[i, j]) = ([0.185992 0.14505; 0.14505 0.321413], [0.813678 0.0987898; 0.0987898 0.677805])\n", "Trait2Trait4\n", - "(Σa[i, j], Σe[i, j]) = ([0.186131 0.0746032; 0.0746032 0.265122], [0.813293 0.221109; 0.221109 0.730814])\n", + "(Σa[i, j], Σe[i, j]) = ([0.18554 0.074367; 0.074367 0.26577], [0.814123 0.22144; 0.22144 0.730512])\n", "Trait2Trait5\n", - "(Σa[i, j], Σe[i, j]) = ([0.186442 -0.0118093; -0.0118093 0.280842], [0.812987 -0.0365191; -0.0365191 0.717195])\n", + "(Σa[i, j], Σe[i, j]) = ([0.185854 -0.0115021; -0.0115021 0.281443], [0.813815 -0.0368393; -0.0368393 0.716964])\n", "Trait2Trait6\n", - "(Σa[i, j], Σe[i, j]) = ([0.18649 -0.00366533; -0.00366533 0.282471], [0.812941 -0.0206271; -0.0206271 0.717046])\n", + "(Σa[i, j], Σe[i, j]) = ([0.185903 -0.00350261; -0.00350261 0.283064], [0.813767 -0.0207933; -0.0207933 0.716823])\n", "Trait2Trait7\n", - "(Σa[i, j], Σe[i, j]) = ([0.186104 -0.030665; -0.030665 0.215304], [0.81332 -0.000667009; -0.000667009 0.781755])\n", + "(Σa[i, j], Σe[i, j]) = ([0.185498 -0.0301226; -0.0301226 0.215413], [0.814164 -0.00124563; -0.00124563 0.781929])\n", "Trait2Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.187023 0.0331783; 0.0331783 0.195259], [0.812421 -0.0326343; -0.0326343 0.804415])\n", + "(Σa[i, j], Σe[i, j]) = ([0.186397 0.032688; 0.032688 0.194625], [0.813285 -0.0321045; -0.0321045 0.805301])\n", "Trait2Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.185032 -0.085334; -0.085334 0.245909], [0.814386 -0.0809638; -0.0809638 0.752433])\n", + "(Σa[i, j], Σe[i, j]) = ([0.184426 -0.084907; -0.084907 0.246288], [0.815228 -0.0814978; -0.0814978 0.752373])\n", "Trait2Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.186587 -0.123303; -0.123303 0.0987387], [0.812872 -0.273083; -0.273083 0.901229])\n", + "(Σa[i, j], Σe[i, j]) = ([0.18604 -0.123261; -0.123261 0.0992567], [0.813659 -0.273285; -0.273285 0.90084])\n", "Trait2Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.185484 -0.117256; -0.117256 0.167776], [0.81393 -0.296772; -0.296772 0.830934])\n", + "(Σa[i, j], Σe[i, j]) = ([0.184894 -0.116422; -0.116422 0.165892], [0.814757 -0.297755; -0.297755 0.833033])\n", "Trait2Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.185907 -0.0909104; -0.0909104 0.0827171], [0.813555 0.0457924; 0.0457924 0.916135])\n", + "(Σa[i, j], Σe[i, j]) = ([0.185297 -0.0908872; -0.0908872 0.0814927], [0.814404 0.0456529; 0.0456529 0.917465])\n", "Trait2Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.185979 -0.0720811; -0.0720811 0.0568238], [0.8135 0.0751703; 0.0751703 0.942424])\n", + "(Σa[i, j], Σe[i, j]) = ([0.185348 -0.07104; -0.07104 0.0546791], [0.814367 0.0740412; 0.0740412 0.944638])\n", "Trait3Trait4\n", - "(Σa[i, j], Σe[i, j]) = ([0.3188 -0.154562; -0.154562 0.264323], [0.679917 -0.303223; -0.303223 0.731591])\n", + "(Σa[i, j], Σe[i, j]) = ([0.319575 -0.15468; -0.15468 0.265006], [0.679563 -0.303309; -0.303309 0.731254])\n", "Trait3Trait5\n", - "(Σa[i, j], Σe[i, j]) = ([0.319216 0.183527; 0.183527 0.282063], [0.679514 0.33724; 0.33724 0.716008])\n", + "(Σa[i, j], Σe[i, j]) = ([0.320016 0.184736; 0.184736 0.282719], [0.679136 0.336276; 0.336276 0.715725])\n", "Trait3Trait6\n", - "(Σa[i, j], Σe[i, j]) = ([0.319776 0.165672; 0.165672 0.284448], [0.678972 0.298667; 0.298667 0.715124])\n", + "(Σa[i, j], Σe[i, j]) = ([0.320608 0.166929; 0.166929 0.285124], [0.678565 0.297633; 0.297633 0.714823])\n", "Trait3Trait7\n", - "(Σa[i, j], Σe[i, j]) = ([0.318838 0.166283; 0.166283 0.215261], [0.67988 0.347706; 0.347706 0.781796])\n", + "(Σa[i, j], Σe[i, j]) = ([0.319614 0.167317; 0.167317 0.215458], [0.679526 0.346897; 0.346897 0.781883])\n", "Trait3Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.320718 0.0566397; 0.0566397 0.197764], [0.678063 0.0451569; 0.0451569 0.801956])\n", + "(Σa[i, j], Σe[i, j]) = ([0.321532 0.0573397; 0.0573397 0.197221], [0.677674 0.0445311; 0.0445311 0.802757])\n", "Trait3Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.319001 0.137699; 0.137699 0.246142], [0.679722 0.266704; 0.266704 0.752197])\n", + "(Σa[i, j], Σe[i, j]) = ([0.319777 0.138268; 0.138268 0.246547], [0.679369 0.266317; 0.266317 0.752112])\n", "Trait3Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.31908 -0.076513; -0.076513 0.0996001], [0.679646 -0.142905; -0.142905 0.900298])\n", + "(Σa[i, j], Σe[i, j]) = ([0.319905 -0.0779601; -0.0779601 0.100192], [0.679245 -0.141562; -0.141562 0.899839])\n", "Trait3Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.318094 -0.0177494; -0.0177494 0.16629], [0.6806 -0.1144; -0.1144 0.832376])\n", + "(Σa[i, j], Σe[i, j]) = ([0.318882 -0.0177352; -0.0177352 0.164446], [0.680233 -0.114434; -0.114434 0.834435])\n", "Trait3Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.321164 0.0843842; 0.0843842 0.0874609], [0.677639 0.0341558; 0.0341558 0.911368])\n", + "(Σa[i, j], Σe[i, j]) = ([0.321958 0.0844644; 0.0844644 0.0863388], [0.67727 0.034186; 0.034186 0.912603])\n", "Trait3Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.323273 0.109443; 0.109443 0.0634295], [0.675635 -0.0060525; -0.0060525 0.935819])\n", + "(Σa[i, j], Σe[i, j]) = ([0.32405 0.109731; 0.109731 0.0613165], [0.675287 -0.00619989; -0.00619989 0.938011])\n", "Trait4Trait5\n", - "(Σa[i, j], Σe[i, j]) = ([0.26525 -0.215125; -0.215125 0.282572], [0.73068 -0.377406; -0.377406 0.715518])\n", + "(Σa[i, j], Σe[i, j]) = ([0.26593 -0.216116; -0.216116 0.283173], [0.730347 -0.376697; -0.376697 0.715289])\n", "Trait4Trait6\n", - "(Σa[i, j], Σe[i, j]) = ([0.265715 -0.199714; -0.199714 0.283942], [0.730231 -0.347732; -0.347732 0.715619])\n", + "(Σa[i, j], Σe[i, j]) = ([0.266395 -0.200793; -0.200793 0.284547], [0.729899 -0.346915; -0.346915 0.715387])\n", "Trait4Trait7\n", - "(Σa[i, j], Σe[i, j]) = ([0.26407 -0.18238; -0.18238 0.214324], [0.731843 -0.32655; -0.32655 0.782733])\n", + "(Σa[i, j], Σe[i, j]) = ([0.264731 -0.182912; -0.182912 0.21441], [0.731526 -0.326257; -0.326257 0.782928])\n", "Trait4Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.266229 -0.0965381; -0.0965381 0.196655], [0.729739 -0.151461; -0.151461 0.803044])\n", + "(Σa[i, j], Σe[i, j]) = ([0.266914 -0.0976281; -0.0976281 0.196119], [0.729401 -0.150498; -0.150498 0.803836])\n", "Trait4Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.269627 -0.226931; -0.226931 0.247265], [0.726443 -0.416085; -0.416085 0.751086])\n", + "(Σa[i, j], Σe[i, j]) = ([0.270267 -0.227463; -0.227463 0.247665], [0.726155 -0.415849; -0.415849 0.751008])\n", "Trait4Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.265098 -0.0352926; -0.0352926 0.0981462], [0.730847 -0.226248; -0.226248 0.901736])\n", + "(Σa[i, j], Σe[i, j]) = ([0.265756 -0.0339156; -0.0339156 0.0987618], [0.730536 -0.227666; -0.227666 0.901251])\n", "Trait4Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.265178 -0.0970634; -0.0970634 0.164885], [0.73076 -0.272291; -0.272291 0.833762])\n", + "(Σa[i, j], Σe[i, j]) = ([0.265858 -0.0963336; -0.0963336 0.163013], [0.730427 -0.273145; -0.273145 0.835846])\n", "Trait4Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.267732 -0.140985; -0.140985 0.081029], [0.728323 -0.0834791; -0.0834791 0.917815])\n", + "(Σa[i, j], Σe[i, j]) = ([0.268394 -0.141452; -0.141452 0.0796851], [0.72801 -0.0831954; -0.0831954 0.919263])\n", "Trait4Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.265695 -0.0970238; -0.0970238 0.0564809], [0.730259 -0.226115; -0.226115 0.942736])\n", + "(Σa[i, j], Σe[i, j]) = ([0.266334 -0.0971798; -0.0971798 0.0541478], [0.729966 -0.226082; -0.226082 0.945138])\n", "Trait5Trait6\n", - "(Σa[i, j], Σe[i, j]) = ([0.281198 0.280259; 0.280259 0.281764], [0.716855 0.661013; 0.661013 0.717735])\n", + "(Σa[i, j], Σe[i, j]) = ([0.281794 0.281033; 0.281033 0.282364], [0.716628 0.660607; 0.660607 0.717505])\n", "Trait5Trait7\n", - "(Σa[i, j], Σe[i, j]) = ([0.280442 0.231918; 0.231918 0.211837], [0.717597 0.674491; 0.674491 0.785172])\n", + "(Σa[i, j], Σe[i, j]) = ([0.281023 0.232223; 0.232223 0.211943], [0.717383 0.67449; 0.67449 0.785343])\n", "Trait5Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.280958 0.163168; 0.163168 0.193315], [0.717089 0.221817; 0.221817 0.806314])\n", + "(Σa[i, j], Σe[i, j]) = ([0.281551 0.164011; 0.164011 0.192737], [0.716865 0.221187; 0.221187 0.807144])\n", "Trait5Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.283544 0.243884; 0.243884 0.240564], [0.714585 0.509072; 0.509072 0.757631])\n", + "(Σa[i, j], Σe[i, j]) = ([0.284085 0.244479; 0.244479 0.240901], [0.714415 0.508795; 0.508795 0.757606])\n", "Trait5Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.281378 -0.0454427; -0.0454427 0.100081], [0.716678 -0.0579778; -0.0579778 0.899822])\n", + "(Σa[i, j], Σe[i, j]) = ([0.281993 -0.0460624; -0.0460624 0.100628], [0.716433 -0.0574194; -0.0574194 0.899407])\n", "Trait5Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.280066 0.0195669; 0.0195669 0.165607], [0.71795 -0.0345589; -0.0345589 0.833047])\n", + "(Σa[i, j], Σe[i, j]) = ([0.280652 0.0199425; 0.0199425 0.163738], [0.717732 -0.0349079; -0.0349079 0.83513])\n", "Trait5Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.28101 0.0592641; 0.0592641 0.0831831], [0.717036 0.0552788; 0.0552788 0.915608])\n", + "(Σa[i, j], Σe[i, j]) = ([0.281652 0.0612882; 0.0612882 0.0820476], [0.716765 0.0533362; 0.0533362 0.916851])\n", "Trait5Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.281854 0.0680641; 0.0680641 0.0591899], [0.716223 0.0551992; 0.0551992 0.940027])\n", + "(Σa[i, j], Σe[i, j]) = ([0.282494 0.0696137; 0.0696137 0.0570685], [0.715955 0.0537392; 0.0537392 0.942222])\n", "Trait6Trait7\n", - "(Σa[i, j], Σe[i, j]) = ([0.282435 0.220236; 0.220236 0.213997], [0.71708 0.581507; 0.581507 0.783041])\n", + "(Σa[i, j], Σe[i, j]) = ([0.283018 0.220794; 0.220794 0.214128], [0.716868 0.58124; 0.58124 0.783191])\n", "Trait6Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.282435 0.18375; 0.18375 0.192999], [0.717081 0.436932; 0.436932 0.80663])\n", + "(Σa[i, j], Σe[i, j]) = ([0.283018 0.183995; 0.183995 0.192405], [0.716869 0.436926; 0.436926 0.807474])\n", "Trait6Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.284516 0.233768; 0.233768 0.242478], [0.715071 0.477502; 0.477502 0.755765])\n", + "(Σa[i, j], Σe[i, j]) = ([0.285057 0.234311; 0.234311 0.24283], [0.714902 0.477263; 0.477263 0.755728])\n", "Trait6Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.283087 -0.0427658; -0.0427658 0.100634], [0.716449 -0.0599491; -0.0599491 0.899275])\n", + "(Σa[i, j], Σe[i, j]) = ([0.283699 -0.0433873; -0.0433873 0.101183], [0.716209 -0.0593852; -0.0593852 0.89886])\n", "Trait6Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.281046 0.0272144; 0.0272144 0.165044], [0.71843 -0.0516242; -0.0516242 0.833601])\n", + "(Σa[i, j], Σe[i, j]) = ([0.281626 0.027639; 0.027639 0.163178], [0.718219 -0.0520113; -0.0520113 0.83568])\n", "Trait6Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.28256 0.0548537; 0.0548537 0.083133], [0.716961 0.0502064; 0.0502064 0.915658])\n", + "(Σa[i, j], Σe[i, j]) = ([0.283196 0.0569533; 0.0569533 0.0819944], [0.716699 0.0481832; 0.0481832 0.916905])\n", "Trait6Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.283231 0.0585667; 0.0585667 0.0592752], [0.716314 0.055827; 0.055827 0.939942])\n", + "(Σa[i, j], Σe[i, j]) = ([0.283862 0.0600111; 0.0600111 0.0571748], [0.716057 0.0544601; 0.0544601 0.942116])\n", "Trait7Trait8\n", - "(Σa[i, j], Σe[i, j]) = ([0.213998 0.0875641; 0.0875641 0.192993], [0.78304 -0.055939; -0.055939 0.806635])\n", + "(Σa[i, j], Σe[i, j]) = ([0.214129 0.0883281; 0.0883281 0.192399], [0.78319 -0.0565873; -0.0565873 0.80748])\n", "Trait7Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.219039 0.216925; 0.216925 0.243338], [0.778156 0.463024; 0.463024 0.754935])\n", + "(Σa[i, j], Σe[i, j]) = ([0.219059 0.217228; 0.217228 0.243722], [0.778422 0.463004; 0.463004 0.754868])\n", "Trait7Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.216296 -0.0412106; -0.0412106 0.100663], [0.780785 -0.0868086; -0.0868086 0.899246])\n", + "(Σa[i, j], Σe[i, j]) = ([0.216477 -0.042081; -0.042081 0.101258], [0.78089 -0.0859952; -0.0859952 0.898786])\n", "Trait7Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.2142 0.0204227; 0.0204227 0.165077], [0.782833 -0.0478727; -0.0478727 0.833569])\n", + "(Σa[i, j], Σe[i, j]) = ([0.214308 0.0205244; 0.0205244 0.163203], [0.783006 -0.0479474; -0.0479474 0.835656])\n", "Trait7Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.215054 0.0738562; 0.0738562 0.0814228], [0.782012 0.0366272; 0.0366272 0.917365])\n", + "(Σa[i, j], Σe[i, j]) = ([0.215192 0.0752708; 0.0752708 0.0802075], [0.782157 0.0353123; 0.0353123 0.918686])\n", "Trait7Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.216093 0.0728515; 0.0728515 0.0570272], [0.781006 0.0409945; 0.0409945 0.942189])\n", + "(Σa[i, j], Σe[i, j]) = ([0.216198 0.0740251; 0.0740251 0.0547527], [0.781186 0.0399163; 0.0399163 0.944534])\n", "Trait8Trait9\n", - "(Σa[i, j], Σe[i, j]) = ([0.195154 0.111756; 0.111756 0.246453], [0.804528 0.185842; 0.185842 0.751896])\n", + "(Σa[i, j], Σe[i, j]) = ([0.194551 0.112282; 0.112282 0.246832], [0.805384 0.185461; 0.185461 0.751837])\n", "Trait8Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.195015 -0.015506; -0.015506 0.0990776], [0.804651 0.0118538; 0.0118538 0.900815])\n", + "(Σa[i, j], Σe[i, j]) = ([0.194399 -0.0154035; -0.0154035 0.0995883], [0.805519 0.0117323; 0.0117323 0.900435])\n", "Trait8Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.194421 0.0215044; 0.0215044 0.166226], [0.805231 -0.026247; -0.026247 0.83244])\n", + "(Σa[i, j], Σe[i, j]) = ([0.193837 0.022072; 0.022072 0.164393], [0.806067 -0.0267843; -0.0267843 0.834489])\n", "Trait8Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.194491 -0.00425152; -0.00425152 0.0832711], [0.805162 0.0349872; 0.0349872 0.915518])\n", + "(Σa[i, j], Σe[i, j]) = ([0.193914 -0.00259164; -0.00259164 0.0821237], [0.805992 0.0333257; 0.0333257 0.916773])\n", "Trait8Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.19448 0.00235501; 0.00235501 0.0589351], [0.805173 0.0396048; 0.0396048 0.940275])\n", + "(Σa[i, j], Σe[i, j]) = ([0.193913 0.00330925; 0.00330925 0.0569126], [0.805993 0.0386552; 0.0386552 0.94237])\n", "Trait9Trait10\n", - "(Σa[i, j], Σe[i, j]) = ([0.246455 -0.00257997; -0.00257997 0.0984563], [0.751895 0.0743439; 0.0743439 0.901429])\n", - "Trait9Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.247001 0.0303415; 0.0303415 0.166421], [0.75136 0.153765; 0.153765 0.832248])\n", + "(Σa[i, j], Σe[i, j]) = ([0.246837 -0.00323396; -0.00323396 0.0989732], [0.751834 0.074992; 0.074992 0.901043])\n", + "Trait9Trait11\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(Σa[i, j], Σe[i, j]) = ([0.247395 0.0305136; 0.0305136 0.164575], [0.751288 0.153633; 0.153633 0.834308])\n", "Trait9Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.249421 0.0829968; 0.0829968 0.0890874], [0.749007 0.109331; 0.109331 0.909778])\n", + "(Σa[i, j], Σe[i, j]) = ([0.249928 0.0844216; 0.0844216 0.0880992], [0.748829 0.108018; 0.108018 0.910884])\n", "Trait9Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.24861 0.0916799; 0.0916799 0.0602352], [0.749811 0.100027; 0.100027 0.939032])\n", + "(Σa[i, j], Σe[i, j]) = ([0.248998 0.0925603; 0.0925603 0.0580327], [0.749747 0.0992651; 0.0992651 0.94131])\n", "Trait10Trait11\n", - "(Σa[i, j], Σe[i, j]) = ([0.0914658 0.100613; 0.100613 0.166501], [0.908376 0.473847; 0.473847 0.83217])\n", + "(Σa[i, j], Σe[i, j]) = ([0.0923059 0.0990858; 0.0990858 0.164638], [0.907657 0.475503; 0.475503 0.834248])\n", "Trait10Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.0951392 0.0588424; 0.0588424 0.0796744], [0.904735 0.0828862; 0.0828862 0.919115])\n", + "(Σa[i, j], Σe[i, j]) = ([0.0957712 0.0574496; 0.0574496 0.0785646], [0.904228 0.0843534; 0.0843534 0.920326])\n", "Trait10Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.0995192 -0.0257171; -0.0257171 0.0598595], [0.900397 0.163778; 0.163778 0.939368])\n", + "(Σa[i, j], Σe[i, j]) = ([0.100109 -0.0266483; -0.0266483 0.0578511], [0.899941 0.164674; 0.164674 0.941451])\n", "Trait11Trait12\n", - "(Σa[i, j], Σe[i, j]) = ([0.165386 0.0579914; 0.0579914 0.0796005], [0.83327 0.144637; 0.144637 0.919166])\n", + "(Σa[i, j], Σe[i, j]) = ([0.163544 0.0571549; 0.0571549 0.0784378], [0.835325 0.145549; 0.145549 0.920432])\n", "Trait11Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.166417 -0.000985185; -0.000985185 0.0595681], [0.832265 0.200012; 0.200012 0.939646])\n", + "(Σa[i, j], Σe[i, j]) = ([0.164571 -0.00169939; -0.00169939 0.0575331], [0.834326 0.200725; 0.200725 0.941755])\n", "Trait12Trait13\n", - "(Σa[i, j], Σe[i, j]) = ([0.085082 0.0696185; 0.0696185 0.0569655], [0.913729 0.572041; 0.572041 0.942247])\n", - "elapsed time: 3.587337102 seconds\n" + "(Σa[i, j], Σe[i, j]) = ([0.08398 0.0685417; 0.0685417 0.0559416], [0.91494 0.573206; 0.573206 0.943343])\n", + " 4.430316 seconds (4.62 M allocations: 310.482 MiB, 3.51% gc time)\n" ] - }, - { - "data": { - "text/plain": [ - "3.587337102" - ] - }, - "execution_count": 52, - "metadata": {}, - "output_type": "execute_result" } ], "source": [ "# additive genetic effects (2x2 psd matrices) from bavariate trait analysis;\n", - "Σa = Array{Matrix{Float64}}(13, 13)\n", + "Σa = Array{Matrix{Float64}}(undef, 13, 13)\n", "# environmental effects (2x2 psd matrices) from bavariate trait analysis;\n", - "Σe = Array{Matrix{Float64}}(13, 13)\n", + "Σe = Array{Matrix{Float64}}(undef, 13, 13)\n", + "\n", "\n", - "tic()\n", - "for i in 1:13\n", + "@time for i in 1:13\n", " for j in (i+1):13\n", " println(names(cg10k_trait)[i + 2], names(cg10k_trait)[j + 2])\n", " # form data set for (trait1, trait2)\n", @@ -1367,16 +1256,12 @@ " Σe[i, j] = traitij_model.Σ[2]\n", " @show Σa[i, j], Σe[i, j]\n", " end\n", - "end\n", - "toc()" + "end" ] }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## 3-trait analysis\n", "\n", @@ -1385,18 +1270,14 @@ }, { "cell_type": "code", - "execution_count": 53, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 50, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "This is Ipopt version 3.12.4, running with linear solver mumps.\n", + "This is Ipopt version 3.12.10, running with linear solver mumps.\n", "NOTE: Other linear solvers might be more efficient (see Ipopt documentation).\n", "\n", "Number of nonzeros in equality constraint Jacobian...: 0\n", @@ -1414,26 +1295,26 @@ " inequality constraints with only upper bounds: 0\n", "\n", "iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls\n", - " 0 3.0247565e+04 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 \n", - " 5 1.6835078e+04 0.00e+00 4.08e+02 -11.0 3.64e-01 - 1.00e+00 1.00e+00f 1 MaxS\n", - " 10 1.4742941e+04 0.00e+00 1.10e+02 -11.0 2.35e-01 - 1.00e+00 1.00e+00f 1 MaxS\n", - " 15 1.4701394e+04 0.00e+00 1.16e+01 -11.0 7.78e-02 -4.5 1.00e+00 1.00e+00f 1 MaxS\n", - " 20 1.4701019e+04 0.00e+00 5.75e-01 -11.0 1.51e-04 -6.9 1.00e+00 1.00e+00f 1 MaxS\n", - " 25 1.4701018e+04 0.00e+00 2.40e-02 -11.0 6.38e-06 -9.2 1.00e+00 1.00e+00f 1 MaxS\n", - " 30 1.4701018e+04 0.00e+00 9.98e-04 -11.0 2.66e-07 -11.6 1.00e+00 1.00e+00f 1 MaxS\n", - " 35 1.4701018e+04 0.00e+00 4.15e-05 -11.0 1.10e-08 -14.0 1.00e+00 1.00e+00h 1 MaxS\n", - " 40 1.4701018e+04 0.00e+00 1.72e-06 -11.0 4.59e-10 -16.4 1.00e+00 1.00e+00f 1 MaxSA\n", - " 45 1.4701018e+04 0.00e+00 7.17e-08 -11.0 1.91e-11 -18.8 1.00e+00 1.00e+00h 1 MaxSA\n", + " 0 3.0244169e+04 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 \n", + " 5 1.6834042e+04 0.00e+00 4.07e+02 -11.0 3.66e-01 - 1.00e+00 1.00e+00f 1 MaxS\n", + " 10 1.4744248e+04 0.00e+00 1.12e+02 -11.0 2.36e-01 - 1.00e+00 1.00e+00f 1 MaxS\n", + " 15 1.4701415e+04 0.00e+00 1.25e+01 -11.0 1.08e-01 -4.5 1.00e+00 1.00e+00f 1 MaxS\n", + " 20 1.4700955e+04 0.00e+00 6.34e-01 -11.0 1.66e-04 -6.9 1.00e+00 1.00e+00f 1 MaxS\n", + " 25 1.4700954e+04 0.00e+00 2.63e-02 -11.0 7.01e-06 -9.2 1.00e+00 1.00e+00f 1 MaxS\n", + " 30 1.4700954e+04 0.00e+00 1.09e-03 -11.0 2.90e-07 -11.6 1.00e+00 1.00e+00f 1 MaxS\n", + " 35 1.4700954e+04 0.00e+00 4.49e-05 -11.0 1.20e-08 -14.0 1.00e+00 1.00e+00f 1 MaxS\n", + " 40 1.4700954e+04 0.00e+00 1.86e-06 -11.0 4.96e-10 -16.4 1.00e+00 1.00e+00f 1 MaxSA\n", + " 45 1.4700954e+04 0.00e+00 7.67e-08 -11.0 2.05e-11 -18.8 1.00e+00 1.00e+00h 1 MaxSA\n", "iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls\n", "\n", "Number of Iterations....: 49\n", "\n", " (scaled) (unscaled)\n", - "Objective...............: 4.4720359684330265e+02 1.4701017692082147e+04\n", - "Dual infeasibility......: 5.6081357364421780e-09 1.8435742302386474e-07\n", + "Objective...............: 4.4662368766169095e+02 1.4700954216526397e+04\n", + "Dual infeasibility......: 5.8982229663448267e-09 1.9414444012378635e-07\n", "Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00\n", "Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00\n", - "Overall NLP error.......: 5.6081357364421780e-09 1.8435742302386474e-07\n", + "Overall NLP error.......: 5.8982229663448267e-09 1.9414444012378635e-07\n", "\n", "\n", "Number of objective function evaluations = 50\n", @@ -1443,20 +1324,20 @@ "Number of equality constraint Jacobian evaluations = 0\n", "Number of inequality constraint Jacobian evaluations = 0\n", "Number of Lagrangian Hessian evaluations = 49\n", - "Total CPU secs in IPOPT (w/o function evaluations) = 0.014\n", - "Total CPU secs in NLP function evaluations = 0.076\n", + "Total CPU secs in IPOPT (w/o function evaluations) = 0.019\n", + "Total CPU secs in NLP function evaluations = 0.054\n", "\n", "EXIT: Optimal Solution Found.\n", - " 0.097955 seconds (55.15 k allocations: 5.632 MiB)\n" + " 0.084479 seconds (46.62 k allocations: 5.057 MiB)\n" ] }, { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.280777 0.279441 0.232208; 0.279441 0.28422 0.219831; 0.232208 0.219831 0.212832], [0.717266 0.66183 0.674206; 0.66183 0.715287 0.581891; 0.674206 0.581891 0.784183]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.28137 0.280153 0.232512; 0.280153 0.284916 0.220363; 0.232512 0.220363 0.212921], [0.717042 0.661484 0.674207; 0.661484 0.714964 0.581648; 0.674207 0.581648 0.784373]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf)" ] }, - "execution_count": 53, + "execution_count": 50, "metadata": {}, "output_type": "execute_result" } @@ -1475,22 +1356,15 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "We then run the MM algorithm, starting from the Fisher scoring answer. MM finds an improved solution with objective value 8.955397e+03." ] }, { "cell_type": "code", - "execution_count": 54, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 51, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -1500,19 +1374,19 @@ " MM Algorithm\n", " Iter Objective \n", "-------- -------------\n", - " 0 -1.470102e+04\n", - " 1 -1.470102e+04\n", + " 0 -1.470095e+04\n", + " 1 -1.470095e+04\n", "\n", - " 0.003006 seconds (21.01 k allocations: 1.551 MiB)\n" + " 0.505115 seconds (1.01 M allocations: 50.370 MiB, 5.09% gc time)\n" ] }, { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.280777 0.279441 0.232208; 0.279441 0.28422 0.219831; 0.232208 0.219831 0.212832], [0.717266 0.66183 0.674206; 0.66183 0.715287 0.581891; 0.674206 0.581891 0.784183]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.28137 0.280153 0.232512; 0.280153 0.284916 0.220363; 0.232512 0.220363 0.212921], [0.717042 0.661484 0.674207; 0.661484 0.714964 0.581648; 0.674207 0.581648 0.784373]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf)" ] }, - "execution_count": 54, + "execution_count": 51, "metadata": {}, "output_type": "execute_result" } @@ -1525,22 +1399,15 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "Do another run of MM algorithm from default starting point. It leads to a slightly better local optimum -1.470104e+04, slighly worse than the Fisher scoring result. Follow up anlaysis should use the Fisher scoring result." ] }, { "cell_type": "code", - "execution_count": 55, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 52, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -1550,139 +1417,138 @@ " MM Algorithm\n", " Iter Objective \n", "-------- -------------\n", - " 0 -3.024757e+04\n", - " 1 -2.040300e+04\n", - " 2 -1.656070e+04\n", - " 3 -1.528529e+04\n", - " 4 -1.490986e+04\n", - " 5 -1.480638e+04\n", - " 6 -1.477811e+04\n", - " 7 -1.476968e+04\n", - " 8 -1.476639e+04\n", - " 9 -1.476444e+04\n", - " 10 -1.476286e+04\n", - " 20 -1.475000e+04\n", - " 30 -1.474011e+04\n", - " 40 -1.473248e+04\n", - " 50 -1.472658e+04\n", - " 60 -1.472200e+04\n", - " 70 -1.471840e+04\n", - " 80 -1.471555e+04\n", - " 90 -1.471328e+04\n", - " 100 -1.471145e+04\n", - " 110 -1.470997e+04\n", - " 120 -1.470875e+04\n", - " 130 -1.470775e+04\n", - " 140 -1.470691e+04\n", - " 150 -1.470621e+04\n", - " 160 -1.470562e+04\n", - " 170 -1.470511e+04\n", - " 180 -1.470469e+04\n", - " 190 -1.470432e+04\n", - " 200 -1.470400e+04\n", - " 210 -1.470372e+04\n", - " 220 -1.470348e+04\n", - " 230 -1.470326e+04\n", - " 240 -1.470308e+04\n", - " 250 -1.470291e+04\n", - " 260 -1.470276e+04\n", - " 270 -1.470263e+04\n", - " 280 -1.470251e+04\n", - " 290 -1.470241e+04\n", - " 300 -1.470231e+04\n", - " 310 -1.470223e+04\n", - " 320 -1.470215e+04\n", - " 330 -1.470208e+04\n", - " 340 -1.470201e+04\n", - " 350 -1.470195e+04\n", - " 360 -1.470190e+04\n", - " 370 -1.470185e+04\n", - " 380 -1.470180e+04\n", - " 390 -1.470176e+04\n", - " 400 -1.470172e+04\n", - " 410 -1.470168e+04\n", - " 420 -1.470165e+04\n", - " 430 -1.470162e+04\n", - " 440 -1.470159e+04\n", - " 450 -1.470156e+04\n", - " 460 -1.470153e+04\n", - " 470 -1.470151e+04\n", - " 480 -1.470149e+04\n", - " 490 -1.470147e+04\n", - " 500 -1.470145e+04\n", - " 510 -1.470143e+04\n", - " 520 -1.470141e+04\n", - " 530 -1.470139e+04\n", - " 540 -1.470138e+04\n", - " 550 -1.470136e+04\n", - " 560 -1.470135e+04\n", - " 570 -1.470133e+04\n", - " 580 -1.470132e+04\n", - " 590 -1.470131e+04\n", - " 600 -1.470130e+04\n", - " 610 -1.470129e+04\n", - " 620 -1.470128e+04\n", - " 630 -1.470127e+04\n", - " 640 -1.470126e+04\n", - " 650 -1.470125e+04\n", - " 660 -1.470124e+04\n", - " 670 -1.470123e+04\n", - " 680 -1.470122e+04\n", - " 690 -1.470122e+04\n", - " 700 -1.470121e+04\n", - " 710 -1.470120e+04\n", - " 720 -1.470120e+04\n", - " 730 -1.470119e+04\n", - " 740 -1.470118e+04\n", - " 750 -1.470118e+04\n", - " 760 -1.470117e+04\n", - " 770 -1.470117e+04\n", - " 780 -1.470116e+04\n", - " 790 -1.470116e+04\n", - " 800 -1.470115e+04\n", - " 810 -1.470115e+04\n", - " 820 -1.470114e+04\n", - " 830 -1.470114e+04\n", - " 840 -1.470114e+04\n", - " 850 -1.470113e+04\n", - " 860 -1.470113e+04\n", - " 870 -1.470112e+04\n", - " 880 -1.470112e+04\n", - " 890 -1.470112e+04\n", - " 900 -1.470111e+04\n", - " 910 -1.470111e+04\n", - " 920 -1.470111e+04\n", - " 930 -1.470111e+04\n", - " 940 -1.470110e+04\n", - " 950 -1.470110e+04\n", - " 960 -1.470110e+04\n", - " 970 -1.470109e+04\n", - " 980 -1.470109e+04\n", - " 990 -1.470109e+04\n", - " 1000 -1.470109e+04\n", - " 1010 -1.470109e+04\n", - " 1020 -1.470108e+04\n", - " 1030 -1.470108e+04\n", - " 1040 -1.470108e+04\n", - " 1050 -1.470108e+04\n", - " 1060 -1.470108e+04\n", - " 1070 -1.470107e+04\n", - " 1080 -1.470107e+04\n", - " 1090 -1.470107e+04\n", - " 1100 -1.470107e+04\n", - " 1110 -1.470107e+04\n", - " 1120 -1.470107e+04\n", + " 0 -3.024417e+04\n", + " 1 -2.040172e+04\n", + " 2 -1.656053e+04\n", + " 3 -1.528554e+04\n", + " 4 -1.491023e+04\n", + " 5 -1.480677e+04\n", + " 6 -1.477849e+04\n", + " 7 -1.477005e+04\n", + " 8 -1.476675e+04\n", + " 9 -1.476478e+04\n", + " 10 -1.476318e+04\n", + " 20 -1.475020e+04\n", + " 30 -1.474023e+04\n", + " 40 -1.473255e+04\n", + " 50 -1.472662e+04\n", + " 60 -1.472201e+04\n", + " 70 -1.471839e+04\n", + " 80 -1.471554e+04\n", + " 90 -1.471326e+04\n", + " 100 -1.471142e+04\n", + " 110 -1.470993e+04\n", + " 120 -1.470871e+04\n", + " 130 -1.470770e+04\n", + " 140 -1.470686e+04\n", + " 150 -1.470616e+04\n", + " 160 -1.470557e+04\n", + " 170 -1.470506e+04\n", + " 180 -1.470463e+04\n", + " 190 -1.470426e+04\n", + " 200 -1.470394e+04\n", + " 210 -1.470366e+04\n", + " 220 -1.470342e+04\n", + " 230 -1.470321e+04\n", + " 240 -1.470302e+04\n", + " 250 -1.470285e+04\n", + " 260 -1.470270e+04\n", + " 270 -1.470257e+04\n", + " 280 -1.470245e+04\n", + " 290 -1.470234e+04\n", + " 300 -1.470225e+04\n", + " 310 -1.470216e+04\n", + " 320 -1.470208e+04\n", + " 330 -1.470201e+04\n", + " 340 -1.470195e+04\n", + " 350 -1.470189e+04\n", + " 360 -1.470183e+04\n", + " 370 -1.470178e+04\n", + " 380 -1.470173e+04\n", + " 390 -1.470169e+04\n", + " 400 -1.470165e+04\n", + " 410 -1.470162e+04\n", + " 420 -1.470158e+04\n", + " 430 -1.470155e+04\n", + " 440 -1.470152e+04\n", + " 450 -1.470149e+04\n", + " 460 -1.470147e+04\n", + " 470 -1.470144e+04\n", + " 480 -1.470142e+04\n", + " 490 -1.470140e+04\n", + " 500 -1.470138e+04\n", + " 510 -1.470136e+04\n", + " 520 -1.470134e+04\n", + " 530 -1.470132e+04\n", + " 540 -1.470131e+04\n", + " 550 -1.470129e+04\n", + " 560 -1.470128e+04\n", + " 570 -1.470127e+04\n", + " 580 -1.470125e+04\n", + " 590 -1.470124e+04\n", + " 600 -1.470123e+04\n", + " 610 -1.470122e+04\n", + " 620 -1.470121e+04\n", + " 630 -1.470120e+04\n", + " 640 -1.470119e+04\n", + " 650 -1.470118e+04\n", + " 660 -1.470117e+04\n", + " 670 -1.470116e+04\n", + " 680 -1.470116e+04\n", + " 690 -1.470115e+04\n", + " 700 -1.470114e+04\n", + " 710 -1.470113e+04\n", + " 720 -1.470113e+04\n", + " 730 -1.470112e+04\n", + " 740 -1.470112e+04\n", + " 750 -1.470111e+04\n", + " 760 -1.470110e+04\n", + " 770 -1.470110e+04\n", + " 780 -1.470109e+04\n", + " 790 -1.470109e+04\n", + " 800 -1.470108e+04\n", + " 810 -1.470108e+04\n", + " 820 -1.470108e+04\n", + " 830 -1.470107e+04\n", + " 840 -1.470107e+04\n", + " 850 -1.470106e+04\n", + " 860 -1.470106e+04\n", + " 870 -1.470106e+04\n", + " 880 -1.470105e+04\n", + " 890 -1.470105e+04\n", + " 900 -1.470105e+04\n", + " 910 -1.470104e+04\n", + " 920 -1.470104e+04\n", + " 930 -1.470104e+04\n", + " 940 -1.470103e+04\n", + " 950 -1.470103e+04\n", + " 960 -1.470103e+04\n", + " 970 -1.470103e+04\n", + " 980 -1.470102e+04\n", + " 990 -1.470102e+04\n", + " 1000 -1.470102e+04\n", + " 1010 -1.470102e+04\n", + " 1020 -1.470102e+04\n", + " 1030 -1.470101e+04\n", + " 1040 -1.470101e+04\n", + " 1050 -1.470101e+04\n", + " 1060 -1.470101e+04\n", + " 1070 -1.470101e+04\n", + " 1080 -1.470101e+04\n", + " 1090 -1.470100e+04\n", + " 1100 -1.470100e+04\n", + " 1110 -1.470100e+04\n", "\n", - " 0.794377 seconds (168.12 k allocations: 15.640 MiB, 0.80% gc time)\n" + " 0.744170 seconds (134.31 k allocations: 13.430 MiB, 1.57% gc time)\n" ] }, { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.2808 0.279454 0.232256; 0.279454 0.284312 0.219977; 0.232256 0.219977 0.213052], [0.717243 0.661816 0.674158; 0.661816 0.715193 0.581746; 0.674158 0.581746 0.783965]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.281394 0.280169 0.232564; 0.280169 0.285001 0.220511; 0.232564 0.220511 0.213147], [0.717018 0.661467 0.674155; 0.661467 0.714877 0.581502; 0.674155 0.581502 0.784149]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf)" ] }, - "execution_count": 55, + "execution_count": 52, "metadata": {}, "output_type": "execute_result" } @@ -1696,32 +1562,25 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "Heritability from 3-variate estimate and their standard errors." ] }, { "cell_type": "code", - "execution_count": 56, - "metadata": { - "collapsed": false, - "deletable": true, - "editable": true - }, + "execution_count": 53, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2×3 Array{Float64,2}:\n", - " 0.281351 0.284453 0.213689\n", - " 0.0778252 0.077378 0.084084" + " 0.281842 0.285036 0.213725 \n", + " 0.0777056 0.0772501 0.0840114" ] }, - "execution_count": 56, + "execution_count": 53, "metadata": {}, "output_type": "execute_result" } @@ -1742,16 +1601,14 @@ }, { "cell_type": "code", - "execution_count": 57, - "metadata": { - "collapsed": false - }, + "execution_count": 54, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "This is Ipopt version 3.12.4, running with linear solver mumps.\n", + "This is Ipopt version 3.12.10, running with linear solver mumps.\n", "NOTE: Other linear solvers might be more efficient (see Ipopt documentation).\n", "\n", "Number of nonzeros in equality constraint Jacobian...: 0\n", @@ -1769,32 +1626,34 @@ " inequality constraints with only upper bounds: 0\n", "\n", "iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls\n", - " 0 1.3113371e+05 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 \n", - " 5 8.2233766e+04 0.00e+00 6.03e+02 -11.0 2.32e+00 - 1.00e+00 1.00e+00f 1 MaxS\n", - " 10 1.1960260e+05 0.00e+00 8.76e+02 -11.0 6.20e+01 -5.4 1.00e+00 1.00e+00h 1 MaxS\n", - " 15 2.4416551e+05 0.00e+00 2.50e+02 -11.0 8.69e+02 -7.8 1.00e+00 1.00e+00f 1 MaxS\n" + " 0 1.3111983e+05 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 \n", + " 5 8.2228529e+04 0.00e+00 6.03e+02 -11.0 2.42e+00 - 1.00e+00 1.00e+00f 1 MaxS\n", + " 10 1.2570490e+05 0.00e+00 9.38e+02 -11.0 6.72e+01 -5.4 1.00e+00 1.00e+00h 1 MaxS\n" ] }, { - "ename": "LoadError", - "evalue": "DomainError:\nlog will only return a complex result if called with a complex argument. Try log(complex(x)).", + "ename": "PosDefException", + "evalue": "PosDefException: matrix is not positive definite; Cholesky factorization failed.", "output_type": "error", "traceback": [ - "DomainError:\nlog will only return a complex result if called with a complex argument. Try log(complex(x)).", + "PosDefException: matrix is not positive definite; Cholesky factorization failed.", "", "Stacktrace:", - " [1] \u001b[1mnan_dom_err\u001b[22m\u001b[22m at \u001b[1m./math.jl:300\u001b[22m\u001b[22m [inlined]", - " [2] \u001b[1mlog\u001b[22m\u001b[22m at \u001b[1m./math.jl:419\u001b[22m\u001b[22m [inlined]", - " [3] \u001b[1mlogdet\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Float64,2}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./linalg/generic.jl:1244\u001b[22m\u001b[22m", - " [4] \u001b[1mVarianceComponentModels.TwoVarCompModelRotate\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/VarianceComponentModels/src/VarianceComponentModels.jl:127\u001b[22m\u001b[22m", - " [5] \u001b[1meval_f\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentModels.VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}, ::Array{Float64,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/VarianceComponentModels/src/two_variance_component.jl:683\u001b[22m\u001b[22m", - " [6] \u001b[1m(::Ipopt.#eval_f_cb#4{VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentModels.VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}})\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Float64,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/Ipopt/src/IpoptSolverInterface.jl:53\u001b[22m\u001b[22m", - " [7] \u001b[1meval_f_wrapper\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Int32, ::Ptr{Float64}, ::Int32, ::Ptr{Float64}, ::Ptr{Void}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/Ipopt/src/Ipopt.jl:89\u001b[22m\u001b[22m", - " [8] \u001b[1msolveProblem\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Ipopt.IpoptProblem\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/Ipopt/src/Ipopt.jl:304\u001b[22m\u001b[22m", - " [9] \u001b[1moptimize!\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Ipopt.IpoptMathProgModel\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/Ipopt/src/IpoptSolverInterface.jl:120\u001b[22m\u001b[22m", - " [10] \u001b[1m#mle_fs!#29\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Int64, ::Symbol, ::Symbol, ::Bool, ::Function, ::VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/Users/huazhou/.julia/v0.6/VarianceComponentModels/src/two_variance_component.jl:893\u001b[22m\u001b[22m", - " [11] \u001b[1m(::VarianceComponentModels.#kw##mle_fs!)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Any,1}, ::VarianceComponentModels.#mle_fs!, ::VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./:0\u001b[22m\u001b[22m", - " [12] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./loading.jl:515\u001b[22m\u001b[22m" + " [1] chkposdef at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/lapack.jl:50 [inlined]", + " [2] sygvd!(::Int64, ::Char, ::Char, ::Array{Float64,2}, ::Array{Float64,2}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/lapack.jl:5075", + " [3] eigen! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/symmetric.jl:646 [inlined]", + " [4] eigen(::Symmetric{Float64,Array{Float64,2}}, ::Symmetric{Float64,Array{Float64,2}}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/eigen.jl:383", + " [5] TwoVarCompModelRotate(::VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}) at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/VarianceComponentModels.jl:119", + " [6] logpdf at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/two_variance_component.jl:62 [inlined]", + " [7] eval_f(::VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}, ::Array{Float64,1}) at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/two_variance_component.jl:731", + " [8] (::getfield(Ipopt, Symbol(\"#eval_f_cb#6\")){VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}})(::Array{Float64,1}) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/MPB_wrapper.jl:64", + " [9] eval_f_wrapper(::Int32, ::Ptr{Float64}, ::Int32, ::Ptr{Float64}, ::Ptr{Nothing}) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/Ipopt.jl:128", + " [10] solveProblem(::Ipopt.IpoptProblem) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/Ipopt.jl:346", + " [11] optimize!(::Ipopt.IpoptMathProgModel) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/MPB_wrapper.jl:141", + " [12] #mle_fs!#27(::Int64, ::Symbol, ::Symbol, ::Bool, ::Function, ::VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}) at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/two_variance_component.jl:949", + " [13] (::getfield(VarianceComponentModels, Symbol(\"#kw##mle_fs!\")))(::NamedTuple{(:solver, :verbose),Tuple{Symbol,Bool}}, ::typeof(mle_fs!), ::VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}) at ./none:0", + " [14] top-level scope at util.jl:156", + " [15] top-level scope at In[54]:3" ] } ], @@ -1814,10 +1673,8 @@ }, { "cell_type": "code", - "execution_count": 58, - "metadata": { - "collapsed": false - }, + "execution_count": 55, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -1827,136 +1684,135 @@ " MM Algorithm\n", " Iter Objective \n", "-------- -------------\n", - " 0 -1.311337e+05\n", - " 1 -8.002108e+04\n", - " 2 -5.806935e+04\n", - " 3 -4.926111e+04\n", - " 4 -4.611059e+04\n", - " 5 -4.511606e+04\n", - " 6 -4.482679e+04\n", - " 7 -4.474294e+04\n", - " 8 -4.471496e+04\n", - " 9 -4.470174e+04\n", - " 10 -4.469246e+04\n", - " 20 -4.462243e+04\n", - " 30 -4.456888e+04\n", - " 40 -4.452774e+04\n", - " 50 -4.449601e+04\n", - " 60 -4.447134e+04\n", - " 70 -4.445199e+04\n", - " 80 -4.443665e+04\n", - " 90 -4.442436e+04\n", - " 100 -4.441442e+04\n", - " 110 -4.440630e+04\n", - " 120 -4.439961e+04\n", - " 130 -4.439405e+04\n", - " 140 -4.438938e+04\n", - " 150 -4.438544e+04\n", - " 160 -4.438210e+04\n", - " 170 -4.437923e+04\n", - " 180 -4.437676e+04\n", - " 190 -4.437463e+04\n", - " 200 -4.437277e+04\n", - " 210 -4.437115e+04\n", - " 220 -4.436972e+04\n", - " 230 -4.436846e+04\n", - " 240 -4.436735e+04\n", - " 250 -4.436636e+04\n", - " 260 -4.436548e+04\n", - " 270 -4.436469e+04\n", - " 280 -4.436399e+04\n", - " 290 -4.436335e+04\n", - " 300 -4.436278e+04\n", - " 310 -4.436226e+04\n", - " 320 -4.436179e+04\n", - " 330 -4.436137e+04\n", - " 340 -4.436098e+04\n", - " 350 -4.436063e+04\n", - " 360 -4.436030e+04\n", - " 370 -4.436001e+04\n", - " 380 -4.435974e+04\n", - " 390 -4.435949e+04\n", - " 400 -4.435926e+04\n", - " 410 -4.435905e+04\n", - " 420 -4.435886e+04\n", - " 430 -4.435868e+04\n", - " 440 -4.435851e+04\n", - " 450 -4.435836e+04\n", - " 460 -4.435822e+04\n", - " 470 -4.435809e+04\n", - " 480 -4.435797e+04\n", - " 490 -4.435785e+04\n", - " 500 -4.435775e+04\n", - " 510 -4.435765e+04\n", - " 520 -4.435756e+04\n", - " 530 -4.435747e+04\n", - " 540 -4.435739e+04\n", - " 550 -4.435732e+04\n", - " 560 -4.435725e+04\n", - " 570 -4.435718e+04\n", - " 580 -4.435712e+04\n", - " 590 -4.435706e+04\n", - " 600 -4.435701e+04\n", - " 610 -4.435696e+04\n", - " 620 -4.435691e+04\n", - " 630 -4.435687e+04\n", - " 640 -4.435683e+04\n", - " 650 -4.435679e+04\n", - " 660 -4.435675e+04\n", - " 670 -4.435671e+04\n", - " 680 -4.435668e+04\n", - " 690 -4.435665e+04\n", - " 700 -4.435662e+04\n", - " 710 -4.435659e+04\n", - " 720 -4.435657e+04\n", - " 730 -4.435654e+04\n", - " 740 -4.435652e+04\n", - " 750 -4.435649e+04\n", - " 760 -4.435647e+04\n", - " 770 -4.435645e+04\n", - " 780 -4.435643e+04\n", - " 790 -4.435642e+04\n", - " 800 -4.435640e+04\n", - " 810 -4.435638e+04\n", - " 820 -4.435637e+04\n", - " 830 -4.435635e+04\n", - " 840 -4.435634e+04\n", - " 850 -4.435633e+04\n", - " 860 -4.435631e+04\n", - " 870 -4.435630e+04\n", - " 880 -4.435629e+04\n", - " 890 -4.435628e+04\n", - " 900 -4.435627e+04\n", - " 910 -4.435626e+04\n", - " 920 -4.435625e+04\n", - " 930 -4.435624e+04\n", - " 940 -4.435623e+04\n", - " 950 -4.435622e+04\n", - " 960 -4.435621e+04\n", - " 970 -4.435621e+04\n", - " 980 -4.435620e+04\n", - " 990 -4.435619e+04\n", - " 1000 -4.435619e+04\n", - " 1010 -4.435618e+04\n", - " 1020 -4.435617e+04\n", - " 1030 -4.435617e+04\n", - " 1040 -4.435616e+04\n", - " 1050 -4.435616e+04\n", - " 1060 -4.435615e+04\n", - " 1070 -4.435615e+04\n", - " 1080 -4.435614e+04\n", - " 1090 -4.435614e+04\n", + " 0 -1.311198e+05\n", + " 1 -8.001595e+04\n", + " 2 -5.806836e+04\n", + " 3 -4.926167e+04\n", + " 4 -4.611163e+04\n", + " 5 -4.511722e+04\n", + " 6 -4.482795e+04\n", + " 7 -4.474405e+04\n", + " 8 -4.471603e+04\n", + " 9 -4.470277e+04\n", + " 10 -4.469345e+04\n", + " 20 -4.462308e+04\n", + " 30 -4.456930e+04\n", + " 40 -4.452802e+04\n", + " 50 -4.449620e+04\n", + " 60 -4.447148e+04\n", + " 70 -4.445209e+04\n", + " 80 -4.443672e+04\n", + " 90 -4.442442e+04\n", + " 100 -4.441448e+04\n", + " 110 -4.440635e+04\n", + " 120 -4.439966e+04\n", + " 130 -4.439409e+04\n", + " 140 -4.438943e+04\n", + " 150 -4.438549e+04\n", + " 160 -4.438215e+04\n", + " 170 -4.437928e+04\n", + " 180 -4.437682e+04\n", + " 190 -4.437469e+04\n", + " 200 -4.437283e+04\n", + " 210 -4.437121e+04\n", + " 220 -4.436978e+04\n", + " 230 -4.436853e+04\n", + " 240 -4.436742e+04\n", + " 250 -4.436643e+04\n", + " 260 -4.436555e+04\n", + " 270 -4.436476e+04\n", + " 280 -4.436406e+04\n", + " 290 -4.436342e+04\n", + " 300 -4.436285e+04\n", + " 310 -4.436234e+04\n", + " 320 -4.436187e+04\n", + " 330 -4.436144e+04\n", + " 340 -4.436106e+04\n", + " 350 -4.436071e+04\n", + " 360 -4.436039e+04\n", + " 370 -4.436009e+04\n", + " 380 -4.435982e+04\n", + " 390 -4.435957e+04\n", + " 400 -4.435935e+04\n", + " 410 -4.435914e+04\n", + " 420 -4.435895e+04\n", + " 430 -4.435877e+04\n", + " 440 -4.435860e+04\n", + " 450 -4.435845e+04\n", + " 460 -4.435831e+04\n", + " 470 -4.435818e+04\n", + " 480 -4.435806e+04\n", + " 490 -4.435794e+04\n", + " 500 -4.435784e+04\n", + " 510 -4.435774e+04\n", + " 520 -4.435765e+04\n", + " 530 -4.435757e+04\n", + " 540 -4.435749e+04\n", + " 550 -4.435741e+04\n", + " 560 -4.435734e+04\n", + " 570 -4.435728e+04\n", + " 580 -4.435722e+04\n", + " 590 -4.435716e+04\n", + " 600 -4.435711e+04\n", + " 610 -4.435706e+04\n", + " 620 -4.435701e+04\n", + " 630 -4.435696e+04\n", + " 640 -4.435692e+04\n", + " 650 -4.435688e+04\n", + " 660 -4.435685e+04\n", + " 670 -4.435681e+04\n", + " 680 -4.435678e+04\n", + " 690 -4.435675e+04\n", + " 700 -4.435672e+04\n", + " 710 -4.435669e+04\n", + " 720 -4.435666e+04\n", + " 730 -4.435664e+04\n", + " 740 -4.435662e+04\n", + " 750 -4.435659e+04\n", + " 760 -4.435657e+04\n", + " 770 -4.435655e+04\n", + " 780 -4.435653e+04\n", + " 790 -4.435652e+04\n", + " 800 -4.435650e+04\n", + " 810 -4.435648e+04\n", + " 820 -4.435647e+04\n", + " 830 -4.435645e+04\n", + " 840 -4.435644e+04\n", + " 850 -4.435643e+04\n", + " 860 -4.435641e+04\n", + " 870 -4.435640e+04\n", + " 880 -4.435639e+04\n", + " 890 -4.435638e+04\n", + " 900 -4.435637e+04\n", + " 910 -4.435636e+04\n", + " 920 -4.435635e+04\n", + " 930 -4.435634e+04\n", + " 940 -4.435633e+04\n", + " 950 -4.435633e+04\n", + " 960 -4.435632e+04\n", + " 970 -4.435631e+04\n", + " 980 -4.435630e+04\n", + " 990 -4.435630e+04\n", + " 1000 -4.435629e+04\n", + " 1010 -4.435628e+04\n", + " 1020 -4.435628e+04\n", + " 1030 -4.435627e+04\n", + " 1040 -4.435627e+04\n", + " 1050 -4.435626e+04\n", + " 1060 -4.435626e+04\n", + " 1070 -4.435625e+04\n", + " 1080 -4.435625e+04\n", "\n", - " 3.551301 seconds (178.42 k allocations: 70.115 MiB, 0.42% gc time)\n" + " 3.314963 seconds (131.35 k allocations: 65.449 MiB, 0.65% gc time)\n" ] }, { "data": { "text/plain": [ - "(-44356.138529861186, VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,13), ([0.272384 0.190358 … -0.128222 -0.0980655; 0.190358 0.21692 … -0.0689912 -0.0444349; … ; -0.128222 -0.0689912 … 0.118227 0.0909188; -0.0980655 -0.0444349 … 0.0909188 0.107456], [0.724562 0.56992 … -0.0590518 -0.124939; 0.56992 0.782639 … 0.0238629 0.0475408; … ; -0.0590518 0.0238629 … 0.880671 0.550889; -0.124939 0.0475408 … 0.550889 0.891929]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf), ([0.0111619 0.0131088 … 0.0128956 0.0127641; 0.0131091 0.0151759 … 0.017162 0.0171466; … ; 0.0128956 0.017162 … 0.0173994 0.0182002; 0.0127643 0.0171461 … 0.0182003 0.0187848], [0.0112235 0.0133094 … 0.0130111 0.0127861; 0.01331 0.0158262 … 0.017867 0.017798; … ; 0.013011 0.0178666 … 0.0179487 0.0187579; 0.012786 0.0177975 … 0.0187578 0.0193328]), [0.000124587 7.24074e-5 … -3.35716e-7 -1.40982e-5; 7.24411e-5 0.000171849 … -2.05381e-5 -3.17975e-6; … ; -3.60221e-7 -2.05683e-5 … 0.000351859 -1.5168e-5; -1.40799e-5 -3.16738e-6 … -1.51641e-5 0.000373756], Array{Float64}(0,13), Array{Float64}(0,0))" + "(-44356.24416259489, VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,13), ([0.272112 0.190023 … -0.128464 -0.0976418; 0.190023 0.216814 … -0.0687833 -0.04341; … ; -0.128464 -0.0687833 … 0.116994 0.0900933; -0.0976418 -0.04341 … 0.0900933 0.105876], [0.725183 0.570497 … -0.0589748 -0.125486; 0.570497 0.783023 … 0.0235685 0.0464638; … ; -0.0589748 0.0235685 … 0.882056 0.551829; -0.125486 0.0464638 … 0.551829 0.893642]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf), ([0.0111603 0.013107 … 0.0128915 0.0127587; 0.0131027 0.0151631 … 0.017153 0.0171359; … ; 0.0128908 0.017153 … 0.0174176 0.018212; 0.0127586 0.0171361 … 0.0182122 0.0188002], [0.0112235 0.0133041 … 0.0130038 0.0127778; 0.0133005 0.0158053 … 0.0178518 0.0177823; … ; 0.0130043 0.0178518 … 0.0179557 0.0187638; 0.0127775 0.0177823 … 0.0187637 0.0193477]), [0.000124552 7.23469e-5 … -3.6584e-7 -1.40474e-5; 7.23585e-5 0.00017168 … -2.04611e-5 -3.18804e-6; … ; -3.70686e-7 -2.04634e-5 … 0.000352082 -1.46096e-5; -1.4039e-5 -3.1795e-6 … -1.46073e-5 0.000374334], Array{Float64}(0,13), Array{Float64}(0,0))" ] }, - "execution_count": 58, + "execution_count": 55, "metadata": {}, "output_type": "execute_result" } @@ -1977,44 +1833,38 @@ }, { "cell_type": "markdown", - "metadata": { - "deletable": true, - "editable": true - }, + "metadata": {}, "source": [ "## Save analysis results" ] }, { "cell_type": "code", - "execution_count": 59, + "execution_count": null, "metadata": { - "collapsed": false, - "deletable": true, - "editable": true + "collapsed": true }, "outputs": [], "source": [ - "#using JLD\n", - "#@save \"copd.jld\"\n", - "#whos()" + "#using JLD2, FileIO\n", + "#save(\"copd.jld2\")\n", + "#varinfo()" ] } ], "metadata": { - "anaconda-cloud": {}, "kernelspec": { - "display_name": "Julia 0.6.0", + "display_name": "Julia 1.1.0", "language": "julia", - "name": "julia-0.6" + "name": "julia-1.1" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", - "version": "0.6.0" + "version": "1.1.0" } }, "nbformat": 4, - "nbformat_minor": 0 + "nbformat_minor": 2 } diff --git a/docs/mle_reml.ipynb b/docs/mle_reml.ipynb index d7f69dc..3a904ea 100644 --- a/docs/mle_reml.ipynb +++ b/docs/mle_reml.ipynb @@ -16,25 +16,21 @@ }, { "cell_type": "code", - "execution_count": 37, - "metadata": { - "collapsed": false - }, + "execution_count": 1, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Julia Version 0.6.0\n", - "Commit 903644385b (2017-06-19 13:05 UTC)\n", + "Julia Version 1.1.0\n", + "Commit 80516ca202 (2019-01-21 21:24 UTC)\n", "Platform Info:\n", - " OS: macOS (x86_64-apple-darwin13.4.0)\n", - " CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz\n", + " OS: macOS (x86_64-apple-darwin14.5.0)\n", + " CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz\n", " WORD_SIZE: 64\n", - " BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)\n", - " LAPACK: libopenblas64_\n", " LIBM: libopenlibm\n", - " LLVM: libLLVM-3.9.1 (ORCJIT, haswell)\n" + " LLVM: libLLVM-6.0.1 (ORCJIT, skylake)\n" ] } ], @@ -58,14 +54,15 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 3, "metadata": { - "collapsed": false + "collapsed": true }, "outputs": [], "source": [ "# generate data from a d-variate response variane component model\n", - "srand(123)\n", + "using Random, LinearAlgebra\n", + "Random.seed!(123)\n", "n = 1000 # no. observations\n", "d = 2 # dimension of responses\n", "m = 2 # no. variance components\n", @@ -78,23 +75,23 @@ "V = ntuple(x -> zeros(n, n), m) \n", "for i = 1:m-1\n", " Vi = randn(n, 50)\n", - " copy!(V[i], Vi * Vi')\n", + " copyto!(V[i], Vi * Vi')\n", "end\n", - "copy!(V[m], eye(n)) # last covarianec matrix is idendity\n", + "copyto!(V[m], Matrix(I, n, n)) # last covarianec matrix is idendity\n", "# a tuple of m d-by-d variance component parameters\n", "Σ = ntuple(x -> zeros(d, d), m) \n", "for i in 1:m\n", " Σi = randn(d, d)\n", - " copy!(Σ[i], Σi' * Σi)\n", + " copyto!(Σ[i], Σi' * Σi)\n", "end\n", "# form overall nd-by-nd covariance matrix Ω\n", "Ω = zeros(n * d, n * d)\n", "for i = 1:m\n", " Ω += kron(Σ[i], V[i])\n", "end\n", - "Ωchol = cholfact(Ω)\n", + "Ωchol = cholesky(Ω)\n", "# n-by-d responses\n", - "Y = X * B + reshape(Ωchol[:L] * randn(n*d), n, d);" + "Y = X * B + reshape(Ωchol.L * randn(n*d), n, d);" ] }, { @@ -119,21 +116,16 @@ }, { "cell_type": "code", - "execution_count": 39, - "metadata": { - "collapsed": false - }, + "execution_count": 4, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "3-element Array{Symbol,1}:\n", - " :Y\n", - " :X\n", - " :V" + "(:Y, :X, :V)" ] }, - "execution_count": 39, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -141,7 +133,7 @@ "source": [ "using VarianceComponentModels\n", "vcdata = VarianceComponentVariate(Y, X, V)\n", - "fieldnames(vcdata)" + "fieldnames(typeof(vcdata))" ] }, { @@ -165,48 +157,37 @@ }, { "cell_type": "code", - "execution_count": 40, - "metadata": { - "collapsed": false - }, + "execution_count": 5, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "7-element Array{Symbol,1}:\n", - " :B \n", - " :Σ \n", - " :A \n", - " :sense\n", - " :b \n", - " :lb \n", - " :ub " + "(:B, :Σ, :A, :sense, :b, :lb, :ub)" ] }, - "execution_count": 40, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vcmodel = VarianceComponentModel(vcdata)\n", - "fieldnames(vcmodel)" + "fieldnames(typeof(vcmodel))" ] }, { "cell_type": "code", - "execution_count": 41, - "metadata": { - "collapsed": false - }, + "execution_count": 6, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf)" ] }, - "execution_count": 41, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -244,10 +225,8 @@ }, { "cell_type": "code", - "execution_count": 42, - "metadata": { - "collapsed": false - }, + "execution_count": 8, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -258,6 +237,13 @@ " Iter Objective \n", "-------- -------------\n", " 0 -6.253551e+03\n", + "\n", + "******************************************************************************\n", + "This program contains Ipopt, a library for large-scale nonlinear optimization.\n", + " Ipopt is released as open source code under the Eclipse Public License (EPL).\n", + " For more information visit http://projects.coin-or.org/Ipopt\n", + "******************************************************************************\n", + "\n", " 1 -3.881454e+03\n", " 2 -3.853179e+03\n", " 3 -3.846525e+03\n", @@ -269,7 +255,7 @@ " 9 -3.844374e+03\n", " 10 -3.844373e+03\n", "\n", - " 0.290970 seconds (10.45 k allocations: 24.036 MiB, 4.73% gc time)\n" + " 5.031460 seconds (11.29 M allocations: 568.015 MiB, 4.78% gc time)\n" ] } ], @@ -289,18 +275,16 @@ }, { "cell_type": "code", - "execution_count": 43, - "metadata": { - "collapsed": false - }, + "execution_count": 9, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "-3844.3731814180805" + "-3844.3731814180887" ] }, - "execution_count": 43, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } @@ -318,47 +302,36 @@ }, { "cell_type": "code", - "execution_count": 44, - "metadata": { - "collapsed": false - }, + "execution_count": 10, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "7-element Array{Symbol,1}:\n", - " :B \n", - " :Σ \n", - " :A \n", - " :sense\n", - " :b \n", - " :lb \n", - " :ub " + "(:B, :Σ, :A, :sense, :b, :lb, :ub)" ] }, - "execution_count": 44, + "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "fieldnames(vcmodel_mle)" + "fieldnames(typeof(vcmodel_mle))" ] }, { "cell_type": "code", - "execution_count": 45, - "metadata": { - "collapsed": false - }, + "execution_count": 11, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955346 1.01632], ([0.380637 -0.305465; -0.305465 4.51938], [1.84009 0.265569; 0.265569 2.17275]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955346 1.01632], ([0.380637 -0.305465; -0.305465 4.51938], [1.84009 0.265569; 0.265569 2.17275]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf)" ] }, - "execution_count": 45, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -376,10 +349,8 @@ }, { "cell_type": "code", - "execution_count": 46, - "metadata": { - "collapsed": false - }, + "execution_count": 12, + "metadata": {}, "outputs": [ { "data": { @@ -387,7 +358,7 @@ "([0.0765136 0.263047; 0.263047 0.904332], [0.0844292 0.0917441; 0.0917441 0.0996927])" ] }, - "execution_count": 46, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -405,10 +376,8 @@ }, { "cell_type": "code", - "execution_count": 47, - "metadata": { - "collapsed": false - }, + "execution_count": 13, + "metadata": {}, "outputs": [ { "data": { @@ -424,7 +393,7 @@ " -1.557e-7 -1.27444e-6 -1.27444e-6 0.00121477 0.00993864 " ] }, - "execution_count": 47, + "execution_count": 13, "metadata": {}, "output_type": "execute_result" } @@ -442,20 +411,18 @@ }, { "cell_type": "code", - "execution_count": 48, - "metadata": { - "collapsed": false - }, + "execution_count": 14, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2×2 Array{Float64,2}:\n", - " 0.042559 0.0487086\n", - " 0.0430588 0.049178 " + " 0.0425562 0.0483834\n", + " 0.0430596 0.0492809" ] }, - "execution_count": 48, + "execution_count": 14, "metadata": {}, "output_type": "execute_result" } @@ -473,22 +440,20 @@ }, { "cell_type": "code", - "execution_count": 49, - "metadata": { - "collapsed": false - }, + "execution_count": 15, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4×4 Array{Float64,2}:\n", - " 0.00181127 -1.98035e-5 0.000240705 -2.59506e-6 \n", - " -1.98035e-5 0.00185406 -2.59506e-6 0.000247285\n", - " 0.000240705 -2.59506e-6 0.00237252 -2.63542e-5 \n", - " -2.59506e-6 0.000247285 -2.63542e-5 0.00241848 " + " 0.00181103 -1.96485e-5 0.000243441 -4.38252e-6 \n", + " -1.96485e-5 0.00185413 -4.38252e-6 0.000246407\n", + " 0.000243441 -4.38252e-6 0.00234096 -5.73331e-6 \n", + " -4.38252e-6 0.000246407 -5.73331e-6 0.00242861 " ] }, - "execution_count": 49, + "execution_count": 15, "metadata": {}, "output_type": "execute_result" } @@ -520,10 +485,8 @@ }, { "cell_type": "code", - "execution_count": 50, - "metadata": { - "collapsed": false - }, + "execution_count": 16, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -545,7 +508,7 @@ " 9 -3.846631e+03\n", " 10 -3.846630e+03\n", "\n", - " 0" + " 0.726373 seconds (388.90 k allocations: 82.673 MiB, 13.22% gc time)\n" ] } ], @@ -570,18 +533,16 @@ }, { "cell_type": "code", - "execution_count": 51, - "metadata": { - "collapsed": false - }, + "execution_count": 17, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "-3844.3777179025096" + "-3844.3777179025055" ] }, - "execution_count": 51, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } @@ -599,47 +560,36 @@ }, { "cell_type": "code", - "execution_count": 52, - "metadata": { - "collapsed": false - }, + "execution_count": 18, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "7-element Array{Symbol,1}:\n", - " :B \n", - " :Σ \n", - " :A \n", - " :sense\n", - " :b \n", - " :lb \n", - " :ub " + "(:B, :Σ, :A, :sense, :b, :lb, :ub)" ] }, - "execution_count": 52, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "fieldnames(vcmodel_reml)" + "fieldnames(typeof(vcmodel_reml))" ] }, { "cell_type": "code", - "execution_count": 53, - "metadata": { - "collapsed": false - }, + "execution_count": 19, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955345 1.01632], ([0.380594 -0.305485; -0.305485 4.51994], [1.84285 0.261963; 0.261963 2.17842]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955345 1.01632], ([0.380594 -0.305485; -0.305485 4.51994], [1.84285 0.261963; 0.261963 2.17842]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf)" ] }, - "execution_count": 53, + "execution_count": 19, "metadata": {}, "output_type": "execute_result" } @@ -652,15 +602,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "* standard errors of the estimated varianec component parameters" + "* standard errors of the estimated variance component parameters" ] }, { "cell_type": "code", - "execution_count": 54, - "metadata": { - "collapsed": false - }, + "execution_count": 20, + "metadata": {}, "outputs": [ { "data": { @@ -668,7 +616,7 @@ "([0.0765055 0.26305; 0.26305 0.904446], [0.0845559 0.0919325; 0.0919325 0.0999526])" ] }, - "execution_count": 54, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } @@ -686,10 +634,8 @@ }, { "cell_type": "code", - "execution_count": 55, - "metadata": { - "collapsed": false - }, + "execution_count": 21, + "metadata": {}, "outputs": [ { "data": { @@ -705,7 +651,7 @@ " -1.51499e-7 -1.26041e-6 -1.26041e-6 0.0012014 0.00999052 " ] }, - "execution_count": 55, + "execution_count": 21, "metadata": {}, "output_type": "execute_result" } @@ -723,20 +669,18 @@ }, { "cell_type": "code", - "execution_count": 56, - "metadata": { - "collapsed": false - }, + "execution_count": 22, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2×2 Array{Float64,2}:\n", - " 0.0425909 0.0487744\n", - " 0.043091 0.0492444" + " 0.0425881 0.0484485\n", + " 0.0430919 0.0493475" ] }, - "execution_count": 56, + "execution_count": 22, "metadata": {}, "output_type": "execute_result" } @@ -754,22 +698,20 @@ }, { "cell_type": "code", - "execution_count": 57, - "metadata": { - "collapsed": false - }, + "execution_count": 23, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4×4 Array{Float64,2}:\n", - " 0.00181398 -1.98331e-5 0.000237127 -2.55589e-6 \n", - " -1.98331e-5 0.00185683 -2.55589e-6 0.000243624\n", - " 0.000237127 -2.55589e-6 0.00237894 -2.6426e-5 \n", - " -2.55589e-6 0.000243624 -2.6426e-5 0.00242501 " + " 0.00181375 -1.96783e-5 0.000239868 -4.34611e-6 \n", + " -1.96783e-5 0.00185691 -4.34611e-6 0.000242745\n", + " 0.000239868 -4.34611e-6 0.00234726 -5.73082e-6 \n", + " -4.34611e-6 0.000242745 -5.73082e-6 0.00243518 " ] }, - "execution_count": 57, + "execution_count": 23, "metadata": {}, "output_type": "execute_result" } @@ -801,30 +743,23 @@ }, { "cell_type": "code", - "execution_count": 58, - "metadata": { - "collapsed": false - }, + "execution_count": 24, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "5-element Array{Symbol,1}:\n", - " :Yrot \n", - " :Xrot \n", - " :eigval \n", - " :eigvec \n", - " :logdetV2" + "(:Yrot, :Xrot, :eigval, :eigvec, :logdetV2)" ] }, - "execution_count": 58, + "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vcdatarot = TwoVarCompVariateRotate(vcdata)\n", - "fieldnames(vcdatarot)" + "fieldnames(typeof(vcdatarot))" ] }, { @@ -836,10 +771,8 @@ }, { "cell_type": "code", - "execution_count": 59, - "metadata": { - "collapsed": false - }, + "execution_count": 25, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -861,7 +794,7 @@ " 9 -3.844374e+03\n", " 10 -3.844373e+03\n", "\n", - " 0.018754 seconds (9.15 k allocations: 680.172 KiB)\n" + " 0.055578 seconds (21.91 k allocations: 1.394 MiB)\n" ] } ], @@ -872,10 +805,8 @@ }, { "cell_type": "code", - "execution_count": 60, - "metadata": { - "collapsed": false - }, + "execution_count": 26, + "metadata": {}, "outputs": [ { "data": { @@ -885,7 +816,7 @@ " 0.955346 1.01632" ] }, - "execution_count": 60, + "execution_count": 26, "metadata": {}, "output_type": "execute_result" } @@ -897,10 +828,8 @@ }, { "cell_type": "code", - "execution_count": 61, - "metadata": { - "collapsed": false - }, + "execution_count": 27, + "metadata": {}, "outputs": [ { "data": { @@ -908,7 +837,7 @@ "([0.380637 -0.305465; -0.305465 4.51938], [1.84009 0.265569; 0.265569 2.17275])" ] }, - "execution_count": 61, + "execution_count": 27, "metadata": {}, "output_type": "execute_result" } @@ -927,16 +856,14 @@ }, { "cell_type": "code", - "execution_count": 62, - "metadata": { - "collapsed": false - }, + "execution_count": 28, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "This is Ipopt version 3.12.4, running with linear solver mumps.\n", + "This is Ipopt version 3.12.10, running with linear solver mumps.\n", "NOTE: Other linear solvers might be more efficient (see Ipopt documentation).\n", "\n", "Number of nonzeros in equality constraint Jacobian...: 0\n", @@ -967,16 +894,16 @@ "iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls\n", " 50 3.8443732e+03 0.00e+00 6.76e-06 -11.0 4.08e-07 - 1.00e+00 1.00e+00f 1 MaxSA\n", " 55 3.8443732e+03 0.00e+00 1.83e-06 -11.0 1.11e-07 - 1.00e+00 1.00e+00f 1 MaxSA\n", - " 60 3.8443732e+03 0.00e+00 4.97e-07 -11.0 3.00e-08 - 1.00e+00 1.00e+00f 1 MaxSA\n", + " 60 3.8443732e+03 0.00e+00 4.97e-07 -11.0 3.00e-08 - 1.00e+00 1.00e+00h 1 MaxSA\n", "\n", "Number of Iterations....: 63\n", "\n", " (scaled) (unscaled)\n", - "Objective...............: 3.4496886481728075e+02 3.8443731733053696e+03\n", - "Dual infeasibility......: 2.2693631692678575e-07 2.5290047242499938e-06\n", + "Objective...............: 3.4496886481728791e+02 3.8443731733053728e+03\n", + "Dual infeasibility......: 2.2693631660531264e-07 2.5290047206674095e-06\n", "Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00\n", "Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00\n", - "Overall NLP error.......: 2.2693631692678575e-07 2.5290047242499938e-06\n", + "Overall NLP error.......: 2.2693631660531264e-07 2.5290047206674095e-06\n", "\n", "\n", "Number of objective function evaluations = 64\n", @@ -986,11 +913,11 @@ "Number of equality constraint Jacobian evaluations = 0\n", "Number of inequality constraint Jacobian evaluations = 0\n", "Number of Lagrangian Hessian evaluations = 63\n", - "Total CPU secs in IPOPT (w/o function evaluations) = 0.020\n", - "Total CPU secs in NLP function evaluations = 0.256\n", + "Total CPU secs in IPOPT (w/o function evaluations) = 1.739\n", + "Total CPU secs in NLP function evaluations = 0.293\n", "\n", "EXIT: Solved To Acceptable Level.\n", - " " + " 2.745554 seconds (4.30 M allocations: 210.935 MiB, 2.63% gc time)\n" ] } ], @@ -1002,10 +929,8 @@ }, { "cell_type": "code", - "execution_count": 63, - "metadata": { - "collapsed": false - }, + "execution_count": 29, + "metadata": {}, "outputs": [ { "data": { @@ -1015,7 +940,7 @@ " 0.955346 1.01632" ] }, - "execution_count": 63, + "execution_count": 29, "metadata": {}, "output_type": "execute_result" } @@ -1027,10 +952,8 @@ }, { "cell_type": "code", - "execution_count": 64, - "metadata": { - "collapsed": false - }, + "execution_count": 30, + "metadata": {}, "outputs": [ { "data": { @@ -1038,7 +961,7 @@ "([0.380552 -0.305594; -0.305594 4.52106], [1.84008 0.265385; 0.265385 2.17287])" ] }, - "execution_count": 64, + "execution_count": 30, "metadata": {}, "output_type": "execute_result" } @@ -1072,7 +995,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Starting point" + "## Starting point\n" ] }, { @@ -1093,11 +1016,10 @@ }, { "cell_type": "markdown", - "metadata": { - "collapsed": true - }, + "metadata": {}, "source": [ - "## Constrained estimation of `B`" + "## Constrained estimation of `B`\n", + "\n" ] }, { @@ -1109,18 +1031,16 @@ }, { "cell_type": "code", - "execution_count": 65, - "metadata": { - "collapsed": false - }, + "execution_count": 31, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), [1.0 0.0 -1.0 0.0], '=', 0.0, 0.0, 2.0)" + "VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), [1.0 0.0 -1.0 0.0], '=', 0.0, 0.0, 2.0)" ] }, - "execution_count": 65, + "execution_count": 31, "metadata": {}, "output_type": "execute_result" } @@ -1145,10 +1065,8 @@ }, { "cell_type": "code", - "execution_count": 66, - "metadata": { - "collapsed": false - }, + "execution_count": 32, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -1170,7 +1088,7 @@ " 9 -3.844650e+03\n", " 10 -3.844650e+03\n", "\n", - " 0.031954 seconds (10.70 k allocations: 781.828 KiB)\n" + " 0.185885 seconds (179.51 k allocations: 9.295 MiB)\n" ] } ], @@ -1181,39 +1099,28 @@ }, { "cell_type": "code", - "execution_count": 67, - "metadata": { - "collapsed": false - }, + "execution_count": 33, + "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "7-element Array{Symbol,1}:\n", - " :B \n", - " :Σ \n", - " :A \n", - " :sense\n", - " :b \n", - " :lb \n", - " :ub " + "(:B, :Σ, :A, :sense, :b, :lb, :ub)" ] }, - "execution_count": 67, + "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "fieldnames(vcmodel_constr)" + "fieldnames(typeof(vcmodel_constr))" ] }, { "cell_type": "code", - "execution_count": 68, - "metadata": { - "collapsed": false - }, + "execution_count": 34, + "metadata": {}, "outputs": [ { "data": { @@ -1223,7 +1130,7 @@ " 0.955683 1.01591" ] }, - "execution_count": 68, + "execution_count": 34, "metadata": {}, "output_type": "execute_result" } @@ -1234,10 +1141,8 @@ }, { "cell_type": "code", - "execution_count": 69, - "metadata": { - "collapsed": false - }, + "execution_count": 35, + "metadata": {}, "outputs": [ { "data": { @@ -1245,7 +1150,7 @@ "([0.380624 -0.305498; -0.305498 4.51948], [1.84051 0.265065; 0.265065 2.17336])" ] }, - "execution_count": 69, + "execution_count": 35, "metadata": {}, "output_type": "execute_result" } @@ -1263,16 +1168,14 @@ }, { "cell_type": "code", - "execution_count": 70, - "metadata": { - "collapsed": false - }, + "execution_count": 36, + "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "This is Ipopt version 3.12.4, running with linear solver mumps.\n", + "This is Ipopt version 3.12.10, running with linear solver mumps.\n", "NOTE: Other linear solvers might be more efficient (see Ipopt documentation).\n", "\n", "Number of nonzeros in equality constraint Jacobian...: 0\n", @@ -1308,11 +1211,11 @@ "Number of Iterations....: 63\n", "\n", " (scaled) (unscaled)\n", - "Objective...............: 3.4484507551949008e+02 3.8446498170293403e+03\n", - "Dual infeasibility......: 2.2694405349430929e-07 2.5301808715939735e-06\n", + "Objective...............: 3.4484507551949685e+02 3.8446498170293398e+03\n", + "Dual infeasibility......: 2.2694405475622814e-07 2.5301808856629548e-06\n", "Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00\n", "Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00\n", - "Overall NLP error.......: 2.2694405349430929e-07 2.5301808715939735e-06\n", + "Overall NLP error.......: 2.2694405475622814e-07 2.5301808856629548e-06\n", "\n", "\n", "Number of objective function evaluations = 64\n", @@ -1322,11 +1225,11 @@ "Number of equality constraint Jacobian evaluations = 0\n", "Number of inequality constraint Jacobian evaluations = 0\n", "Number of Lagrangian Hessian evaluations = 63\n", - "Total CPU secs in IPOPT (w/o function evaluations) = 0.016\n", - "Total CPU secs in NLP function evaluations = 0.417\n", + "Total CPU secs in IPOPT (w/o function evaluations) = 0.028\n", + "Total CPU secs in NLP function evaluations = 0.634\n", "\n", "EXIT: Solved To Acceptable Level.\n", - " " + " 0.760983 seconds (102.63 k allocations: 8.135 MiB)\n" ] } ], @@ -1344,10 +1247,8 @@ }, { "cell_type": "code", - "execution_count": 71, - "metadata": { - "collapsed": false - }, + "execution_count": 37, + "metadata": {}, "outputs": [ { "data": { @@ -1357,7 +1258,7 @@ " 0.955683 1.01591" ] }, - "execution_count": 71, + "execution_count": 37, "metadata": {}, "output_type": "execute_result" } @@ -1368,10 +1269,8 @@ }, { "cell_type": "code", - "execution_count": 72, - "metadata": { - "collapsed": false - }, + "execution_count": 38, + "metadata": {}, "outputs": [ { "data": { @@ -1379,7 +1278,7 @@ "([0.380539 -0.305626; -0.305626 4.52116], [1.8405 0.264881; 0.264881 2.17348])" ] }, - "execution_count": 72, + "execution_count": 38, "metadata": {}, "output_type": "execute_result" } @@ -1390,19 +1289,18 @@ } ], "metadata": { - "anaconda-cloud": {}, "kernelspec": { - "display_name": "Julia 0.6.0", + "display_name": "Julia 1.1.0", "language": "julia", - "name": "julia-0.6" + "name": "julia-1.1" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", - "version": "0.6.0" + "version": "1.1.0" } }, "nbformat": 4, - "nbformat_minor": 0 + "nbformat_minor": 2 } diff --git a/docs/src/index.md b/docs/src/index.md index 3739daa..3901bfe 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -28,11 +28,16 @@ and **parameters** are ## Installation -Use the Julia package manager to install VarianceComponentModels.jl. +This package requires Julia v0.7.0 or later, which can be obtained from +https://julialang.org/downloads/ or by building Julia from the sources in the +https://github.com/JuliaLang/julia repository. + +The package has not yet been registered and must be installed using the repository location. +Start julia and use the `]` key to switch to the package manager REPL ```julia -Pkg.clone("https://github.com/OpenMendel/VarianceComponentModels.jl.git") +(v0.7) pkg> add https://github.com/OpenMendel/VarianceComponentModels.jl.git#Julia-0.7 ``` -This package supports Julia `0.6`. +Use the backspace key to return to the Julia REPL. ## Manual Outline diff --git a/docs/src/man/fig1_heritability.png b/docs/src/man/fig1_heritability.png new file mode 100644 index 0000000..9c74baa Binary files /dev/null and b/docs/src/man/fig1_heritability.png differ diff --git a/docs/src/man/fig2_heritability.png b/docs/src/man/fig2_heritability.png new file mode 100644 index 0000000..bd78c58 Binary files /dev/null and b/docs/src/man/fig2_heritability.png differ diff --git a/docs/src/man/heritability.md b/docs/src/man/heritability.md index bf6842d..dd42f40 100644 --- a/docs/src/man/heritability.md +++ b/docs/src/man/heritability.md @@ -1,6 +1,7 @@ # Heritability Analysis + As an application of the variance component model, this note demonstrates the workflow for heritability analysis in genetics, using a sample data set `cg10k` with **6,670** individuals and **630,860** SNPs. Person IDs and phenotype names are masked for privacy. `cg10k.bed`, `cg10k.bim`, and `cg10k.fam` is a set of Plink files in binary format. `cg10k_traits.txt` contains 13 phenotypes of the 6,670 individuals. @@ -21,25 +22,23 @@ Machine information: versioninfo() ``` - Julia Version 0.6.0 - Commit 903644385b (2017-06-19 13:05 UTC) + Julia Version 1.1.0 + Commit 80516ca202 (2019-01-21 21:24 UTC) Platform Info: - OS: macOS (x86_64-apple-darwin13.4.0) - CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz + OS: macOS (x86_64-apple-darwin14.5.0) + CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz WORD_SIZE: 64 - BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell) - LAPACK: libopenblas64_ LIBM: libopenlibm - LLVM: libLLVM-3.9.1 (ORCJIT, haswell) + LLVM: libLLVM-6.0.1 (ORCJIT, skylake) ## Read in binary SNP data -We will use the [`SnpArrays.jl`](https://github.com/OpenMendel/SnpArrays.jl) package to read in binary SNP data and compute the empirical kinship matrix. Issue +We will use the [`SnpArrays.jl`](https://github.com/OpenMendel/SnpArrays.jl) package to read in binary SNP data and compute the empirical kinship matrix. The package has not yet been registered and must be installed using the repository location. Start julia and use the `]` key to switch to the package manager REPL ```julia -Pkg.clone("https://github.com/OpenMendel/SnpArrays.jl.git") +(v0.7) pkg> add https://github.com/OpenMendel/SnpArrays.jl.git#juliav0.7 ``` -within `Julia` to install the `SnpArrays` package. +Use the backspace key to return to the Julia REPL. ```julia @@ -49,42 +48,42 @@ using SnpArrays ```julia # read in genotype data from Plink binary file (~50 secs on my laptop) -@time cg10k = SnpArray("cg10k") +@time cg10k = SnpArray("cg10k.bed") ``` - 22.902730 seconds (51.62 k allocations: 1005.845 MiB, 0.11% gc time) + 0.030779 seconds (54 allocations: 19.467 MiB, 28.08% gc time) - 6670×630860 SnpArrays.SnpArray{2}: - (false, true) (false, true) … (true, true) (true, true) - (true, true) (true, true) (false, true) (true, false) - (true, true) (true, true) (true, true) (true, true) - (true, true) (true, true) (false, true) (true, true) - (true, true) (true, true) (true, true) (false, true) - (false, true) (false, true) … (true, true) (true, true) - (false, false) (false, false) (true, true) (true, true) - (true, true) (true, true) (true, true) (false, true) - (true, true) (true, true) (true, true) (true, true) - (true, true) (true, true) (false, true) (true, true) - (true, true) (true, true) … (true, true) (true, true) - (false, true) (false, true) (true, true) (false, true) - (true, true) (true, true) (true, true) (false, true) - ⋮ ⋱ - (false, true) (false, true) (false, true) (false, true) - (false, true) (false, true) (false, true) (true, true) - (true, true) (true, true) … (false, true) (true, true) - (false, true) (false, true) (true, true) (false, true) - (true, true) (true, true) (false, true) (true, true) - (true, true) (true, true) (false, false) (false, true) - (true, true) (true, true) (true, true) (false, true) - (true, true) (true, true) … (true, true) (true, true) - (true, true) (true, true) (false, true) (true, true) - (true, true) (true, true) (true, true) (false, true) - (false, true) (false, true) (true, true) (true, true) - (true, true) (true, true) (true, true) (true, true) + 6670×630860 SnpArray: + 0x02 0x02 0x03 0x03 0x02 0x03 … 0x02 0x03 0x03 0x03 0x03 0x03 + 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x02 0x02 0x03 0x02 0x01 + 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x03 0x03 0x03 0x03 0x03 + 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02 0x02 0x02 0x02 0x03 + 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02 + 0x02 0x02 0x03 0x03 0x02 0x03 … 0x03 0x03 0x03 0x03 0x03 0x03 + 0x00 0x00 0x03 0x03 0x00 0x03 0x03 0x02 0x03 0x03 0x03 0x03 + 0x03 0x03 0x03 0x03 0x03 0x02 0x03 0x03 0x03 0x03 0x03 0x02 + 0x03 0x03 0x02 0x03 0x02 0x03 0x02 0x02 0x03 0x03 0x03 0x03 + 0x03 0x03 0x03 0x00 0x03 0x03 0x02 0x02 0x03 0x03 0x02 0x03 + 0x03 0x03 0x02 0x03 0x02 0x02 … 0x02 0x03 0x03 0x03 0x03 0x03 + 0x02 0x02 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x03 0x03 0x02 + 0x03 0x03 0x03 0x00 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02 + ⋮ ⋮ ⋱ ⋮ + 0x02 0x02 0x03 0x03 0x02 0x03 0x03 0x02 0x03 0x03 0x02 0x02 + 0x02 0x02 0x02 0x03 0x00 0x02 0x02 0x03 0x03 0x03 0x02 0x03 + 0x03 0x03 0x02 0x02 0x02 0x03 … 0x03 0x02 0x03 0x03 0x02 0x03 + 0x02 0x02 0x03 0x03 0x02 0x03 0x03 0x02 0x03 0x03 0x03 0x02 + 0x03 0x03 0x03 0x02 0x03 0x02 0x03 0x03 0x03 0x03 0x02 0x03 + 0x03 0x03 0x01 0x02 0x02 0x01 0x03 0x00 0x00 0x02 0x00 0x02 + 0x03 0x03 0x03 0x02 0x03 0x03 0x03 0x03 0x03 0x03 0x03 0x02 + 0x03 0x03 0x02 0x02 0x02 0x03 … 0x00 0x03 0x03 0x03 0x03 0x03 + 0x03 0x03 0x03 0x02 0x02 0x03 0x02 0x02 0x02 0x02 0x02 0x03 + 0x03 0x03 0x03 0x02 0x03 0x02 0x02 0x03 0x02 0x03 0x03 0x02 + 0x02 0x02 0x03 0x03 0x02 0x03 0x03 0x03 0x01 0x03 0x03 0x03 + 0x03 0x03 0x03 0x03 0x03 0x02 0x03 0x00 0x03 0x03 0x03 0x03 @@ -102,18 +101,105 @@ people, snps = size(cg10k) +The positions of the missing data are evaluated by + ```julia -# summary statistics (~50 secs on my laptop) -@time maf, _, missings_by_snp, = summarize(cg10k); +mp = missingpos(cg10k) ``` - 24 + + + + 6670×630860 SparseArrays.SparseMatrixCSC{Bool,Int32} with 5524131 stored entries: + [5688 , 1] = true + [6445 , 1] = true + [109 , 3] = true + [189 , 3] = true + [216 , 3] = true + [254 , 3] = true + [331 , 3] = true + [522 , 3] = true + [525 , 3] = true + [597 , 3] = true + [619 , 3] = true + [672 , 3] = true + ⋮ + [4929 , 630860] = true + [5594 , 630860] = true + [5650 , 630860] = true + [5780 , 630860] = true + [5854 , 630860] = true + [5867 , 630860] = true + [6084 , 630860] = true + [6175 , 630860] = true + [6178 , 630860] = true + [6239 , 630860] = true + [6478 , 630860] = true + [6511 , 630860] = true + + + +The number of missing data values in each column can be evaluated as + + +```julia +missings_by_snp = sum(mp, dims=1) +``` + + + + + 1×630860 Array{Int64,2}: + 2 0 132 77 0 27 2 2 6 27 2 … 6 4 5 11 0 0 4 29 0 5 43 + + + +Minor allele frequencies (MAF) for each SNP. + + +```julia +maf_cg10k = maf(cg10k) +``` + + + + + 630860-element Array{Float64,1}: + 0.1699160167966407 + 0.17098950524737633 + 0.11402569593147749 + 0.2686940694676172 + 0.21926536731634183 + 0.23934969140448592 + 0.19061187762447507 + 0.20200959808038388 + 0.027160864345738278 + 0.2997139846454915 + 0.24625074985003004 + 0.05555555555555558 + 0.3659067046647667 + ⋮ + 0.22547254725472543 + 0.4035864345738295 + 0.20799579957995795 + 0.44801200300075017 + 0.2954647845021775 + 0.14265367316341826 + 0.1709145427286357 + 0.2814281428142814 + 0.06113537117903933 + 0.052473763118440764 + 0.13930982745686427 + 0.1324128564961521 + + ```julia # 5 number summary and average MAF (minor allele frequencies) -quantile(maf, [0.0 .25 .5 .75 1.0]), mean(maf) +using Statistics +Statistics.quantile(maf_cg10k, [0.0 .25 .5 .75 1.0]), mean(maf_cg10k) ``` @@ -125,18 +211,19 @@ quantile(maf, [0.0 .25 .5 .75 1.0]), mean(maf) ```julia -# Pkg.add("Plots") -# Pkg.add("PyPlot") -using Plots -pyplot() +#using Pkg +#pkg "add Plots" +#pkg"add PyPlot" -histogram(maf, xlab = "Minor Allele Frequency (MAF)", label = "MAF") +using Plots +gr(size=(600,500), html_output_format=:png) +histogram(maf_cg10k, xlab = "Minor Allele Frequency (MAF)", label = "MAF") ``` - +![png](fig1_heritability.png) @@ -156,7 +243,7 @@ sum(missings_by_snp) / length(cg10k) ```julia # proportion of rare SNPs with maf < 0.05 -countnz(maf .< 0.05) / length(maf) +count(!iszero, maf_cg10k .< 0.05) / length(maf_cg10k) ``` @@ -172,44 +259,45 @@ We estimate empirical kinship based on all SNPs by the genetic relation matrix ( ```julia -# GRM using SNPs with maf > 0.01 (default) (~10 mins on my laptop) -srand(123) +## GRM using SNPs with maf > 0.01 (default) (~10 mins on my laptop) +using Random +Random.seed!(123) @time Φgrm = grm(cg10k; method = :GRM) ``` - 396.943890 seconds (8.43 G allocations: 127.378 GiB, 4.38% gc time) + 568.176660 seconds (2.91 M allocations: 494.981 MiB, 0.40% gc time) 6670×6670 Array{Float64,2}: - 0.503024 0.00335505 -0.000120075 … -5.45185e-5 -0.00278072 - 0.00335505 0.498958 -0.00195952 0.000868471 0.0034285 - -0.000120075 -0.00195952 0.493828 0.000174648 -0.000381467 - 0.000923828 -0.00329169 -0.00194166 -0.00223595 -0.00123508 - -8.39649e-5 -0.00353358 0.0018709 0.00222858 -0.00171176 - 0.00204208 0.000572952 0.00254025 … 0.000861385 2.99785e-5 - 0.000569323 0.0024786 -0.00185743 0.00117649 -0.00118027 - -0.000642144 0.00317992 -0.00099777 0.00354182 -0.000260645 - -0.00102913 -0.00123475 -0.00061138 0.00173885 0.00177727 - -0.00139442 0.00208423 0.000124525 -0.00145156 -0.001011 - -0.00204555 0.00011055 -0.000419398 … -0.000198235 -0.00110353 - 0.000947587 0.00167346 0.00184451 -0.000690143 -0.00304087 - 0.000322759 -0.000899805 0.00303981 0.000739331 -0.00118835 + 0.502735 0.00328112 -6.79435e-5 … -6.09591e-5 -0.00277687 + 0.00328112 0.49807 -0.00195548 0.000884994 0.00341591 + -6.79435e-5 -0.00195548 0.492348 0.000198191 -0.000337529 + 0.00087876 -0.00322605 -0.00192088 -0.00235314 -0.00124267 + -5.03227e-5 -0.00352498 0.00184588 0.00219109 -0.00163189 + 0.00203199 0.000597697 0.00251237 … 0.00088428 2.21226e-5 + 0.000560862 0.00244517 -0.00183233 0.00120214 -0.00120686 + -0.000656949 0.00322249 -0.00101472 0.00355832 -0.000240444 + -0.00103874 -0.00125164 -0.000599731 0.00176492 0.00176928 + -0.00137058 0.00209596 0.000146711 -0.0014453 -0.00103066 + -0.00209312 0.000140721 -0.000442031 … -0.000214963 -0.00107391 + 0.000933587 0.00168842 0.00185731 -0.000787908 -0.00311063 + 0.000334572 -0.000886623 0.00304182 0.000752285 -0.00123334 ⋮ ⋱ - 0.00298012 0.00130003 0.000998861 4.18454e-6 0.00303991 - -0.00207748 0.00274717 -0.00191741 -0.00107073 0.00368267 - 0.000545569 -0.00244439 -0.00299578 … -0.000669885 0.00221027 - -0.00423186 -0.00208514 -0.00108833 -0.000622127 -0.000567483 - -0.00325644 -0.000781353 0.0030423 0.000501423 -0.00010267 - 0.00041055 -0.00200772 0.00274867 -0.00624933 -0.00521365 - 0.00210519 0.000879889 -0.00107817 -0.000797878 -0.000557352 - -0.00230058 -0.000119132 0.000116817 … 0.000867087 -0.00233512 - -0.0020119 0.00230772 -0.00128837 0.00194798 -0.00048733 - -0.000944942 -0.000928073 -0.000175096 0.00126911 -0.00303766 - -5.45185e-5 0.000868471 0.000174648 0.500829 0.000469478 - -0.00278072 0.0034285 -0.000381467 0.000469478 0.500627 + 0.00298109 0.00121739 0.00102822 9.66935e-6 0.00306953 + -0.00209728 0.00271452 -0.00182325 -0.00108139 0.00366901 + 0.000549425 -0.00244419 -0.00301368 … -0.000631961 0.00215641 + -0.00423362 -0.00208073 -0.00107904 -0.000619315 -0.000593852 + -0.00326697 -0.000769552 0.00310511 0.000520658 -0.000113441 + 0.000430563 -0.0020236 0.00265425 -0.00635493 -0.00520252 + 0.00218746 0.000798767 -0.00105684 -0.000918245 -0.00061484 + -0.00230525 -0.000101149 0.000117936 … 0.000879829 -0.00233479 + -0.00201305 0.00233864 -0.00134496 0.00197044 -0.000486275 + -0.000990534 -0.000924159 -9.12302e-5 0.00122311 -0.00298296 + -6.09591e-5 0.000884994 0.000198191 0.499289 0.000481492 + -0.00277687 0.00341591 -0.000337529 0.000481492 0.499799 @@ -219,17 +307,14 @@ Read in the phenotype data and compute descriptive statistics. ```julia -# Pkg.add("DataFrames") -using DataFrames - -cg10k_trait = readtable( - "cg10k_traits.txt"; - separator = ' ', - names = [:FID; :IID; :Trait1; :Trait2; :Trait3; :Trait4; :Trait5; :Trait6; - :Trait7; :Trait8; :Trait9; :Trait10; :Trait11; :Trait12; :Trait13], - eltypes = [String; String; Float64; Float64; Float64; Float64; Float64; - Float64; Float64; Float64; Float64; Float64; Float64; Float64; Float64] - ) +#using Pkg +#pkg"add CSV DataFrames" +#using CSV, DataFrames + +cg10k_trait = CSV.File("cg10k_traits.txt"; + delim = ' ') |> DataFrame +names!(cg10k_trait, [:FID; :IID; :Trait1; :Trait2; :Trait3; :Trait4; :Trait5; :Trait6; + :Trait7; :Trait8; :Trait9; :Trait10; :Trait11; :Trait12; :Trait13]) # do not display FID and IID for privacy cg10k_trait[:, 3:end] ``` @@ -237,7 +322,7 @@ cg10k_trait[:, 3:end] -
Trait1Trait2Trait3Trait4Trait5Trait6Trait7Trait8Trait9Trait10Trait11Trait12Trait13
1-1.81573145026234-0.946150461472831.11363077580442-2.098671211191590.7444166141117480.001391718840801310.934732480409667-1.226773154181031.1160784277875-0.44362803350290.824465656443384-1.02852542216546-0.394049201727681
2-1.244400943787290.1096599925471790.467119394241789-1.621313040975891.05667583556830.9789469794191811.000146339460470.324874271402281.162321752196962.69227069487053.082636724610471.090649547860130.0256616415357438
31.455669145023051.538669329232431.094029593765550.586655272226893-0.32796454430367-0.30337709778827-0.0334354881314741-0.464463064285437-0.3319396273436-0.486839089635991-1.10648681564373-1.42015780427231-0.687463456644413
4-0.7688092766985480.5134908855142490.244263028382142-1.317402544756911.193937743268451.173441277342881.087374266752320.5360225837322610.8027592407620680.2341594117498150.394174866891074-0.7673658924760290.0635385761884935
5-0.264415132547719-0.348240421825694-0.02390650834136060.004739158022449481.256191917121931.20388836676311.298007390426270.3101136602473110.6261598610593520.8992891298312240.549967833508120.5406878095420480.179675416046033
6-1.37617270917293-1.471919677445640.291179894254146-0.803110740704731-0.264239977442213-0.260573027836772-0.165372266287781-0.2192572941183621.04702422290318-0.09858155346164820.9473934380684480.5940148120314380.245407436348479
70.1009416296374-0.191615722103455-0.5674213215966770.378571487240382-0.246656179817904-0.6088107500538580.189081058215596-1.27077787326519-0.4524761991439650.7025628772977240.3326362189571790.00269165036261810.317117176705358
8-0.3198182763674641.357744806572830.818689545938528-1.155655316443520.634483681022590.2914619086346790.933323714954726-0.7410832896824920.647477683507572-0.9708776270779660.2208611654113040.852512250237764-0.225904624283945
9-0.2883341733420320.5660825380907520.254958336116175-0.6525783028697140.6689215592773470.9783091991705580.1228629660419381.47909263782140.06721324241734490.07959039175278270.1675324552432320.2469155794421390.539932616458363
10-1.15759732583991-0.781198583545165-0.595807759833517-1.005549802604020.7898288859333210.5710584133790440.951304176233755-0.2959629829848160.990420024797070.5613093669889830.733100030623233-1.73467772245684-1.35278484330654
110.7405691504590311.408738467554150.7346899994400880.0208322841295094-0.337440968561619-0.458304040611395-0.142582512772326-0.580392297464107-0.684684998101516-0.00785381461893456-0.712244337518008-0.313345561230878-0.345419463162219
12-0.6758924864549950.2798926138296820.267915996308248-1.041036653929850.9107417156458880.8660276185131711.074144317020050.03817510035383020.766355377018601-0.340118016143495-0.8090139585050590.548521663785885-0.0201828675962336
13-0.795410435603455-0.6999899397627380.3991295030063-0.5104762619007361.515522454168441.287430329394671.537723932509030.1339891601177021.020257368860370.499018733899186-0.36948273277931-1.10153460436318-0.598132438886619
14-0.193483122930324-0.286021160323518-0.6914942252629950.01315816787006991.523374706867821.40106380722621.531146204518960.3330664834780751.043724803810990.163206783570466-0.422883765001728-0.383527976713573-0.489221907788158
150.1512462033797182.091851089936142.03800472474384-1.124747171435311.665570243907131.625356751095761.587510704836550.6358521860437760.8425777846059790.450761870778952-1.39479033623028-0.5609841075677680.289349776549287
16-0.4646087408127120.361276947723031.2327673928287-0.8260337310863831.434752247099831.744518238188460.2110968874846382.648164251405481.025114331460960.119757316031840.0596832073448267-0.631231612661616-0.207878671782927
17-0.732977488012215-0.5262234258897790.61657871336593-0.554479743325930.9474848590251040.9368332141381730.9725168063355240.2902510138652271.012853597257230.516207422283291-0.03006891719881940.87873225245830.450254629309513
18-0.1673264596221190.1753271654872370.287467725892572-0.4026525320842460.5511815094180560.5222047432909750.4368376600946530.2995649338455790.583109520896067-0.704415820005353-0.730810367994577-1.95140580379896-0.933504665700164
191.411594857874181.787224079010170.843976395853640.481278083772991-0.0887673728508268-0.499577574268580.304195897924847-1.23884208383369-0.153475724036624-0.8704861027883290.0955473331150403-0.983708050882817-0.3563445644514
20-1.42997091652825-0.4901470450342130.272730237607695-1.610299929541530.9907878171977480.7116875326081841.1885836012715-0.3712291880756381.24703459239952-0.03891623322715160.8834957490728722.589880263210173.33539552370368
21-0.1472472881767650.123284304156520.617549051912237-0.187130771782620.2564381075866940.177949837350830.412611806463263-0.2448091245597370.09476248061364920.723017223849532-0.6839483546334360.0873751276309269-0.262209652750371
22-0.187112676773894-0.270777264595619-1.015568185516060.06028505686002330.2724197577579780.869133161879197-0.6575194614142342.32388522018189-0.9999360115250341.446718441783060.971157886040772-0.358747904241515-0.439657942096136
23-1.82434047163768-0.9334804460680671.29474003766977-1.945452211510360.335846511896540.3592016543028440.513652924365886-0.0731976966969581.571390428120051.533293713267281.820768218595282.227403018678291.50063347195857
24-2.29344084351335-2.491618423444180.40383988742336-2.364880747529481.41052548319561.422441171477921.170241662721720.844766501768551.790268754324950.648181858970515-0.0857231057403538-1.027895352926170.491288088952859
25-0.4341359328883050.7408819890346520.699576357578518-1.024055431877750.7595292239837130.9566561108952880.6332995686565890.7707339322685160.8249885117145261.842874376347691.91045942063443-0.5023172078693660.132670133448219
26-2.1920969546557-2.494656642722710.354854763893431-1.931558486357140.9419794002899380.9789171014141060.8948600972897360.4632394028318731.125371333171631.705284461919550.7177927144791230.6458880491082610.783968250169388
27-1.46602269088422-1.249216771018970.307977693653039-1.550973646609890.6189084944747980.6625081716620420.4759571739060780.4847186745977070.4015648920282490.55987973254026-0.376938143754217-0.9339826292282180.390013151672955
28-1.83317744236881-1.532687878287012.55674262685865-1.518277457838350.7894096017464550.9087477997285880.6499719229414790.6683736499316671.200583035199030.2779632560756371.25049531982753.313704450716382.22035828885342
29-0.7845466282431780.2765825795439313.01104958800057-1.119788432067580.9208238584227070.7502176898861511.26153730009639-0.4033638829224170.400667296857811-0.217597941303479-0.724669537565068-0.391945338467193-0.650023936358253
300.4644559163451351.3326356122229-1.23059563374303-0.3579759589374141.182497469771041.54315938069757-0.603390411540623.383088459584220.823740765148641-0.129951318508883-0.657979878422938-0.499534924074273-0.414476569095651
⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮
+

6,670 rows × 13 columns

Trait1Trait2Trait3Trait4Trait5Trait6Trait7Trait8Trait9Trait10Trait11Trait12Trait13
Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰Float64⍰
1-1.81573-0.946151.11363-2.098670.7444170.001391720.934732-1.226771.11608-0.4436280.824466-1.02853-0.394049
2-1.24440.109660.467119-1.621311.056680.9789471.000150.3248741.162322.692273.082641.090650.0256616
31.455671.538671.094030.586655-0.327965-0.303377-0.0334355-0.464463-0.33194-0.486839-1.10649-1.42016-0.687463
4-0.7688090.5134910.244263-1.31741.193941.173441.087370.5360230.8027590.2341590.394175-0.7673660.0635386
5-0.264415-0.34824-0.02390650.004739161.256191.203891.298010.3101140.626160.8992890.5499680.5406880.179675
6-1.37617-1.471920.29118-0.803111-0.26424-0.260573-0.165372-0.2192571.04702-0.09858160.9473930.5940150.245407
70.100942-0.191616-0.5674210.378571-0.246656-0.6088110.189081-1.27078-0.4524760.7025630.3326360.002691650.317117
8-0.3198181.357740.81869-1.155660.6344840.2914620.933324-0.7410830.647478-0.9708780.2208610.852512-0.225905
9-0.2883340.5660830.254958-0.6525780.6689220.9783090.1228631.479090.06721320.07959040.1675320.2469160.539933
10-1.1576-0.781199-0.595808-1.005550.7898290.5710580.951304-0.2959630.990420.5613090.7331-1.73468-1.35278
110.7405691.408740.734690.0208323-0.337441-0.458304-0.142583-0.580392-0.684685-0.00785381-0.712244-0.313346-0.345419
12-0.6758920.2798930.267916-1.041040.9107420.8660281.074140.03817510.766355-0.340118-0.8090140.548522-0.0201829
13-0.79541-0.699990.39913-0.5104761.515521.287431.537720.1339891.020260.499019-0.369483-1.10153-0.598132
14-0.193483-0.286021-0.6914940.01315821.523371.401061.531150.3330661.043720.163207-0.422884-0.383528-0.489222
150.1512462.091852.038-1.124751.665571.625361.587510.6358520.8425780.450762-1.39479-0.5609840.28935
16-0.4646090.3612771.23277-0.8260341.434751.744520.2110972.648161.025110.1197570.0596832-0.631232-0.207879
17-0.732977-0.5262230.616579-0.554480.9474850.9368330.9725170.2902511.012850.516207-0.03006890.8787320.450255
18-0.1673260.1753270.287468-0.4026530.5511820.5222050.4368380.2995650.58311-0.704416-0.73081-1.95141-0.933505
191.411591.787220.8439760.481278-0.0887674-0.4995780.304196-1.23884-0.153476-0.8704860.0955473-0.983708-0.356345
20-1.42997-0.4901470.27273-1.61030.9907880.7116881.18858-0.3712291.24703-0.03891620.8834962.589883.3354
21-0.1472470.1232840.617549-0.1871310.2564380.177950.412612-0.2448090.09476250.723017-0.6839480.0873751-0.26221
22-0.187113-0.270777-1.015570.06028510.272420.869133-0.6575192.32389-0.9999361.446720.971158-0.358748-0.439658
23-1.82434-0.933481.29474-1.945450.3358470.3592020.513653-0.07319771.571391.533291.820772.22741.50063
24-2.29344-2.491620.40384-2.364881.410531.422441.170240.8447671.790270.648182-0.0857231-1.02790.491288
25-0.4341360.7408820.699576-1.024060.7595290.9566560.63330.7707340.8249891.842871.91046-0.5023170.13267
26-2.1921-2.494660.354855-1.931560.9419790.9789170.894860.4632391.125371.705280.7177930.6458880.783968
27-1.46602-1.249220.307978-1.550970.6189080.6625080.4759570.4847190.4015650.55988-0.376938-0.9339830.390013
28-1.83318-1.532692.55674-1.518280.789410.9087480.6499720.6683741.200580.2779631.25053.31372.22036
29-0.7845470.2765833.01105-1.119790.9208240.7502181.26154-0.4033640.400667-0.217598-0.72467-0.391945-0.650024
300.4644561.33264-1.2306-0.3579761.18251.54316-0.603393.383090.823741-0.129951-0.65798-0.499535-0.414477
⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮
@@ -246,175 +331,11 @@ cg10k_trait[:, 3:end] describe(cg10k_trait[:, 3:end]) ``` - Trait1 - Summary Stats: - Mean: 0.002211 - Minimum: -3.204128 - 1st Quartile: -0.645771 - Median: 0.125010 - 3rd Quartile: 0.723315 - Maximum: 3.479398 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait2 - Summary Stats: - Mean: 0.001353 - Minimum: -3.511659 - 1st Quartile: -0.642621 - Median: 0.033517 - 3rd Quartile: 0.657467 - Maximum: 4.913423 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait3 - Summary Stats: - Mean: -0.001296 - Minimum: -3.938436 - 1st Quartile: -0.640907 - Median: -0.000782 - 3rd Quartile: 0.637108 - Maximum: 7.916299 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait4 - Summary Stats: - Mean: 0.002309 - Minimum: -3.608403 - 1st Quartile: -0.546086 - Median: 0.228165 - 3rd Quartile: 0.715291 - Maximum: 3.127688 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait5 - Summary Stats: - Mean: -0.001790 - Minimum: -4.148749 - 1st Quartile: -0.690765 - Median: 0.031034 - 3rd Quartile: 0.734916 - Maximum: 2.717184 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait6 - Summary Stats: - Mean: -0.001196 - Minimum: -3.824792 - 1st Quartile: -0.662796 - Median: 0.036242 - 3rd Quartile: 0.741176 - Maximum: 2.589728 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait7 - Summary Stats: - Mean: -0.001989 - Minimum: -4.272455 - 1st Quartile: -0.638923 - Median: 0.069801 - 3rd Quartile: 0.710423 - Maximum: 2.653779 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait8 - Summary Stats: - Mean: 0.000614 - Minimum: -5.625488 - 1st Quartile: -0.601575 - Median: -0.038630 - 3rd Quartile: 0.527342 - Maximum: 5.805702 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait9 - Summary Stats: - Mean: -0.001810 - Minimum: -5.381968 - 1st Quartile: -0.601429 - Median: 0.106571 - 3rd Quartile: 0.698567 - Maximum: 2.571936 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait10 - Summary Stats: - Mean: -0.000437 - Minimum: -3.548506 - 1st Quartile: -0.633641 - Median: -0.096651 - 3rd Quartile: 0.498610 - Maximum: 6.537820 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait11 - Summary Stats: - Mean: -0.000616 - Minimum: -3.264910 - 1st Quartile: -0.673685 - Median: -0.068044 - 3rd Quartile: 0.655486 - Maximum: 4.262410 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait12 - Summary Stats: - Mean: -0.000589 - Minimum: -8.851909 - 1st Quartile: -0.539686 - Median: -0.141099 - 3rd Quartile: 0.350779 - Maximum: 13.211402 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - - Trait13 - Summary Stats: - Mean: -0.000151 - Minimum: -5.592104 - 1st Quartile: -0.492289 - Median: -0.141022 - 3rd Quartile: 0.324804 - Maximum: 24.174436 - Length: 6670 - Type: Float64 - Number Missing: 0 - % Missing: 0.000000 - + + + +

13 rows × 8 columns

variablemeanminmedianmaxnuniquenmissingeltype
SymbolFloat64Float64Float64Float64NothingInt64DataType
1Trait10.00221138-3.204130.125013.47940Float64
2Trait20.00135253-3.511660.03351734.913420Float64
3Trait3-0.00129591-3.93844-0.0007821627.91630Float64
4Trait40.00230893-3.60840.2281653.127690Float64
5Trait5-0.00179039-4.148750.03103432.717180Float64
6Trait6-0.00119598-3.824790.0362422.589730Float64
7Trait7-0.00198906-4.272460.0698012.653780Float64
8Trait80.000614075-5.62549-0.03863015.80570Float64
9Trait9-0.00180965-5.381970.1065712.571940Float64
10Trait10-0.000437029-3.54851-0.09665076.537820Float64
11Trait11-0.000615918-3.26491-0.06804374.262410Float64
12Trait12-0.000588783-8.85191-0.14109913.21140Float64
13Trait13-0.000151238-5.5921-0.14102224.17440Float64
+ @@ -426,7 +347,7 @@ histogram(Y, layout = 13) - +![png](fig2_heritability.png) @@ -436,20 +357,17 @@ To prepare variance component model fitting, we form an instance of `VarianceCom ```julia -using VarianceComponentModels +using VarianceComponentModels, LinearAlgebra # form data as VarianceComponentVariate -cg10kdata = VarianceComponentVariate(Y, (2Φgrm, eye(size(Y, 1)))) -fieldnames(cg10kdata) +cg10kdata = VarianceComponentVariate(Y, (2Φgrm, Matrix(1.0I, size(Y, 1), size(Y, 1)))) +fieldnames(typeof(cg10kdata)) ``` - 3-element Array{Symbol,1}: - :Y - :X - :V + (:Y, :X, :V) @@ -461,7 +379,7 @@ cg10kdata - VarianceComponentModels.VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}}([-1.81573 -0.94615 … -1.02853 -0.394049; -1.2444 0.10966 … 1.09065 0.0256616; … ; 0.886626 0.487408 … -0.636874 -0.439825; -1.24394 0.213697 … 0.299931 0.392809], Array{Float64}(6670,0), ([1.00605 0.00671009 … -0.000109037 -0.00556144; 0.00671009 0.997916 … 0.00173694 0.00685701; … ; -0.000109037 0.00173694 … 1.00166 0.000938955; -0.00556144 0.00685701 … 0.000938955 1.00125], [1.0 0.0 … 0.0 0.0; 0.0 1.0 … 0.0 0.0; … ; 0.0 0.0 … 1.0 0.0; 0.0 0.0 … 0.0 1.0])) + VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}}([-1.81573 -0.94615 … -1.02853 -0.394049; -1.2444 0.10966 … 1.09065 0.0256616; … ; 0.886626 0.487408 … -0.636874 -0.439825; -1.24394 0.213697 … 0.299931 0.392809], Array{Float64}(6670,0), ([1.00547 0.00656224 … -0.000121918 -0.00555374; 0.00656224 0.99614 … 0.00176999 0.00683183; … ; -0.000121918 0.00176999 … 0.998578 0.000962983; -0.00555374 0.00683183 … 0.000962983 0.999599], [1.0 0.0 … 0.0 0.0; 0.0 1.0 … 0.0 0.0; … ; 0.0 0.0 … 1.0 0.0; 0.0 0.0 … 0.0 1.0])) @@ -471,21 +389,16 @@ Before fitting the variance component model, we pre-compute the eigen-decomposit ```julia # pre-compute eigen-decomposition (~50 secs on my laptop) @time cg10kdata_rotated = TwoVarCompVariateRotate(cg10kdata) -fieldnames(cg10kdata_rotated) +fieldnames(typeof(cg10kdata_rotated)) ``` - 48.837361 seconds (39 allocations: 1021.427 MiB, 0.57% gc time) + 49.812646 seconds (1.74 M allocations: 1.080 GiB, 0.33% gc time) - 5-element Array{Symbol,1}: - :Yrot - :Xrot - :eigval - :eigvec - :logdetV2 + (:Yrot, :Xrot, :eigval, :eigvec, :logdetV2) @@ -493,94 +406,80 @@ fieldnames(cg10kdata_rotated) We don't want to re-compute SnpArray and empirical kinship matrices again and again for heritibility analysis. +To load workspace + ```julia -# # Pkg.add("JLD") -# using JLD -# @save "cg10k.jld" -# whos() +#pkg"add FileIO JLD2" + +using JLD2 +@save "cg10k.jld2" +varinfo() ``` -To load workspace + + + +| name | size | summary | +|:----------------- | ------------:|:-------------------------------------------------------------------------------------- | +| Base | | Module | +| Core | | Module | +| Main | | Module | +| Plots | 2.988 MiB | Module | +| PyPlot | 782.799 KiB | Module | +| Y | 677.461 KiB | 6670×13 Array{Float64,2} | +| cg10k | 1022.983 MiB | 6670×630860 SnpArray | +| cg10k_trait | 1.605 MiB | 6670×15 DataFrame | +| cg10kdata | 679.508 MiB | VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}} | +| cg10kdata_rotated | 340.136 MiB | TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}} | +| maf_cg10k | 4.813 MiB | 630860-element Array{Float64,1} | +| missings_by_snp | 4.813 MiB | 1×630860 Array{Int64,2} | +| mp | 28.748 MiB | 6670×630860 SparseArrays.SparseMatrixCSC{Bool,Int32} | +| people | 8 bytes | Int64 | +| snps | 8 bytes | Int64 | +| startupfile | 57 bytes | String | +| Φgrm | 339.423 MiB | 6670×6670 Array{Float64,2} | + + + ```julia -using SnpArrays, JLD, DataFrames, VarianceComponentModels, Plots +using SnpArrays, JLD2, DataFrames, VarianceComponentModels, Plots pyplot() -@load "cg10k.jld" -whos() + +@load "cg10k.jld2" +varinfo() ``` - Base Module - BinDeps 41348 KB Module - Blosc 41202 KB Module - ColorTypes 41457 KB Module - Colors 41480 KB Module - Compat 41196 KB Module - Conda 41205 KB Module - Core Module - DataArrays 41456 KB Module - DataFrames 41684 KB Module - DataStructures 41356 KB Module - FileIO 41310 KB Module - FixedPointNumbers 41695 KB Module - GZip 41181 KB Module - HDF5 41403 KB Module - IJulia 4185781 KB Module - Ipopt 41172 KB Module - JLD 41376 KB Module - JSON 41245 KB Module - LaTeXStrings 4058 bytes Module - LegacyStrings 41212 KB Module - LinearMaps 22 KB Module - MacroTools 41606 KB Module - Main Module - MathProgBase 41353 KB Module - MbedTLS 41269 KB Module - Measures 41175 KB Module - NaNMath 41200 KB Module - PlotThemes 41167 KB Module - PlotUtils 41332 KB Module - Plots 42960 KB Module - PyCall 41711 KB Module - PyPlot 41771 KB Module - RecipesBase 41283 KB Module - Reexport 41160 KB Module - Requires 41172 KB Module - SHA 62 KB Module - Showoff 41163 KB Module - SnpArrays 41218 KB Module - SortingAlgorithms 41178 KB Module - SpecialFunctions 41252 KB Module - StaticArrays 41744 KB Module - StatsBase 41810 KB Module - URIParser 41171 KB Module - VarianceComponentModels 41278 KB Module - Y 677 KB 6670×13 Array{Float64,2} - ZMQ 41223 KB Module - _ 77 KB 630860-element BitArray{1} - cg10k 1027303 KB 6670×630860 SnpArrays.SnpArray{2} - cg10k_trait 978 KB 6670×15 DataFrames.DataFrame - cg10kdata 695816 KB VarianceComponentModels.VarianceCo… - cg10kdata_rotated 348299 KB VarianceComponentModels.TwoVarComp… - h 24 bytes 3-element Array{Float64,1} - hST 104 bytes 13-element Array{Float64,1} - hST_se 104 bytes 13-element Array{Float64,1} - hse 24 bytes 3-element Array{Float64,1} - maf 4928 KB 630860-element Array{Float64,1} - missings_by_snp 4928 KB 630860-element Array{Int64,1} - people 8 bytes Int64 - snps 8 bytes Int64 - trait57_data 347778 KB VarianceComponentModels.TwoVarComp… - trait57_model 232 bytes VarianceComponentModels.VarianceCo… - traitall_model 2792 bytes VarianceComponentModels.VarianceCo… - traitidx 16 bytes 3-element UnitRange{Int64} - Σa 3848 bytes 13×13 Array{Array{Float64,2},2} - Σcov 2592 bytes 18×18 Array{Float64,2} - Σe 3848 bytes 13×13 Array{Array{Float64,2},2} - Φgrm 347569 KB 6670×6670 Array{Float64,2} - σ2a 104 bytes 13-element Array{Float64,1} - σ2e 104 bytes 13-element Array{Float64,1} + ┌ Warning: type SparseArrays.SparseMatrixCSC{Bool,Int32} does not exist in workspace; reconstructing + └ @ JLD2 /Users/juhyun-kim/.julia/packages/JLD2/KjBIK/src/data.jl:1153 + + + + + +| name | size | summary | +|:----------------- | ------------:|:------------------------------------------------------------------------------------------- | +| Base | | Module | +| Core | | Module | +| Main | | Module | +| Plots | 2.988 MiB | Module | +| PyPlot | 782.799 KiB | Module | +| Y | 677.461 KiB | 6670×13 Array{Float64,2} | +| cg10k | 1022.983 MiB | 6670×630860 SnpArray | +| cg10k_trait | 1.605 MiB | 6670×15 DataFrame | +| cg10kdata | 679.508 MiB | VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}} | +| cg10kdata_rotated | 340.136 MiB | TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}} | +| maf_cg10k | 4.813 MiB | 630860-element Array{Float64,1} | +| missings_by_snp | 4.813 MiB | 1×630860 Array{Int64,2} | +| mp | 28.748 MiB | getfield(JLD2.ReconstructedTypes, Symbol("##SparseArrays.SparseMatrixCSC{Bool,Int32}#386")) | +| people | 8 bytes | Int64 | +| snps | 8 bytes | Int64 | +| startupfile | 57 bytes | String | +| Φgrm | 339.423 MiB | 6670×6670 Array{Float64,2} | + + ## Heritability of single traits @@ -598,8 +497,7 @@ hST_se = zeros(13) # enviromental effects σ2e = zeros(13) -tic() -for trait in 1:13 +@time for trait in 1:13 println(names(cg10k_trait)[trait + 2]) # form data set for trait j traitj_data = TwoVarCompVariateRotate(cg10kdata_rotated.Yrot[:, trait], cg10kdata_rotated.Xrot, @@ -615,43 +513,42 @@ for trait in 1:13 hST[trait] = h[1] hST_se[trait] = hse[1] end -toc() ``` Trait1 - (σ2a[trait], σ2e[trait]) = (0.25978160614793233, 0.7369535197912689) + + ****************************************************************************** + This program contains Ipopt, a library for large-scale nonlinear optimization. + Ipopt is released as open source code under the Eclipse Public License (EPL). + For more information visit http://projects.coin-or.org/Ipopt + ****************************************************************************** + + (σ2a[trait], σ2e[trait]) = (0.2594760477653529, 0.7375938878381831) Trait2 - (σ2a[trait], σ2e[trait]) = (0.18647130348299173, 0.8129591079735827) + (σ2a[trait], σ2e[trait]) = (0.18588150448760332, 0.8137880281704619) Trait3 - (σ2a[trait], σ2e[trait]) = (0.3188368159422607, 0.6798809726936244) + (σ2a[trait], σ2e[trait]) = (0.31960937565352526, 0.6795300238388766) Trait4 - (σ2a[trait], σ2e[trait]) = (0.2651357653143703, 0.7308007669086968) + (σ2a[trait], σ2e[trait]) = (0.26578244697498915, 0.7304994606945259) Trait5 - (σ2a[trait], σ2e[trait]) = (0.28083388108246, 0.7172036435586534) + (σ2a[trait], σ2e[trait]) = (0.28143198005767217, 0.7169747061764987) Trait6 - (σ2a[trait], σ2e[trait]) = (0.2824159905728832, 0.7170988773569172) + (σ2a[trait], σ2e[trait]) = (0.2830055131241748, 0.7168800753377107) Trait7 - (σ2a[trait], σ2e[trait]) = (0.2155274336968625, 0.7815346282986375) + (σ2a[trait], σ2e[trait]) = (0.2156542141311619, 0.7816910320891676) Trait8 - (σ2a[trait], σ2e[trait]) = (0.194687807263945, 0.8049690651320599) + (σ2a[trait], σ2e[trait]) = (0.19408878271207824, 0.8058201577783562) Trait9 - (σ2a[trait], σ2e[trait]) = (0.24706855916591713, 0.7512942998567308) + (σ2a[trait], σ2e[trait]) = (0.24746236011763145, 0.7512222977091793) Trait10 - (σ2a[trait], σ2e[trait]) = (0.098712236297271, 0.9011756660217387) + (σ2a[trait], σ2e[trait]) = (0.0992417256213392, 0.9007769787053657) Trait11 - (σ2a[trait], σ2e[trait]) = (0.1664264642608195, 0.8322427413046204) + (σ2a[trait], σ2e[trait]) = (0.1645726648140337, 0.8343110221526228) Trait12 - (σ2a[trait], σ2e[trait]) = (0.0834296761650666, 0.9153609794266608) + (σ2a[trait], σ2e[trait]) = (0.0822495659186408, 0.9166483378364408) Trait13 - (σ2a[trait], σ2e[trait]) = (0.05893968504298988, 0.940270012443928) - elapsed time: 0.160999612 seconds - - - - - - 0.160999612 - + (σ2a[trait], σ2e[trait]) = (0.05687679106195183, 0.9424058676223598) + 6.203222 seconds (12.80 M allocations: 621.516 MiB, 7.13% gc time) @@ -664,8 +561,8 @@ toc() 2×13 Array{Float64,2}: - 0.260633 0.186578 0.319246 … 0.166648 0.0835307 0.0589863 - 0.0799732 0.0869002 0.0741007 0.08862 0.0944407 0.0953238 + 0.260239 0.185943 0.319885 … 0.164757 0.0823403 0.0569176 + 0.0799434 0.08689 0.0739664 0.0887138 0.0944375 0.0953072 @@ -676,12 +573,12 @@ Joint analysis of multiple traits is subject to intensive research recently. Fol ```julia # additive genetic effects (2x2 psd matrices) from bavariate trait analysis; -Σa = Array{Matrix{Float64}}(13, 13) +Σa = Array{Matrix{Float64}}(undef, 13, 13) # environmental effects (2x2 psd matrices) from bavariate trait analysis; -Σe = Array{Matrix{Float64}}(13, 13) +Σe = Array{Matrix{Float64}}(undef, 13, 13) + -tic() -for i in 1:13 +@time for i in 1:13 for j in (i+1):13 println(names(cg10k_trait)[i + 2], names(cg10k_trait)[j + 2]) # form data set for (trait1, trait2) @@ -696,173 +593,165 @@ for i in 1:13 @show Σa[i, j], Σe[i, j] end end -toc() ``` Trait1Trait2 - (Σa[i, j], Σe[i, j]) = ([0.258822 0.174358; 0.174358 0.185108], [0.737892 0.585751; 0.585751 0.814301]) + (Σa[i, j], Σe[i, j]) = ([0.258529 0.173943; 0.173943 0.184525], [0.738519 0.58639; 0.58639 0.815122]) Trait1Trait3 - (Σa[i, j], Σe[i, j]) = ([0.260236 -0.0144726; -0.0144726 0.319245], [0.736512 -0.11979; -0.11979 0.679488]) + (Σa[i, j], Σe[i, j]) = ([0.259934 -0.0139395; -0.0139395 0.320025], [0.737149 -0.120342; -0.120342 0.67913]) Trait1Trait4 - (Σa[i, j], Σe[i, j]) = ([0.259615 0.222203; 0.222203 0.265149], [0.737116 0.599854; 0.599854 0.730788]) + (Σa[i, j], Σe[i, j]) = ([0.259307 0.222267; 0.222267 0.265791], [0.737759 0.600077; 0.600077 0.730492]) Trait1Trait5 - (Σa[i, j], Σe[i, j]) = ([0.259574 -0.146827; -0.146827 0.28153], [0.737153 -0.254777; -0.254777 0.71653]) + (Σa[i, j], Σe[i, j]) = ([0.259264 -0.147125; -0.147125 0.282102], [0.737798 -0.254669; -0.254669 0.716327]) Trait1Trait6 - (Σa[i, j], Σe[i, j]) = ([0.259476 -0.129115; -0.129115 0.282688], [0.73725 -0.23161; -0.23161 0.716837]) + (Σa[i, j], Σe[i, j]) = ([0.259171 -0.129519; -0.129519 0.283267], [0.73789 -0.231373; -0.231373 0.716628]) Trait1Trait7 - (Σa[i, j], Σe[i, j]) = ([0.259115 -0.140455; -0.140455 0.215297], [0.737606 -0.197616; -0.197616 0.781774]) + (Σa[i, j], Σe[i, j]) = ([0.258784 -0.140357; -0.140357 0.21535], [0.738269 -0.197894; -0.197894 0.782002]) Trait1Trait8 - (Σa[i, j], Σe[i, j]) = ([0.259778 -0.0327756; -0.0327756 0.194698], [0.736957 -0.127026; -0.127026 0.804959]) + (Σa[i, j], Σe[i, j]) = ([0.259467 -0.0336174; -0.0336174 0.194107], [0.737603 -0.12623; -0.12623 0.805802]) Trait1Trait9 - (Σa[i, j], Σe[i, j]) = ([0.261858 -0.204589; -0.204589 0.246027], [0.734961 -0.307734; -0.307734 0.75232]) + (Σa[i, j], Σe[i, j]) = ([0.261524 -0.204467; -0.204467 0.246378], [0.735632 -0.308118; -0.308118 0.752288]) Trait1Trait10 - (Σa[i, j], Σe[i, j]) = ([0.259649 -0.0994858; -0.0994858 0.0956585], [0.737083 -0.303942; -0.303942 0.904218]) + (Σa[i, j], Σe[i, j]) = ([0.259326 -0.0987008; -0.0987008 0.0962217], [0.73774 -0.304854; -0.304854 0.90378]) Trait1Trait11 - (Σa[i, j], Σe[i, j]) = ([0.25947 -0.138603; -0.138603 0.164709], [0.737257 -0.359557; -0.359557 0.83395]) + (Σa[i, j], Σe[i, j]) = ([0.259194 -0.137578; -0.137578 0.162813], [0.737868 -0.36076; -0.36076 0.836058]) Trait1Trait12 - (Σa[i, j], Σe[i, j]) = ([0.261779 -0.145414; -0.145414 0.0807748], [0.735076 -0.041823; -0.041823 0.9181]) + (Σa[i, j], Σe[i, j]) = ([0.26168 -0.145305; -0.145305 0.0806851], [0.735514 -0.0421192; -0.0421192 0.918294]) Trait1Trait13 - (Σa[i, j], Σe[i, j]) = ([0.261125 -0.108774; -0.108774 0.0538214], [0.735674 -0.114123; -0.114123 0.945416]) + (Σa[i, j], Σe[i, j]) = ([0.260776 -0.108277; -0.108277 0.0513909], [0.736359 -0.114756; -0.114756 0.947913]) Trait2Trait3 - (Σa[i, j], Σe[i, j]) = ([0.186541 0.144056; 0.144056 0.320627], [0.812888 0.0995944; 0.0995944 0.678167]) + (Σa[i, j], Σe[i, j]) = ([0.185992 0.14505; 0.14505 0.321413], [0.813678 0.0987898; 0.0987898 0.677805]) Trait2Trait4 - (Σa[i, j], Σe[i, j]) = ([0.186131 0.0746032; 0.0746032 0.265122], [0.813293 0.221109; 0.221109 0.730814]) + (Σa[i, j], Σe[i, j]) = ([0.18554 0.074367; 0.074367 0.26577], [0.814123 0.22144; 0.22144 0.730512]) Trait2Trait5 - (Σa[i, j], Σe[i, j]) = ([0.186442 -0.0118093; -0.0118093 0.280842], [0.812987 -0.0365191; -0.0365191 0.717195]) + (Σa[i, j], Σe[i, j]) = ([0.185854 -0.0115021; -0.0115021 0.281443], [0.813815 -0.0368393; -0.0368393 0.716964]) Trait2Trait6 - (Σa[i, j], Σe[i, j]) = ([0.18649 -0.00366533; -0.00366533 0.282471], [0.812941 -0.0206271; -0.0206271 0.717046]) + (Σa[i, j], Σe[i, j]) = ([0.185903 -0.00350261; -0.00350261 0.283064], [0.813767 -0.0207933; -0.0207933 0.716823]) Trait2Trait7 - (Σa[i, j], Σe[i, j]) = ([0.186104 -0.030665; -0.030665 0.215304], [0.81332 -0.000667009; -0.000667009 0.781755]) + (Σa[i, j], Σe[i, j]) = ([0.185498 -0.0301226; -0.0301226 0.215413], [0.814164 -0.00124563; -0.00124563 0.781929]) Trait2Trait8 - (Σa[i, j], Σe[i, j]) = ([0.187023 0.0331783; 0.0331783 0.195259], [0.812421 -0.0326343; -0.0326343 0.804415]) + (Σa[i, j], Σe[i, j]) = ([0.186397 0.032688; 0.032688 0.194625], [0.813285 -0.0321045; -0.0321045 0.805301]) Trait2Trait9 - (Σa[i, j], Σe[i, j]) = ([0.185032 -0.085334; -0.085334 0.245909], [0.814386 -0.0809638; -0.0809638 0.752433]) + (Σa[i, j], Σe[i, j]) = ([0.184426 -0.084907; -0.084907 0.246288], [0.815228 -0.0814978; -0.0814978 0.752373]) Trait2Trait10 - (Σa[i, j], Σe[i, j]) = ([0.186587 -0.123303; -0.123303 0.0987387], [0.812872 -0.273083; -0.273083 0.901229]) + (Σa[i, j], Σe[i, j]) = ([0.18604 -0.123261; -0.123261 0.0992567], [0.813659 -0.273285; -0.273285 0.90084]) Trait2Trait11 - (Σa[i, j], Σe[i, j]) = ([0.185484 -0.117256; -0.117256 0.167776], [0.81393 -0.296772; -0.296772 0.830934]) + (Σa[i, j], Σe[i, j]) = ([0.184894 -0.116422; -0.116422 0.165892], [0.814757 -0.297755; -0.297755 0.833033]) Trait2Trait12 - (Σa[i, j], Σe[i, j]) = ([0.185907 -0.0909104; -0.0909104 0.0827171], [0.813555 0.0457924; 0.0457924 0.916135]) + (Σa[i, j], Σe[i, j]) = ([0.185297 -0.0908872; -0.0908872 0.0814927], [0.814404 0.0456529; 0.0456529 0.917465]) Trait2Trait13 - (Σa[i, j], Σe[i, j]) = ([0.185979 -0.0720811; -0.0720811 0.0568238], [0.8135 0.0751703; 0.0751703 0.942424]) + (Σa[i, j], Σe[i, j]) = ([0.185348 -0.07104; -0.07104 0.0546791], [0.814367 0.0740412; 0.0740412 0.944638]) Trait3Trait4 - (Σa[i, j], Σe[i, j]) = ([0.3188 -0.154562; -0.154562 0.264323], [0.679917 -0.303223; -0.303223 0.731591]) + (Σa[i, j], Σe[i, j]) = ([0.319575 -0.15468; -0.15468 0.265006], [0.679563 -0.303309; -0.303309 0.731254]) Trait3Trait5 - (Σa[i, j], Σe[i, j]) = ([0.319216 0.183527; 0.183527 0.282063], [0.679514 0.33724; 0.33724 0.716008]) + (Σa[i, j], Σe[i, j]) = ([0.320016 0.184736; 0.184736 0.282719], [0.679136 0.336276; 0.336276 0.715725]) Trait3Trait6 - (Σa[i, j], Σe[i, j]) = ([0.319776 0.165672; 0.165672 0.284448], [0.678972 0.298667; 0.298667 0.715124]) + (Σa[i, j], Σe[i, j]) = ([0.320608 0.166929; 0.166929 0.285124], [0.678565 0.297633; 0.297633 0.714823]) Trait3Trait7 - (Σa[i, j], Σe[i, j]) = ([0.318838 0.166283; 0.166283 0.215261], [0.67988 0.347706; 0.347706 0.781796]) + (Σa[i, j], Σe[i, j]) = ([0.319614 0.167317; 0.167317 0.215458], [0.679526 0.346897; 0.346897 0.781883]) Trait3Trait8 - (Σa[i, j], Σe[i, j]) = ([0.320718 0.0566397; 0.0566397 0.197764], [0.678063 0.0451569; 0.0451569 0.801956]) + (Σa[i, j], Σe[i, j]) = ([0.321532 0.0573397; 0.0573397 0.197221], [0.677674 0.0445311; 0.0445311 0.802757]) Trait3Trait9 - (Σa[i, j], Σe[i, j]) = ([0.319001 0.137699; 0.137699 0.246142], [0.679722 0.266704; 0.266704 0.752197]) + (Σa[i, j], Σe[i, j]) = ([0.319777 0.138268; 0.138268 0.246547], [0.679369 0.266317; 0.266317 0.752112]) Trait3Trait10 - (Σa[i, j], Σe[i, j]) = ([0.31908 -0.076513; -0.076513 0.0996001], [0.679646 -0.142905; -0.142905 0.900298]) + (Σa[i, j], Σe[i, j]) = ([0.319905 -0.0779601; -0.0779601 0.100192], [0.679245 -0.141562; -0.141562 0.899839]) Trait3Trait11 - (Σa[i, j], Σe[i, j]) = ([0.318094 -0.0177494; -0.0177494 0.16629], [0.6806 -0.1144; -0.1144 0.832376]) + (Σa[i, j], Σe[i, j]) = ([0.318882 -0.0177352; -0.0177352 0.164446], [0.680233 -0.114434; -0.114434 0.834435]) Trait3Trait12 - (Σa[i, j], Σe[i, j]) = ([0.321164 0.0843842; 0.0843842 0.0874609], [0.677639 0.0341558; 0.0341558 0.911368]) + (Σa[i, j], Σe[i, j]) = ([0.321958 0.0844644; 0.0844644 0.0863388], [0.67727 0.034186; 0.034186 0.912603]) Trait3Trait13 - (Σa[i, j], Σe[i, j]) = ([0.323273 0.109443; 0.109443 0.0634295], [0.675635 -0.0060525; -0.0060525 0.935819]) + (Σa[i, j], Σe[i, j]) = ([0.32405 0.109731; 0.109731 0.0613165], [0.675287 -0.00619989; -0.00619989 0.938011]) Trait4Trait5 - (Σa[i, j], Σe[i, j]) = ([0.26525 -0.215125; -0.215125 0.282572], [0.73068 -0.377406; -0.377406 0.715518]) + (Σa[i, j], Σe[i, j]) = ([0.26593 -0.216116; -0.216116 0.283173], [0.730347 -0.376697; -0.376697 0.715289]) Trait4Trait6 - (Σa[i, j], Σe[i, j]) = ([0.265715 -0.199714; -0.199714 0.283942], [0.730231 -0.347732; -0.347732 0.715619]) + (Σa[i, j], Σe[i, j]) = ([0.266395 -0.200793; -0.200793 0.284547], [0.729899 -0.346915; -0.346915 0.715387]) Trait4Trait7 - (Σa[i, j], Σe[i, j]) = ([0.26407 -0.18238; -0.18238 0.214324], [0.731843 -0.32655; -0.32655 0.782733]) + (Σa[i, j], Σe[i, j]) = ([0.264731 -0.182912; -0.182912 0.21441], [0.731526 -0.326257; -0.326257 0.782928]) Trait4Trait8 - (Σa[i, j], Σe[i, j]) = ([0.266229 -0.0965381; -0.0965381 0.196655], [0.729739 -0.151461; -0.151461 0.803044]) + (Σa[i, j], Σe[i, j]) = ([0.266914 -0.0976281; -0.0976281 0.196119], [0.729401 -0.150498; -0.150498 0.803836]) Trait4Trait9 - (Σa[i, j], Σe[i, j]) = ([0.269627 -0.226931; -0.226931 0.247265], [0.726443 -0.416085; -0.416085 0.751086]) + (Σa[i, j], Σe[i, j]) = ([0.270267 -0.227463; -0.227463 0.247665], [0.726155 -0.415849; -0.415849 0.751008]) Trait4Trait10 - (Σa[i, j], Σe[i, j]) = ([0.265098 -0.0352926; -0.0352926 0.0981462], [0.730847 -0.226248; -0.226248 0.901736]) + (Σa[i, j], Σe[i, j]) = ([0.265756 -0.0339156; -0.0339156 0.0987618], [0.730536 -0.227666; -0.227666 0.901251]) Trait4Trait11 - (Σa[i, j], Σe[i, j]) = ([0.265178 -0.0970634; -0.0970634 0.164885], [0.73076 -0.272291; -0.272291 0.833762]) + (Σa[i, j], Σe[i, j]) = ([0.265858 -0.0963336; -0.0963336 0.163013], [0.730427 -0.273145; -0.273145 0.835846]) Trait4Trait12 - (Σa[i, j], Σe[i, j]) = ([0.267732 -0.140985; -0.140985 0.081029], [0.728323 -0.0834791; -0.0834791 0.917815]) + (Σa[i, j], Σe[i, j]) = ([0.268394 -0.141452; -0.141452 0.0796851], [0.72801 -0.0831954; -0.0831954 0.919263]) Trait4Trait13 - (Σa[i, j], Σe[i, j]) = ([0.265695 -0.0970238; -0.0970238 0.0564809], [0.730259 -0.226115; -0.226115 0.942736]) + (Σa[i, j], Σe[i, j]) = ([0.266334 -0.0971798; -0.0971798 0.0541478], [0.729966 -0.226082; -0.226082 0.945138]) Trait5Trait6 - (Σa[i, j], Σe[i, j]) = ([0.281198 0.280259; 0.280259 0.281764], [0.716855 0.661013; 0.661013 0.717735]) + (Σa[i, j], Σe[i, j]) = ([0.281794 0.281033; 0.281033 0.282364], [0.716628 0.660607; 0.660607 0.717505]) Trait5Trait7 - (Σa[i, j], Σe[i, j]) = ([0.280442 0.231918; 0.231918 0.211837], [0.717597 0.674491; 0.674491 0.785172]) + (Σa[i, j], Σe[i, j]) = ([0.281023 0.232223; 0.232223 0.211943], [0.717383 0.67449; 0.67449 0.785343]) Trait5Trait8 - (Σa[i, j], Σe[i, j]) = ([0.280958 0.163168; 0.163168 0.193315], [0.717089 0.221817; 0.221817 0.806314]) + (Σa[i, j], Σe[i, j]) = ([0.281551 0.164011; 0.164011 0.192737], [0.716865 0.221187; 0.221187 0.807144]) Trait5Trait9 - (Σa[i, j], Σe[i, j]) = ([0.283544 0.243884; 0.243884 0.240564], [0.714585 0.509072; 0.509072 0.757631]) + (Σa[i, j], Σe[i, j]) = ([0.284085 0.244479; 0.244479 0.240901], [0.714415 0.508795; 0.508795 0.757606]) Trait5Trait10 - (Σa[i, j], Σe[i, j]) = ([0.281378 -0.0454427; -0.0454427 0.100081], [0.716678 -0.0579778; -0.0579778 0.899822]) + (Σa[i, j], Σe[i, j]) = ([0.281993 -0.0460624; -0.0460624 0.100628], [0.716433 -0.0574194; -0.0574194 0.899407]) Trait5Trait11 - (Σa[i, j], Σe[i, j]) = ([0.280066 0.0195669; 0.0195669 0.165607], [0.71795 -0.0345589; -0.0345589 0.833047]) + (Σa[i, j], Σe[i, j]) = ([0.280652 0.0199425; 0.0199425 0.163738], [0.717732 -0.0349079; -0.0349079 0.83513]) Trait5Trait12 - (Σa[i, j], Σe[i, j]) = ([0.28101 0.0592641; 0.0592641 0.0831831], [0.717036 0.0552788; 0.0552788 0.915608]) + (Σa[i, j], Σe[i, j]) = ([0.281652 0.0612882; 0.0612882 0.0820476], [0.716765 0.0533362; 0.0533362 0.916851]) Trait5Trait13 - (Σa[i, j], Σe[i, j]) = ([0.281854 0.0680641; 0.0680641 0.0591899], [0.716223 0.0551992; 0.0551992 0.940027]) + (Σa[i, j], Σe[i, j]) = ([0.282494 0.0696137; 0.0696137 0.0570685], [0.715955 0.0537392; 0.0537392 0.942222]) Trait6Trait7 - (Σa[i, j], Σe[i, j]) = ([0.282435 0.220236; 0.220236 0.213997], [0.71708 0.581507; 0.581507 0.783041]) + (Σa[i, j], Σe[i, j]) = ([0.283018 0.220794; 0.220794 0.214128], [0.716868 0.58124; 0.58124 0.783191]) Trait6Trait8 - (Σa[i, j], Σe[i, j]) = ([0.282435 0.18375; 0.18375 0.192999], [0.717081 0.436932; 0.436932 0.80663]) + (Σa[i, j], Σe[i, j]) = ([0.283018 0.183995; 0.183995 0.192405], [0.716869 0.436926; 0.436926 0.807474]) Trait6Trait9 - (Σa[i, j], Σe[i, j]) = ([0.284516 0.233768; 0.233768 0.242478], [0.715071 0.477502; 0.477502 0.755765]) + (Σa[i, j], Σe[i, j]) = ([0.285057 0.234311; 0.234311 0.24283], [0.714902 0.477263; 0.477263 0.755728]) Trait6Trait10 - (Σa[i, j], Σe[i, j]) = ([0.283087 -0.0427658; -0.0427658 0.100634], [0.716449 -0.0599491; -0.0599491 0.899275]) + (Σa[i, j], Σe[i, j]) = ([0.283699 -0.0433873; -0.0433873 0.101183], [0.716209 -0.0593852; -0.0593852 0.89886]) Trait6Trait11 - (Σa[i, j], Σe[i, j]) = ([0.281046 0.0272144; 0.0272144 0.165044], [0.71843 -0.0516242; -0.0516242 0.833601]) + (Σa[i, j], Σe[i, j]) = ([0.281626 0.027639; 0.027639 0.163178], [0.718219 -0.0520113; -0.0520113 0.83568]) Trait6Trait12 - (Σa[i, j], Σe[i, j]) = ([0.28256 0.0548537; 0.0548537 0.083133], [0.716961 0.0502064; 0.0502064 0.915658]) + (Σa[i, j], Σe[i, j]) = ([0.283196 0.0569533; 0.0569533 0.0819944], [0.716699 0.0481832; 0.0481832 0.916905]) Trait6Trait13 - (Σa[i, j], Σe[i, j]) = ([0.283231 0.0585667; 0.0585667 0.0592752], [0.716314 0.055827; 0.055827 0.939942]) + (Σa[i, j], Σe[i, j]) = ([0.283862 0.0600111; 0.0600111 0.0571748], [0.716057 0.0544601; 0.0544601 0.942116]) Trait7Trait8 - (Σa[i, j], Σe[i, j]) = ([0.213998 0.0875641; 0.0875641 0.192993], [0.78304 -0.055939; -0.055939 0.806635]) + (Σa[i, j], Σe[i, j]) = ([0.214129 0.0883281; 0.0883281 0.192399], [0.78319 -0.0565873; -0.0565873 0.80748]) Trait7Trait9 - (Σa[i, j], Σe[i, j]) = ([0.219039 0.216925; 0.216925 0.243338], [0.778156 0.463024; 0.463024 0.754935]) + (Σa[i, j], Σe[i, j]) = ([0.219059 0.217228; 0.217228 0.243722], [0.778422 0.463004; 0.463004 0.754868]) Trait7Trait10 - (Σa[i, j], Σe[i, j]) = ([0.216296 -0.0412106; -0.0412106 0.100663], [0.780785 -0.0868086; -0.0868086 0.899246]) + (Σa[i, j], Σe[i, j]) = ([0.216477 -0.042081; -0.042081 0.101258], [0.78089 -0.0859952; -0.0859952 0.898786]) Trait7Trait11 - (Σa[i, j], Σe[i, j]) = ([0.2142 0.0204227; 0.0204227 0.165077], [0.782833 -0.0478727; -0.0478727 0.833569]) + (Σa[i, j], Σe[i, j]) = ([0.214308 0.0205244; 0.0205244 0.163203], [0.783006 -0.0479474; -0.0479474 0.835656]) Trait7Trait12 - (Σa[i, j], Σe[i, j]) = ([0.215054 0.0738562; 0.0738562 0.0814228], [0.782012 0.0366272; 0.0366272 0.917365]) + (Σa[i, j], Σe[i, j]) = ([0.215192 0.0752708; 0.0752708 0.0802075], [0.782157 0.0353123; 0.0353123 0.918686]) Trait7Trait13 - (Σa[i, j], Σe[i, j]) = ([0.216093 0.0728515; 0.0728515 0.0570272], [0.781006 0.0409945; 0.0409945 0.942189]) + (Σa[i, j], Σe[i, j]) = ([0.216198 0.0740251; 0.0740251 0.0547527], [0.781186 0.0399163; 0.0399163 0.944534]) Trait8Trait9 - (Σa[i, j], Σe[i, j]) = ([0.195154 0.111756; 0.111756 0.246453], [0.804528 0.185842; 0.185842 0.751896]) + (Σa[i, j], Σe[i, j]) = ([0.194551 0.112282; 0.112282 0.246832], [0.805384 0.185461; 0.185461 0.751837]) Trait8Trait10 - (Σa[i, j], Σe[i, j]) = ([0.195015 -0.015506; -0.015506 0.0990776], [0.804651 0.0118538; 0.0118538 0.900815]) + (Σa[i, j], Σe[i, j]) = ([0.194399 -0.0154035; -0.0154035 0.0995883], [0.805519 0.0117323; 0.0117323 0.900435]) Trait8Trait11 - (Σa[i, j], Σe[i, j]) = ([0.194421 0.0215044; 0.0215044 0.166226], [0.805231 -0.026247; -0.026247 0.83244]) + (Σa[i, j], Σe[i, j]) = ([0.193837 0.022072; 0.022072 0.164393], [0.806067 -0.0267843; -0.0267843 0.834489]) Trait8Trait12 - (Σa[i, j], Σe[i, j]) = ([0.194491 -0.00425152; -0.00425152 0.0832711], [0.805162 0.0349872; 0.0349872 0.915518]) + (Σa[i, j], Σe[i, j]) = ([0.193914 -0.00259164; -0.00259164 0.0821237], [0.805992 0.0333257; 0.0333257 0.916773]) Trait8Trait13 - (Σa[i, j], Σe[i, j]) = ([0.19448 0.00235501; 0.00235501 0.0589351], [0.805173 0.0396048; 0.0396048 0.940275]) + (Σa[i, j], Σe[i, j]) = ([0.193913 0.00330925; 0.00330925 0.0569126], [0.805993 0.0386552; 0.0386552 0.94237]) Trait9Trait10 - (Σa[i, j], Σe[i, j]) = ([0.246455 -0.00257997; -0.00257997 0.0984563], [0.751895 0.0743439; 0.0743439 0.901429]) + (Σa[i, j], Σe[i, j]) = ([0.246837 -0.00323396; -0.00323396 0.0989732], [0.751834 0.074992; 0.074992 0.901043]) Trait9Trait11 - (Σa[i, j], Σe[i, j]) = ([0.247001 0.0303415; 0.0303415 0.166421], [0.75136 0.153765; 0.153765 0.832248]) + (Σa[i, j], Σe[i, j]) = ([0.247395 0.0305136; 0.0305136 0.164575], [0.751288 0.153633; 0.153633 0.834308]) Trait9Trait12 - (Σa[i, j], Σe[i, j]) = ([0.249421 0.0829968; 0.0829968 0.0890874], [0.749007 0.109331; 0.109331 0.909778]) + (Σa[i, j], Σe[i, j]) = ([0.249928 0.0844216; 0.0844216 0.0880992], [0.748829 0.108018; 0.108018 0.910884]) Trait9Trait13 - (Σa[i, j], Σe[i, j]) = ([0.24861 0.0916799; 0.0916799 0.0602352], [0.749811 0.100027; 0.100027 0.939032]) + (Σa[i, j], Σe[i, j]) = ([0.248998 0.0925603; 0.0925603 0.0580327], [0.749747 0.0992651; 0.0992651 0.94131]) Trait10Trait11 - (Σa[i, j], Σe[i, j]) = ([0.0914658 0.100613; 0.100613 0.166501], [0.908376 0.473847; 0.473847 0.83217]) + (Σa[i, j], Σe[i, j]) = ([0.0923059 0.0990858; 0.0990858 0.164638], [0.907657 0.475503; 0.475503 0.834248]) Trait10Trait12 - (Σa[i, j], Σe[i, j]) = ([0.0951392 0.0588424; 0.0588424 0.0796744], [0.904735 0.0828862; 0.0828862 0.919115]) + (Σa[i, j], Σe[i, j]) = ([0.0957712 0.0574496; 0.0574496 0.0785646], [0.904228 0.0843534; 0.0843534 0.920326]) Trait10Trait13 - (Σa[i, j], Σe[i, j]) = ([0.0995192 -0.0257171; -0.0257171 0.0598595], [0.900397 0.163778; 0.163778 0.939368]) + (Σa[i, j], Σe[i, j]) = ([0.100109 -0.0266483; -0.0266483 0.0578511], [0.899941 0.164674; 0.164674 0.941451]) Trait11Trait12 - (Σa[i, j], Σe[i, j]) = ([0.165386 0.0579914; 0.0579914 0.0796005], [0.83327 0.144637; 0.144637 0.919166]) + (Σa[i, j], Σe[i, j]) = ([0.163544 0.0571549; 0.0571549 0.0784378], [0.835325 0.145549; 0.145549 0.920432]) Trait11Trait13 - (Σa[i, j], Σe[i, j]) = ([0.166417 -0.000985185; -0.000985185 0.0595681], [0.832265 0.200012; 0.200012 0.939646]) + (Σa[i, j], Σe[i, j]) = ([0.164571 -0.00169939; -0.00169939 0.0575331], [0.834326 0.200725; 0.200725 0.941755]) Trait12Trait13 - (Σa[i, j], Σe[i, j]) = ([0.085082 0.0696185; 0.0696185 0.0569655], [0.913729 0.572041; 0.572041 0.942247]) - elapsed time: 3.587337102 seconds - - - - - - 3.587337102 - + (Σa[i, j], Σe[i, j]) = ([0.08398 0.0685417; 0.0685417 0.0559416], [0.91494 0.573206; 0.573206 0.943343]) + 4.430316 seconds (4.62 M allocations: 310.482 MiB, 3.51% gc time) ## 3-trait analysis @@ -882,7 +771,7 @@ trait57_model = VarianceComponentModel(trait57_data) trait57_model ``` - This is Ipopt version 3.12.4, running with linear solver mumps. + This is Ipopt version 3.12.10, running with linear solver mumps. NOTE: Other linear solvers might be more efficient (see Ipopt documentation). Number of nonzeros in equality constraint Jacobian...: 0 @@ -900,26 +789,26 @@ trait57_model inequality constraints with only upper bounds: 0 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls - 0 3.0247565e+04 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 - 5 1.6835078e+04 0.00e+00 4.08e+02 -11.0 3.64e-01 - 1.00e+00 1.00e+00f 1 MaxS - 10 1.4742941e+04 0.00e+00 1.10e+02 -11.0 2.35e-01 - 1.00e+00 1.00e+00f 1 MaxS - 15 1.4701394e+04 0.00e+00 1.16e+01 -11.0 7.78e-02 -4.5 1.00e+00 1.00e+00f 1 MaxS - 20 1.4701019e+04 0.00e+00 5.75e-01 -11.0 1.51e-04 -6.9 1.00e+00 1.00e+00f 1 MaxS - 25 1.4701018e+04 0.00e+00 2.40e-02 -11.0 6.38e-06 -9.2 1.00e+00 1.00e+00f 1 MaxS - 30 1.4701018e+04 0.00e+00 9.98e-04 -11.0 2.66e-07 -11.6 1.00e+00 1.00e+00f 1 MaxS - 35 1.4701018e+04 0.00e+00 4.15e-05 -11.0 1.10e-08 -14.0 1.00e+00 1.00e+00h 1 MaxS - 40 1.4701018e+04 0.00e+00 1.72e-06 -11.0 4.59e-10 -16.4 1.00e+00 1.00e+00f 1 MaxSA - 45 1.4701018e+04 0.00e+00 7.17e-08 -11.0 1.91e-11 -18.8 1.00e+00 1.00e+00h 1 MaxSA + 0 3.0244169e+04 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 + 5 1.6834042e+04 0.00e+00 4.07e+02 -11.0 3.66e-01 - 1.00e+00 1.00e+00f 1 MaxS + 10 1.4744248e+04 0.00e+00 1.12e+02 -11.0 2.36e-01 - 1.00e+00 1.00e+00f 1 MaxS + 15 1.4701415e+04 0.00e+00 1.25e+01 -11.0 1.08e-01 -4.5 1.00e+00 1.00e+00f 1 MaxS + 20 1.4700955e+04 0.00e+00 6.34e-01 -11.0 1.66e-04 -6.9 1.00e+00 1.00e+00f 1 MaxS + 25 1.4700954e+04 0.00e+00 2.63e-02 -11.0 7.01e-06 -9.2 1.00e+00 1.00e+00f 1 MaxS + 30 1.4700954e+04 0.00e+00 1.09e-03 -11.0 2.90e-07 -11.6 1.00e+00 1.00e+00f 1 MaxS + 35 1.4700954e+04 0.00e+00 4.49e-05 -11.0 1.20e-08 -14.0 1.00e+00 1.00e+00f 1 MaxS + 40 1.4700954e+04 0.00e+00 1.86e-06 -11.0 4.96e-10 -16.4 1.00e+00 1.00e+00f 1 MaxSA + 45 1.4700954e+04 0.00e+00 7.67e-08 -11.0 2.05e-11 -18.8 1.00e+00 1.00e+00h 1 MaxSA iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls Number of Iterations....: 49 (scaled) (unscaled) - Objective...............: 4.4720359684330265e+02 1.4701017692082147e+04 - Dual infeasibility......: 5.6081357364421780e-09 1.8435742302386474e-07 + Objective...............: 4.4662368766169095e+02 1.4700954216526397e+04 + Dual infeasibility......: 5.8982229663448267e-09 1.9414444012378635e-07 Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00 Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00 - Overall NLP error.......: 5.6081357364421780e-09 1.8435742302386474e-07 + Overall NLP error.......: 5.8982229663448267e-09 1.9414444012378635e-07 Number of objective function evaluations = 50 @@ -929,17 +818,17 @@ trait57_model Number of equality constraint Jacobian evaluations = 0 Number of inequality constraint Jacobian evaluations = 0 Number of Lagrangian Hessian evaluations = 49 - Total CPU secs in IPOPT (w/o function evaluations) = 0.014 - Total CPU secs in NLP function evaluations = 0.076 + Total CPU secs in IPOPT (w/o function evaluations) = 0.019 + Total CPU secs in NLP function evaluations = 0.054 EXIT: Optimal Solution Found. - 0.097955 seconds (55.15 k allocations: 5.632 MiB) + 0.084479 seconds (46.62 k allocations: 5.057 MiB) - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.280777 0.279441 0.232208; 0.279441 0.28422 0.219831; 0.232208 0.219831 0.212832], [0.717266 0.66183 0.674206; 0.66183 0.715287 0.581891; 0.674206 0.581891 0.784183]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.28137 0.280153 0.232512; 0.280153 0.284916 0.220363; 0.232512 0.220363 0.212921], [0.717042 0.661484 0.674207; 0.661484 0.714964 0.581648; 0.674207 0.581648 0.784373]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf) @@ -956,16 +845,16 @@ trait57_model MM Algorithm Iter Objective -------- ------------- - 0 -1.470102e+04 - 1 -1.470102e+04 + 0 -1.470095e+04 + 1 -1.470095e+04 - 0.003006 seconds (21.01 k allocations: 1.551 MiB) + 0.505115 seconds (1.01 M allocations: 50.370 MiB, 5.09% gc time) - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.280777 0.279441 0.232208; 0.279441 0.28422 0.219831; 0.232208 0.219831 0.212832], [0.717266 0.66183 0.674206; 0.66183 0.715287 0.581891; 0.674206 0.581891 0.784183]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.28137 0.280153 0.232512; 0.280153 0.284916 0.220363; 0.232512 0.220363 0.212921], [0.717042 0.661484 0.674207; 0.661484 0.714964 0.581648; 0.674207 0.581648 0.784373]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf) @@ -983,136 +872,135 @@ trait57_model MM Algorithm Iter Objective -------- ------------- - 0 -3.024757e+04 - 1 -2.040300e+04 - 2 -1.656070e+04 - 3 -1.528529e+04 - 4 -1.490986e+04 - 5 -1.480638e+04 - 6 -1.477811e+04 - 7 -1.476968e+04 - 8 -1.476639e+04 - 9 -1.476444e+04 - 10 -1.476286e+04 - 20 -1.475000e+04 - 30 -1.474011e+04 - 40 -1.473248e+04 - 50 -1.472658e+04 - 60 -1.472200e+04 - 70 -1.471840e+04 - 80 -1.471555e+04 - 90 -1.471328e+04 - 100 -1.471145e+04 - 110 -1.470997e+04 - 120 -1.470875e+04 - 130 -1.470775e+04 - 140 -1.470691e+04 - 150 -1.470621e+04 - 160 -1.470562e+04 - 170 -1.470511e+04 - 180 -1.470469e+04 - 190 -1.470432e+04 - 200 -1.470400e+04 - 210 -1.470372e+04 - 220 -1.470348e+04 - 230 -1.470326e+04 - 240 -1.470308e+04 - 250 -1.470291e+04 - 260 -1.470276e+04 - 270 -1.470263e+04 - 280 -1.470251e+04 - 290 -1.470241e+04 - 300 -1.470231e+04 - 310 -1.470223e+04 - 320 -1.470215e+04 - 330 -1.470208e+04 - 340 -1.470201e+04 - 350 -1.470195e+04 - 360 -1.470190e+04 - 370 -1.470185e+04 - 380 -1.470180e+04 - 390 -1.470176e+04 - 400 -1.470172e+04 - 410 -1.470168e+04 - 420 -1.470165e+04 - 430 -1.470162e+04 - 440 -1.470159e+04 - 450 -1.470156e+04 - 460 -1.470153e+04 - 470 -1.470151e+04 - 480 -1.470149e+04 - 490 -1.470147e+04 - 500 -1.470145e+04 - 510 -1.470143e+04 - 520 -1.470141e+04 - 530 -1.470139e+04 - 540 -1.470138e+04 - 550 -1.470136e+04 - 560 -1.470135e+04 - 570 -1.470133e+04 - 580 -1.470132e+04 - 590 -1.470131e+04 - 600 -1.470130e+04 - 610 -1.470129e+04 - 620 -1.470128e+04 - 630 -1.470127e+04 - 640 -1.470126e+04 - 650 -1.470125e+04 - 660 -1.470124e+04 - 670 -1.470123e+04 - 680 -1.470122e+04 - 690 -1.470122e+04 - 700 -1.470121e+04 - 710 -1.470120e+04 - 720 -1.470120e+04 - 730 -1.470119e+04 - 740 -1.470118e+04 - 750 -1.470118e+04 - 760 -1.470117e+04 - 770 -1.470117e+04 - 780 -1.470116e+04 - 790 -1.470116e+04 - 800 -1.470115e+04 - 810 -1.470115e+04 - 820 -1.470114e+04 - 830 -1.470114e+04 - 840 -1.470114e+04 - 850 -1.470113e+04 - 860 -1.470113e+04 - 870 -1.470112e+04 - 880 -1.470112e+04 - 890 -1.470112e+04 - 900 -1.470111e+04 - 910 -1.470111e+04 - 920 -1.470111e+04 - 930 -1.470111e+04 - 940 -1.470110e+04 - 950 -1.470110e+04 - 960 -1.470110e+04 - 970 -1.470109e+04 - 980 -1.470109e+04 - 990 -1.470109e+04 - 1000 -1.470109e+04 - 1010 -1.470109e+04 - 1020 -1.470108e+04 - 1030 -1.470108e+04 - 1040 -1.470108e+04 - 1050 -1.470108e+04 - 1060 -1.470108e+04 - 1070 -1.470107e+04 - 1080 -1.470107e+04 - 1090 -1.470107e+04 - 1100 -1.470107e+04 - 1110 -1.470107e+04 - 1120 -1.470107e+04 + 0 -3.024417e+04 + 1 -2.040172e+04 + 2 -1.656053e+04 + 3 -1.528554e+04 + 4 -1.491023e+04 + 5 -1.480677e+04 + 6 -1.477849e+04 + 7 -1.477005e+04 + 8 -1.476675e+04 + 9 -1.476478e+04 + 10 -1.476318e+04 + 20 -1.475020e+04 + 30 -1.474023e+04 + 40 -1.473255e+04 + 50 -1.472662e+04 + 60 -1.472201e+04 + 70 -1.471839e+04 + 80 -1.471554e+04 + 90 -1.471326e+04 + 100 -1.471142e+04 + 110 -1.470993e+04 + 120 -1.470871e+04 + 130 -1.470770e+04 + 140 -1.470686e+04 + 150 -1.470616e+04 + 160 -1.470557e+04 + 170 -1.470506e+04 + 180 -1.470463e+04 + 190 -1.470426e+04 + 200 -1.470394e+04 + 210 -1.470366e+04 + 220 -1.470342e+04 + 230 -1.470321e+04 + 240 -1.470302e+04 + 250 -1.470285e+04 + 260 -1.470270e+04 + 270 -1.470257e+04 + 280 -1.470245e+04 + 290 -1.470234e+04 + 300 -1.470225e+04 + 310 -1.470216e+04 + 320 -1.470208e+04 + 330 -1.470201e+04 + 340 -1.470195e+04 + 350 -1.470189e+04 + 360 -1.470183e+04 + 370 -1.470178e+04 + 380 -1.470173e+04 + 390 -1.470169e+04 + 400 -1.470165e+04 + 410 -1.470162e+04 + 420 -1.470158e+04 + 430 -1.470155e+04 + 440 -1.470152e+04 + 450 -1.470149e+04 + 460 -1.470147e+04 + 470 -1.470144e+04 + 480 -1.470142e+04 + 490 -1.470140e+04 + 500 -1.470138e+04 + 510 -1.470136e+04 + 520 -1.470134e+04 + 530 -1.470132e+04 + 540 -1.470131e+04 + 550 -1.470129e+04 + 560 -1.470128e+04 + 570 -1.470127e+04 + 580 -1.470125e+04 + 590 -1.470124e+04 + 600 -1.470123e+04 + 610 -1.470122e+04 + 620 -1.470121e+04 + 630 -1.470120e+04 + 640 -1.470119e+04 + 650 -1.470118e+04 + 660 -1.470117e+04 + 670 -1.470116e+04 + 680 -1.470116e+04 + 690 -1.470115e+04 + 700 -1.470114e+04 + 710 -1.470113e+04 + 720 -1.470113e+04 + 730 -1.470112e+04 + 740 -1.470112e+04 + 750 -1.470111e+04 + 760 -1.470110e+04 + 770 -1.470110e+04 + 780 -1.470109e+04 + 790 -1.470109e+04 + 800 -1.470108e+04 + 810 -1.470108e+04 + 820 -1.470108e+04 + 830 -1.470107e+04 + 840 -1.470107e+04 + 850 -1.470106e+04 + 860 -1.470106e+04 + 870 -1.470106e+04 + 880 -1.470105e+04 + 890 -1.470105e+04 + 900 -1.470105e+04 + 910 -1.470104e+04 + 920 -1.470104e+04 + 930 -1.470104e+04 + 940 -1.470103e+04 + 950 -1.470103e+04 + 960 -1.470103e+04 + 970 -1.470103e+04 + 980 -1.470102e+04 + 990 -1.470102e+04 + 1000 -1.470102e+04 + 1010 -1.470102e+04 + 1020 -1.470102e+04 + 1030 -1.470101e+04 + 1040 -1.470101e+04 + 1050 -1.470101e+04 + 1060 -1.470101e+04 + 1070 -1.470101e+04 + 1080 -1.470101e+04 + 1090 -1.470100e+04 + 1100 -1.470100e+04 + 1110 -1.470100e+04 - 0.794377 seconds (168.12 k allocations: 15.640 MiB, 0.80% gc time) + 0.744170 seconds (134.31 k allocations: 13.430 MiB, 1.57% gc time) - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.2808 0.279454 0.232256; 0.279454 0.284312 0.219977; 0.232256 0.219977 0.213052], [0.717243 0.661816 0.674158; 0.661816 0.715193 0.581746; 0.674158 0.581746 0.783965]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,3), ([0.281394 0.280169 0.232564; 0.280169 0.285001 0.220511; 0.232564 0.220511 0.213147], [0.717018 0.661467 0.674155; 0.661467 0.714877 0.581502; 0.674155 0.581502 0.784149]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf) @@ -1128,8 +1016,8 @@ h, hse = heritability(trait57_model.Σ, Σcov) 2×3 Array{Float64,2}: - 0.281351 0.284453 0.213689 - 0.0778252 0.077378 0.084084 + 0.281842 0.285036 0.213725 + 0.0777056 0.0772501 0.0840114 @@ -1145,7 +1033,7 @@ traitall_model = VarianceComponentModel(cg10kdata_rotated) @time mle_fs!(traitall_model, cg10kdata_rotated; solver=:Ipopt, verbose=true) ``` - This is Ipopt version 3.12.4, running with linear solver mumps. + This is Ipopt version 3.12.10, running with linear solver mumps. NOTE: Other linear solvers might be more efficient (see Ipopt documentation). Number of nonzeros in equality constraint Jacobian...: 0 @@ -1163,43 +1051,47 @@ traitall_model = VarianceComponentModel(cg10kdata_rotated) inequality constraints with only upper bounds: 0 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls - 0 1.3113371e+05 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 - 5 8.2233766e+04 0.00e+00 6.03e+02 -11.0 2.32e+00 - 1.00e+00 1.00e+00f 1 MaxS - 10 1.1960260e+05 0.00e+00 8.76e+02 -11.0 6.20e+01 -5.4 1.00e+00 1.00e+00h 1 MaxS - 15 2.4416551e+05 0.00e+00 2.50e+02 -11.0 8.69e+02 -7.8 1.00e+00 1.00e+00f 1 MaxS + 0 1.3111983e+05 0.00e+00 1.00e+02 0.0 0.00e+00 - 0.00e+00 0.00e+00 0 + 5 8.2228529e+04 0.00e+00 6.03e+02 -11.0 2.42e+00 - 1.00e+00 1.00e+00f 1 MaxS + 10 1.2570490e+05 0.00e+00 9.38e+02 -11.0 6.72e+01 -5.4 1.00e+00 1.00e+00h 1 MaxS - DomainError: - log will only return a complex result if called with a complex argument. Try log(complex(x)). + PosDefException: matrix is not positive definite; Cholesky factorization failed. Stacktrace: - [1] nan_dom_err at ./math.jl:300 [inlined] + [1] chkposdef at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/lapack.jl:50 [inlined] + + [2] sygvd!(::Int64, ::Char, ::Char, ::Array{Float64,2}, ::Array{Float64,2}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/lapack.jl:5075 + + [3] eigen! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/symmetric.jl:646 [inlined] + + [4] eigen(::Symmetric{Float64,Array{Float64,2}}, ::Symmetric{Float64,Array{Float64,2}}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/eigen.jl:383 - [2] log at ./math.jl:419 [inlined] + [5] TwoVarCompModelRotate(::VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}) at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/VarianceComponentModels.jl:119 - [3] logdet(::Array{Float64,2}) at ./linalg/generic.jl:1244 + [6] logpdf at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/two_variance_component.jl:62 [inlined] - [4] VarianceComponentModels.TwoVarCompModelRotate(::VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}) at /Users/huazhou/.julia/v0.6/VarianceComponentModels/src/VarianceComponentModels.jl:127 + [7] eval_f(::VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}, ::Array{Float64,1}) at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/two_variance_component.jl:731 - [5] eval_f(::VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentModels.VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}, ::Array{Float64,1}) at /Users/huazhou/.julia/v0.6/VarianceComponentModels/src/two_variance_component.jl:683 + [8] (::getfield(Ipopt, Symbol("#eval_f_cb#6")){VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}})(::Array{Float64,1}) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/MPB_wrapper.jl:64 - [6] (::Ipopt.#eval_f_cb#4{VarianceComponentModels.TwoVarCompOptProb{VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}},VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}},Array{Float64,2},Array{Float64,1},VarianceComponentModels.VarianceComponentAuxData{Array{Float64,2},Array{Float64,1}}}})(::Array{Float64,1}) at /Users/huazhou/.julia/v0.6/Ipopt/src/IpoptSolverInterface.jl:53 + [9] eval_f_wrapper(::Int32, ::Ptr{Float64}, ::Int32, ::Ptr{Float64}, ::Ptr{Nothing}) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/Ipopt.jl:128 - [7] eval_f_wrapper(::Int32, ::Ptr{Float64}, ::Int32, ::Ptr{Float64}, ::Ptr{Void}) at /Users/huazhou/.julia/v0.6/Ipopt/src/Ipopt.jl:89 + [10] solveProblem(::Ipopt.IpoptProblem) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/Ipopt.jl:346 - [8] solveProblem(::Ipopt.IpoptProblem) at /Users/huazhou/.julia/v0.6/Ipopt/src/Ipopt.jl:304 + [11] optimize!(::Ipopt.IpoptMathProgModel) at /Users/juhyun-kim/.julia/packages/Ipopt/f6QJl/src/MPB_wrapper.jl:141 - [9] optimize!(::Ipopt.IpoptMathProgModel) at /Users/huazhou/.julia/v0.6/Ipopt/src/IpoptSolverInterface.jl:120 + [12] #mle_fs!#27(::Int64, ::Symbol, ::Symbol, ::Bool, ::Function, ::VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}) at /Users/juhyun-kim/.julia/packages/VarianceComponentModels/Mh6LK/src/two_variance_component.jl:949 - [10] #mle_fs!#29(::Int64, ::Symbol, ::Symbol, ::Bool, ::Function, ::VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}) at /Users/huazhou/.julia/v0.6/VarianceComponentModels/src/two_variance_component.jl:893 + [13] (::getfield(VarianceComponentModels, Symbol("#kw##mle_fs!")))(::NamedTuple{(:solver, :verbose),Tuple{Symbol,Bool}}, ::typeof(mle_fs!), ::VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}) at ./none:0 - [11] (::VarianceComponentModels.#kw##mle_fs!)(::Array{Any,1}, ::VarianceComponentModels.#mle_fs!, ::VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}, ::VarianceComponentModels.TwoVarCompVariateRotate{Float64,Array{Float64,2},Array{Float64,2}}) at ./:0 + [14] top-level scope at util.jl:156 - [12] include_string(::String, ::String) at ./loading.jl:515 + [15] top-level scope at In[54]:3 From the output we can see the Fisher scoring algorithm ran into some numerical issues. Let's try the **MM algorithm**. @@ -1216,133 +1108,132 @@ traitall_model = VarianceComponentModel(cg10kdata_rotated) MM Algorithm Iter Objective -------- ------------- - 0 -1.311337e+05 - 1 -8.002108e+04 - 2 -5.806935e+04 - 3 -4.926111e+04 - 4 -4.611059e+04 - 5 -4.511606e+04 - 6 -4.482679e+04 - 7 -4.474294e+04 - 8 -4.471496e+04 - 9 -4.470174e+04 - 10 -4.469246e+04 - 20 -4.462243e+04 - 30 -4.456888e+04 - 40 -4.452774e+04 - 50 -4.449601e+04 - 60 -4.447134e+04 - 70 -4.445199e+04 - 80 -4.443665e+04 - 90 -4.442436e+04 - 100 -4.441442e+04 - 110 -4.440630e+04 - 120 -4.439961e+04 - 130 -4.439405e+04 - 140 -4.438938e+04 - 150 -4.438544e+04 - 160 -4.438210e+04 - 170 -4.437923e+04 - 180 -4.437676e+04 - 190 -4.437463e+04 - 200 -4.437277e+04 - 210 -4.437115e+04 - 220 -4.436972e+04 - 230 -4.436846e+04 - 240 -4.436735e+04 - 250 -4.436636e+04 - 260 -4.436548e+04 - 270 -4.436469e+04 - 280 -4.436399e+04 - 290 -4.436335e+04 - 300 -4.436278e+04 - 310 -4.436226e+04 - 320 -4.436179e+04 - 330 -4.436137e+04 - 340 -4.436098e+04 - 350 -4.436063e+04 - 360 -4.436030e+04 - 370 -4.436001e+04 - 380 -4.435974e+04 - 390 -4.435949e+04 - 400 -4.435926e+04 - 410 -4.435905e+04 - 420 -4.435886e+04 - 430 -4.435868e+04 - 440 -4.435851e+04 - 450 -4.435836e+04 - 460 -4.435822e+04 - 470 -4.435809e+04 - 480 -4.435797e+04 - 490 -4.435785e+04 - 500 -4.435775e+04 - 510 -4.435765e+04 - 520 -4.435756e+04 - 530 -4.435747e+04 - 540 -4.435739e+04 - 550 -4.435732e+04 - 560 -4.435725e+04 - 570 -4.435718e+04 - 580 -4.435712e+04 - 590 -4.435706e+04 - 600 -4.435701e+04 - 610 -4.435696e+04 - 620 -4.435691e+04 - 630 -4.435687e+04 - 640 -4.435683e+04 - 650 -4.435679e+04 - 660 -4.435675e+04 - 670 -4.435671e+04 - 680 -4.435668e+04 - 690 -4.435665e+04 - 700 -4.435662e+04 - 710 -4.435659e+04 - 720 -4.435657e+04 - 730 -4.435654e+04 - 740 -4.435652e+04 - 750 -4.435649e+04 - 760 -4.435647e+04 - 770 -4.435645e+04 - 780 -4.435643e+04 - 790 -4.435642e+04 - 800 -4.435640e+04 - 810 -4.435638e+04 - 820 -4.435637e+04 - 830 -4.435635e+04 - 840 -4.435634e+04 - 850 -4.435633e+04 - 860 -4.435631e+04 - 870 -4.435630e+04 - 880 -4.435629e+04 - 890 -4.435628e+04 - 900 -4.435627e+04 - 910 -4.435626e+04 - 920 -4.435625e+04 - 930 -4.435624e+04 - 940 -4.435623e+04 - 950 -4.435622e+04 - 960 -4.435621e+04 - 970 -4.435621e+04 - 980 -4.435620e+04 - 990 -4.435619e+04 - 1000 -4.435619e+04 - 1010 -4.435618e+04 - 1020 -4.435617e+04 - 1030 -4.435617e+04 - 1040 -4.435616e+04 - 1050 -4.435616e+04 - 1060 -4.435615e+04 - 1070 -4.435615e+04 - 1080 -4.435614e+04 - 1090 -4.435614e+04 + 0 -1.311198e+05 + 1 -8.001595e+04 + 2 -5.806836e+04 + 3 -4.926167e+04 + 4 -4.611163e+04 + 5 -4.511722e+04 + 6 -4.482795e+04 + 7 -4.474405e+04 + 8 -4.471603e+04 + 9 -4.470277e+04 + 10 -4.469345e+04 + 20 -4.462308e+04 + 30 -4.456930e+04 + 40 -4.452802e+04 + 50 -4.449620e+04 + 60 -4.447148e+04 + 70 -4.445209e+04 + 80 -4.443672e+04 + 90 -4.442442e+04 + 100 -4.441448e+04 + 110 -4.440635e+04 + 120 -4.439966e+04 + 130 -4.439409e+04 + 140 -4.438943e+04 + 150 -4.438549e+04 + 160 -4.438215e+04 + 170 -4.437928e+04 + 180 -4.437682e+04 + 190 -4.437469e+04 + 200 -4.437283e+04 + 210 -4.437121e+04 + 220 -4.436978e+04 + 230 -4.436853e+04 + 240 -4.436742e+04 + 250 -4.436643e+04 + 260 -4.436555e+04 + 270 -4.436476e+04 + 280 -4.436406e+04 + 290 -4.436342e+04 + 300 -4.436285e+04 + 310 -4.436234e+04 + 320 -4.436187e+04 + 330 -4.436144e+04 + 340 -4.436106e+04 + 350 -4.436071e+04 + 360 -4.436039e+04 + 370 -4.436009e+04 + 380 -4.435982e+04 + 390 -4.435957e+04 + 400 -4.435935e+04 + 410 -4.435914e+04 + 420 -4.435895e+04 + 430 -4.435877e+04 + 440 -4.435860e+04 + 450 -4.435845e+04 + 460 -4.435831e+04 + 470 -4.435818e+04 + 480 -4.435806e+04 + 490 -4.435794e+04 + 500 -4.435784e+04 + 510 -4.435774e+04 + 520 -4.435765e+04 + 530 -4.435757e+04 + 540 -4.435749e+04 + 550 -4.435741e+04 + 560 -4.435734e+04 + 570 -4.435728e+04 + 580 -4.435722e+04 + 590 -4.435716e+04 + 600 -4.435711e+04 + 610 -4.435706e+04 + 620 -4.435701e+04 + 630 -4.435696e+04 + 640 -4.435692e+04 + 650 -4.435688e+04 + 660 -4.435685e+04 + 670 -4.435681e+04 + 680 -4.435678e+04 + 690 -4.435675e+04 + 700 -4.435672e+04 + 710 -4.435669e+04 + 720 -4.435666e+04 + 730 -4.435664e+04 + 740 -4.435662e+04 + 750 -4.435659e+04 + 760 -4.435657e+04 + 770 -4.435655e+04 + 780 -4.435653e+04 + 790 -4.435652e+04 + 800 -4.435650e+04 + 810 -4.435648e+04 + 820 -4.435647e+04 + 830 -4.435645e+04 + 840 -4.435644e+04 + 850 -4.435643e+04 + 860 -4.435641e+04 + 870 -4.435640e+04 + 880 -4.435639e+04 + 890 -4.435638e+04 + 900 -4.435637e+04 + 910 -4.435636e+04 + 920 -4.435635e+04 + 930 -4.435634e+04 + 940 -4.435633e+04 + 950 -4.435633e+04 + 960 -4.435632e+04 + 970 -4.435631e+04 + 980 -4.435630e+04 + 990 -4.435630e+04 + 1000 -4.435629e+04 + 1010 -4.435628e+04 + 1020 -4.435628e+04 + 1030 -4.435627e+04 + 1040 -4.435627e+04 + 1050 -4.435626e+04 + 1060 -4.435626e+04 + 1070 -4.435625e+04 + 1080 -4.435625e+04 - 3.551301 seconds (178.42 k allocations: 70.115 MiB, 0.42% gc time) + 3.314963 seconds (131.35 k allocations: 65.449 MiB, 0.65% gc time) - (-44356.138529861186, VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,13), ([0.272384 0.190358 … -0.128222 -0.0980655; 0.190358 0.21692 … -0.0689912 -0.0444349; … ; -0.128222 -0.0689912 … 0.118227 0.0909188; -0.0980655 -0.0444349 … 0.0909188 0.107456], [0.724562 0.56992 … -0.0590518 -0.124939; 0.56992 0.782639 … 0.0238629 0.0475408; … ; -0.0590518 0.0238629 … 0.880671 0.550889; -0.124939 0.0475408 … 0.550889 0.891929]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf), ([0.0111619 0.0131088 … 0.0128956 0.0127641; 0.0131091 0.0151759 … 0.017162 0.0171466; … ; 0.0128956 0.017162 … 0.0173994 0.0182002; 0.0127643 0.0171461 … 0.0182003 0.0187848], [0.0112235 0.0133094 … 0.0130111 0.0127861; 0.01331 0.0158262 … 0.017867 0.017798; … ; 0.013011 0.0178666 … 0.0179487 0.0187579; 0.012786 0.0177975 … 0.0187578 0.0193328]), [0.000124587 7.24074e-5 … -3.35716e-7 -1.40982e-5; 7.24411e-5 0.000171849 … -2.05381e-5 -3.17975e-6; … ; -3.60221e-7 -2.05683e-5 … 0.000351859 -1.5168e-5; -1.40799e-5 -3.16738e-6 … -1.51641e-5 0.000373756], Array{Float64}(0,13), Array{Float64}(0,0)) + (-44356.24416259489, VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}(Array{Float64}(0,13), ([0.272112 0.190023 … -0.128464 -0.0976418; 0.190023 0.216814 … -0.0687833 -0.04341; … ; -0.128464 -0.0687833 … 0.116994 0.0900933; -0.0976418 -0.04341 … 0.0900933 0.105876], [0.725183 0.570497 … -0.0589748 -0.125486; 0.570497 0.783023 … 0.0235685 0.0464638; … ; -0.0589748 0.0235685 … 0.882056 0.551829; -0.125486 0.0464638 … 0.551829 0.893642]), Array{Float64}(0,0), Char[], Float64[], -Inf, Inf), ([0.0111603 0.013107 … 0.0128915 0.0127587; 0.0131027 0.0151631 … 0.017153 0.0171359; … ; 0.0128908 0.017153 … 0.0174176 0.018212; 0.0127586 0.0171361 … 0.0182122 0.0188002], [0.0112235 0.0133041 … 0.0130038 0.0127778; 0.0133005 0.0158053 … 0.0178518 0.0177823; … ; 0.0130043 0.0178518 … 0.0179557 0.0187638; 0.0127775 0.0177823 … 0.0187637 0.0193477]), [0.000124552 7.23469e-5 … -3.6584e-7 -1.40474e-5; 7.23585e-5 0.00017168 … -2.04611e-5 -3.18804e-6; … ; -3.70686e-7 -2.04634e-5 … 0.000352082 -1.46096e-5; -1.4039e-5 -3.1795e-6 … -1.46073e-5 0.000374334], Array{Float64}(0,13), Array{Float64}(0,0)) @@ -1352,7 +1243,7 @@ It converges after ~1000 iterations. ```julia -#using JLD -#@save "copd.jld" -#whos() +#using JLD2, FileIO +#save("copd.jld2") +#varinfo() ``` diff --git a/docs/src/man/mle_reml.md b/docs/src/man/mle_reml.md index a9c4a10..d54c46f 100644 --- a/docs/src/man/mle_reml.md +++ b/docs/src/man/mle_reml.md @@ -8,16 +8,14 @@ Machine information versioninfo() ``` - Julia Version 0.6.0 - Commit 903644385b (2017-06-19 13:05 UTC) + Julia Version 1.1.0 + Commit 80516ca202 (2019-01-21 21:24 UTC) Platform Info: - OS: macOS (x86_64-apple-darwin13.4.0) - CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz + OS: macOS (x86_64-apple-darwin14.5.0) + CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz WORD_SIZE: 64 - BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell) - LAPACK: libopenblas64_ LIBM: libopenlibm - LLVM: libLLVM-3.9.1 (ORCJIT, haswell) + LLVM: libLLVM-6.0.1 (ORCJIT, skylake) ## Demo data @@ -27,7 +25,8 @@ For demonstration, we generate a random data set. ```julia # generate data from a d-variate response variane component model -srand(123) +using Random, LinearAlgebra +Random.seed!(123) n = 1000 # no. observations d = 2 # dimension of responses m = 2 # no. variance components @@ -40,23 +39,23 @@ B = ones(p, d) V = ntuple(x -> zeros(n, n), m) for i = 1:m-1 Vi = randn(n, 50) - copy!(V[i], Vi * Vi') + copyto!(V[i], Vi * Vi') end -copy!(V[m], eye(n)) # last covarianec matrix is idendity +copyto!(V[m], Matrix(I, n, n)) # last covarianec matrix is idendity # a tuple of m d-by-d variance component parameters Σ = ntuple(x -> zeros(d, d), m) for i in 1:m Σi = randn(d, d) - copy!(Σ[i], Σi' * Σi) + copyto!(Σ[i], Σi' * Σi) end # form overall nd-by-nd covariance matrix Ω Ω = zeros(n * d, n * d) for i = 1:m Ω += kron(Σ[i], V[i]) end -Ωchol = cholfact(Ω) +Ωchol = cholesky(Ω) # n-by-d responses -Y = X * B + reshape(Ωchol[:L] * randn(n*d), n, d); +Y = X * B + reshape(Ωchol.L * randn(n*d), n, d); ``` ## Maximum likelihood estimation (MLE) @@ -73,16 +72,13 @@ To find the MLE of parameters $(B,\Sigma_1,\ldots,\Sigma_m)$, we take 3 steps: ```julia using VarianceComponentModels vcdata = VarianceComponentVariate(Y, X, V) -fieldnames(vcdata) +fieldnames(typeof(vcdata)) ``` - 3-element Array{Symbol,1}: - :Y - :X - :V + (:Y, :X, :V) @@ -98,20 +94,13 @@ When constructed from a `VarianceComponentVariate` instance, the mean parameters ```julia vcmodel = VarianceComponentModel(vcdata) -fieldnames(vcmodel) +fieldnames(typeof(vcmodel)) ``` - 7-element Array{Symbol,1}: - :B - :Σ - :A - :sense - :b - :lb - :ub + (:B, :Σ, :A, :sense, :b, :lb, :ub) @@ -123,7 +112,7 @@ vcmodel - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf) @@ -150,6 +139,13 @@ vcmodel_mle = deepcopy(vcmodel) Iter Objective -------- ------------- 0 -6.253551e+03 + + ****************************************************************************** + This program contains Ipopt, a library for large-scale nonlinear optimization. + Ipopt is released as open source code under the Eclipse Public License (EPL). + For more information visit http://projects.coin-or.org/Ipopt + ****************************************************************************** + 1 -3.881454e+03 2 -3.853179e+03 3 -3.846525e+03 @@ -161,7 +157,7 @@ vcmodel_mle = deepcopy(vcmodel) 9 -3.844374e+03 10 -3.844373e+03 - 0.290970 seconds (10.45 k allocations: 24.036 MiB, 4.73% gc time) + 5.031460 seconds (11.29 M allocations: 568.015 MiB, 4.78% gc time) The output of `fit_mle!` contains @@ -176,7 +172,7 @@ logl - -3844.3731814180805 + -3844.3731814180887 @@ -184,20 +180,13 @@ logl ```julia -fieldnames(vcmodel_mle) +fieldnames(typeof(vcmodel_mle)) ``` - 7-element Array{Symbol,1}: - :B - :Σ - :A - :sense - :b - :lb - :ub + (:B, :Σ, :A, :sense, :b, :lb, :ub) @@ -209,7 +198,7 @@ vcmodel_mle - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955346 1.01632], ([0.380637 -0.305465; -0.305465 4.51938], [1.84009 0.265569; 0.265569 2.17275]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955346 1.01632], ([0.380637 -0.305465; -0.305465 4.51938], [1.84009 0.265569; 0.265569 2.17275]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf) @@ -260,8 +249,8 @@ Bse 2×2 Array{Float64,2}: - 0.042559 0.0487086 - 0.0430588 0.049178 + 0.0425562 0.0483834 + 0.0430596 0.0492809 @@ -276,10 +265,10 @@ Bcov 4×4 Array{Float64,2}: - 0.00181127 -1.98035e-5 0.000240705 -2.59506e-6 - -1.98035e-5 0.00185406 -2.59506e-6 0.000247285 - 0.000240705 -2.59506e-6 0.00237252 -2.63542e-5 - -2.59506e-6 0.000247285 -2.63542e-5 0.00241848 + 0.00181103 -1.96485e-5 0.000243441 -4.38252e-6 + -1.96485e-5 0.00185413 -4.38252e-6 0.000246407 + 0.000243441 -4.38252e-6 0.00234096 -5.73331e-6 + -4.38252e-6 0.000246407 -5.73331e-6 0.00242861 @@ -311,7 +300,8 @@ vcmodel_reml = deepcopy(vcmodel) 9 -3.846631e+03 10 -3.846630e+03 - 0 + 0.726373 seconds (388.90 k allocations: 82.673 MiB, 13.22% gc time) + The output of `fit_reml!` contains @@ -325,7 +315,7 @@ logl - -3844.3777179025096 + -3844.3777179025055 @@ -333,20 +323,13 @@ logl ```julia -fieldnames(vcmodel_reml) +fieldnames(typeof(vcmodel_reml)) ``` - 7-element Array{Symbol,1}: - :B - :Σ - :A - :sense - :b - :lb - :ub + (:B, :Σ, :A, :sense, :b, :lb, :ub) @@ -358,11 +341,11 @@ vcmodel_reml - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955345 1.01632], ([0.380594 -0.305485; -0.305485 4.51994], [1.84285 0.261963; 0.261963 2.17842]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.092 1.04727; 0.955345 1.01632], ([0.380594 -0.305485; -0.305485 4.51994], [1.84285 0.261963; 0.261963 2.17842]), Array{Float64}(0,4), Char[], Float64[], -Inf, Inf) -* standard errors of the estimated varianec component parameters +* standard errors of the estimated variance component parameters ```julia @@ -409,8 +392,8 @@ Bse 2×2 Array{Float64,2}: - 0.0425909 0.0487744 - 0.043091 0.0492444 + 0.0425881 0.0484485 + 0.0430919 0.0493475 @@ -425,10 +408,10 @@ Bcov 4×4 Array{Float64,2}: - 0.00181398 -1.98331e-5 0.000237127 -2.55589e-6 - -1.98331e-5 0.00185683 -2.55589e-6 0.000243624 - 0.000237127 -2.55589e-6 0.00237894 -2.6426e-5 - -2.55589e-6 0.000243624 -2.6426e-5 0.00242501 + 0.00181375 -1.96783e-5 0.000239868 -4.34611e-6 + -1.96783e-5 0.00185691 -4.34611e-6 0.000242745 + 0.000239868 -4.34611e-6 0.00234726 -5.73082e-6 + -4.34611e-6 0.000242745 -5.73082e-6 0.00243518 @@ -441,18 +424,13 @@ In general the optimization algorithm needs to invert the $nd$ by $nd$ overall c ```julia vcdatarot = TwoVarCompVariateRotate(vcdata) -fieldnames(vcdatarot) +fieldnames(typeof(vcdatarot)) ``` - 5-element Array{Symbol,1}: - :Yrot - :Xrot - :eigval - :eigvec - :logdetV2 + (:Yrot, :Xrot, :eigval, :eigvec, :logdetV2) @@ -480,7 +458,7 @@ vcmodel_mm = deepcopy(vcmodel) 9 -3.844374e+03 10 -3.844373e+03 - 0.018754 seconds (9.15 k allocations: 680.172 KiB) + 0.055578 seconds (21.91 k allocations: 1.394 MiB) @@ -520,7 +498,7 @@ vcmodel_ipopt = deepcopy(vcmodel) @time mle_fs!(vcmodel_ipopt, vcdatarot; solver=:Ipopt, maxiter=1000, verbose=true); ``` - This is Ipopt version 3.12.4, running with linear solver mumps. + This is Ipopt version 3.12.10, running with linear solver mumps. NOTE: Other linear solvers might be more efficient (see Ipopt documentation). Number of nonzeros in equality constraint Jacobian...: 0 @@ -551,16 +529,16 @@ vcmodel_ipopt = deepcopy(vcmodel) iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls 50 3.8443732e+03 0.00e+00 6.76e-06 -11.0 4.08e-07 - 1.00e+00 1.00e+00f 1 MaxSA 55 3.8443732e+03 0.00e+00 1.83e-06 -11.0 1.11e-07 - 1.00e+00 1.00e+00f 1 MaxSA - 60 3.8443732e+03 0.00e+00 4.97e-07 -11.0 3.00e-08 - 1.00e+00 1.00e+00f 1 MaxSA + 60 3.8443732e+03 0.00e+00 4.97e-07 -11.0 3.00e-08 - 1.00e+00 1.00e+00h 1 MaxSA Number of Iterations....: 63 (scaled) (unscaled) - Objective...............: 3.4496886481728075e+02 3.8443731733053696e+03 - Dual infeasibility......: 2.2693631692678575e-07 2.5290047242499938e-06 + Objective...............: 3.4496886481728791e+02 3.8443731733053728e+03 + Dual infeasibility......: 2.2693631660531264e-07 2.5290047206674095e-06 Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00 Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00 - Overall NLP error.......: 2.2693631692678575e-07 2.5290047242499938e-06 + Overall NLP error.......: 2.2693631660531264e-07 2.5290047206674095e-06 Number of objective function evaluations = 64 @@ -570,11 +548,12 @@ vcmodel_ipopt = deepcopy(vcmodel) Number of equality constraint Jacobian evaluations = 0 Number of inequality constraint Jacobian evaluations = 0 Number of Lagrangian Hessian evaluations = 63 - Total CPU secs in IPOPT (w/o function evaluations) = 0.020 - Total CPU secs in NLP function evaluations = 0.256 + Total CPU secs in IPOPT (w/o function evaluations) = 1.739 + Total CPU secs in NLP function evaluations = 0.293 EXIT: Solved To Acceptable Level. - + 2.745554 seconds (4.30 M allocations: 210.935 MiB, 2.63% gc time) + ```julia @@ -621,6 +600,7 @@ vcmodel_knitro.Σ ## Starting point + Here are a few strategies for successful optimization. * For $d>1$ (multivariate response), initialize $B, \Sigma$ from univariate estimates. @@ -629,6 +609,8 @@ Here are a few strategies for successful optimization. ## Constrained estimation of `B` + + Many applications invoke constraints on the mean parameters `B`. For demonstration, we enforce `B[1,1]=B[1,2]` and all entries of `B` are within [0, 2]. @@ -646,7 +628,7 @@ vcmodel_constr - VarianceComponentModels.VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), [1.0 0.0 -1.0 0.0], '=', 0.0, 0.0, 2.0) + VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([0.0 0.0; 0.0 0.0], ([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0]), [1.0 0.0 -1.0 0.0], '=', 0.0, 0.0, 2.0) @@ -674,25 +656,18 @@ We first try the MM algorithm. 9 -3.844650e+03 10 -3.844650e+03 - 0.031954 seconds (10.70 k allocations: 781.828 KiB) + 0.185885 seconds (179.51 k allocations: 9.295 MiB) ```julia -fieldnames(vcmodel_constr) +fieldnames(typeof(vcmodel_constr)) ``` - 7-element Array{Symbol,1}: - :B - :Σ - :A - :sense - :b - :lb - :ub + (:B, :Σ, :A, :sense, :b, :lb, :ub) @@ -737,7 +712,7 @@ vcmodel_constr @time mle_fs!(vcmodel_constr, vcdatarot; solver=:Ipopt, maxiter=1000, verbose=true); ``` - This is Ipopt version 3.12.4, running with linear solver mumps. + This is Ipopt version 3.12.10, running with linear solver mumps. NOTE: Other linear solvers might be more efficient (see Ipopt documentation). Number of nonzeros in equality constraint Jacobian...: 0 @@ -773,11 +748,11 @@ vcmodel_constr Number of Iterations....: 63 (scaled) (unscaled) - Objective...............: 3.4484507551949008e+02 3.8446498170293403e+03 - Dual infeasibility......: 2.2694405349430929e-07 2.5301808715939735e-06 + Objective...............: 3.4484507551949685e+02 3.8446498170293398e+03 + Dual infeasibility......: 2.2694405475622814e-07 2.5301808856629548e-06 Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00 Complementarity.........: 0.0000000000000000e+00 0.0000000000000000e+00 - Overall NLP error.......: 2.2694405349430929e-07 2.5301808715939735e-06 + Overall NLP error.......: 2.2694405475622814e-07 2.5301808856629548e-06 Number of objective function evaluations = 64 @@ -787,11 +762,12 @@ vcmodel_constr Number of equality constraint Jacobian evaluations = 0 Number of inequality constraint Jacobian evaluations = 0 Number of Lagrangian Hessian evaluations = 63 - Total CPU secs in IPOPT (w/o function evaluations) = 0.016 - Total CPU secs in NLP function evaluations = 0.417 + Total CPU secs in IPOPT (w/o function evaluations) = 0.028 + Total CPU secs in NLP function evaluations = 0.634 EXIT: Solved To Acceptable Level. - + 0.760983 seconds (102.63 k allocations: 8.135 MiB) + ```julia