default mtry and "projection type" for low-dimensional data

[jovo]

i never seriously considered defaults for really low dimensions.
it occurs to me that it might make sense for mtry to never be much smaller than 100,
and rather than trinary for continuous to be the default,
otherwise, we don't get enough strength from the trees.

thoughts @falkben @MrAE @ttomita @jbrowne6 @megh1241

[falkben]

This would make our defaults vastly different than sklearns (just pointing it out). mtry at a high value makes it slow -- perhaps with low dimensions speed doesn't matter as much though for low dimensions, the way we are currently sampling each mtry (w/ replacement), I think we would get a lot of duplicated feature combinations which wouldn't add anything but would slow us down. we probably want to: 1. fix it so that each mtry samples w/o replacement from the matrix p x d. 2. add continuous rerf. 3. run experiments to determine appropriate default mtry for low/med/high dimensions

…

On Sun, May 19, 2019 at 6:51 PM joshua vogelstein ***@***.***> wrote: i never seriously considered defaults for really low dimensions. it occurs to me that it might make sense for mtry to never be much smaller than 100, and rather than trinary for continuous to be the default, otherwise, we don't get enough strength from the trees. thoughts @falkben <https://github.com/falkben> @MrAE <https://github.com/MrAE> @ttomita <https://github.com/ttomita> @jbrowne6 <https://github.com/jbrowne6> @megh1241 <https://github.com/megh1241> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#270>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAE7NZ2B7PGSBTHNFEMQPITPWHKXPANCNFSM4HN5RXKQ> .

[jovo]

yah, this would depend on continuous rerf, and yah, this is part of a larger effort to understand the hyperparameters...

…

On Mon, May 20, 2019 at 10:14 AM Ben Falk ***@***.***> wrote: This would make our defaults vastly different than sklearns (just pointing it out). mtry at a high value makes it slow -- perhaps with low dimensions speed doesn't matter as much though for low dimensions, the way we are currently sampling each mtry (w/ replacement), I think we would get a lot of duplicated feature combinations which wouldn't add anything but would slow us down. we probably want to: 1. fix it so that each mtry samples w/o replacement from the matrix p x d. 2. add continuous rerf. 3. run experiments to determine appropriate default mtry for low/med/high dimensions On Sun, May 19, 2019 at 6:51 PM joshua vogelstein < ***@***.***> wrote: > i never seriously considered defaults for really low dimensions. > it occurs to me that it might make sense for mtry to never be much smaller > than 100, > and rather than trinary for continuous to be the default, > otherwise, we don't get enough strength from the trees. > > thoughts @falkben <https://github.com/falkben> @MrAE > <https://github.com/MrAE> @ttomita <https://github.com/ttomita> @jbrowne6 > <https://github.com/jbrowne6> @megh1241 <https://github.com/megh1241> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #270 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAE7NZ2B7PGSBTHNFEMQPITPWHKXPANCNFSM4HN5RXKQ > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#270>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAKG4VXZNXJJPUCLFJUAQ3PWKW3JANCNFSM4HN5RXKQ> .

-- the glass is all full: half water, half air. neurodata.io