NIST-based entropy calculation should not be the default

[Yggdrasil]

The wikipedia article mentions the following:

Later research into human-selected password entropy using newly available real world data has demonstrated that the NIST scheme does not provide a valid metric for entropy estimation of human-selected passwords.

The linked blog post and research paper clarify this statement. NIST relies on the Shannon entropy model, which has been proven to incorrectly model real-world password entropy considering use by humans and real-life attack methods. Having this method as the default is therefore unwise and should be deprecated.

I'm not sure how the Wolfram Alpha calculation is constructed, so cannot say if it performs better, but from my personal tests it also has some weak spots. A method like Dropbox's zxcvbn algorithm seems to be a better match, although it hasn't been around long enough to be properly reviewed. Still it seems like a reasonable method that would be good to implement since it can be used in combination with the front-end javascript library. Something to consider though.

[rchouinard]

Thanks for bringing this to my attention. When I was researching entropy calculation methods, I found that there are few standard methods and almost all are disputed. I chose NIST as the default simply because it had, well, NIST's stamp of approval. I'll have to do some more research on the subject and revisit this decision. I'll also look into implementing more choices in algorithm.