Reduce binary size by reducing templates

[dimztimz]

See the git branch less-templates for initial work.

Some more work can be done to avoid branches (if in C++). The compile-time branches became run-time branches in that code and that lowers the performance for a few percent. Some of those ifs are invariant in those hot loops and can be moved to higher levels.