RFC: Add julia ldlt factorization and solver for SymTridiagonal and julia lu factorization for Tridiagonal

[andreasnoack]

@alanedelman suggested Julia implementation for factorizations of SymTridiagonals. I have followed LAPACK and used ldlt for SymTridiagonal because it doesn't require allocation. I also wrote a solver for the factorization and removed the calls to LAPACK.

For Tridiagonal, I translated the lu from LAPACK to Julia since I didn't dare to open a pull request with a version without pivoting. To do so, I have changed the Tridiagonal type to only have one extra vector allocation. This is required for the pivoting. This also caused some cleanup of the tridiagonal code. I have changed all solve methods to A_ldiv_B to be consistent with the rest of Base. If anyone uses the Tridiagonal code please have a look and test.

A style innovation that I would like some comments on is that I have parametrized lu and ldlt on matrix type. In consequence there is only one lu type definition because it works for ordinary matrices as well as tridiagonals. This change also makes it possible to use array views in factorizations. I would like to extend this innovation to the other special matrices such that e.g. Triangular can be any AbstractMatrix.

Finally, I Implement convert(AbstractArray,x) and convert(Factorization,x) methods as proposed by @JeffBezanson in #6078. I get the same type of warnings could not show value of type Tuple. @JeffBezanson can you figure out what is wrong?

cc: @jiahao

[jiahao]

I like the idea of parameterizing the factorization by matrix type. It would certainly clean up the documentation quite a bit too; the updated explanation of lufact was getting slightly out of hand.

I'm not sure it's necessary to deprecate solve everywhere as part of this PR. I'd prefer the scope of this PR to just deprecate solve for Tridiagonals and take the rest from there.

[timholy]

For Tridiagonal, I translated the lu from LAPACK to Julia since I didn't dare to open a pull request with a version without pivoting.

😄

[andreasnoack]

I'm not sure it's necessary to deprecate solve everywhere as part of this PR. I'd prefer the scope of this PR to just deprecate solve for Tridiagonals and take the rest from there.

I agree, but I think it makes sense to change the Woodbury solve also because it only worked with the solve(Tridiagonal). The solves in the sparse code can stay for now.

[andreasnoack]

@JeffBezanson There is an issue with the ambiguity warnings beginning at line 840 here.

[alanedelman]

Can we have pivoting=false as an option? Many use cases are symmetric positive definite matrices
where pivoting is not required for stability

[andreasnoack]

Yes. I can add that option. It would also be useful to have that option for dense matrices as discussed [here](that option). Just curious, why do you prefer the lu over the ldlt?

[andreasnoack]

@alanedelman I have now merged the LDLt and LU for tridiagonal matrices and made pivoting optional, so you can write lufact(A,pivot=false).

@jiahao Unfortunately, I don't think it is possible to simplify the documentation for LU. I like the structure for the factorizations, by the way.