Check CHOLMOD version at runtime

[andreasnoack]

This one should help to fix problem like #10342 where the version of CHOLMOD linked at runtime differs from the version used to create the shim suitesparse_wrapper. If the two versions don't match, an error is thrown.

[tkelman]

differs from the version used when Julia was built

[andreasnoack]

Thanks.

[tkelman]

Does throwing exceptions in __init__ end up as a warning, or would that be an error immediately on startup making Julia unusable even for people who don't need cholmod?

[andreasnoack]

Just checked and the result right now is

Hej
Warning: error initializing module CHOLMOD:
Base.SparseMatrix.CHOLMOD.CHOLMODException(msg="your version of CHOLMOD differs from the version used when Julia was built. These versions must match.")
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+3625 (2015-02-28 19:55 UTC)
 _/ |\__'_|_|_|\__'_|  |  anj/cholmod/297b698* (fork: 1 commits, 0 days)
|__/                   |  x86_64-linux-gnu

though I plan to remove the "Hej".

So you can still use Julia even if you have an old or wrong version of CHOLMOD.

[ivarne]

This error will likely be encountered by people using various distributions, so there is a high probability they will not know what CHOLMOD is for, and they will have a hard time figuring out if the message is relevant to them. It is not even mentioned in the README.md, but the Lincence.md seems to indicate that it is related to some operations on sparce matrices.

I will also award a few bonus points for including the compiled version, and the current version. It's usually a struggle to make users find such information.

eg:

Julia was compiled with CHOLMOD version x.y.z, but is currently
linked to version x.y.Z, this might cause errors when working with
[whatever operations CHOLMOD provides] on sparse matrices.

[andreasnoack]

It is a good idea to mention the version numbers in the error message. They are calculated anyway. However, real users shouldn't see this error. It is mainly meant as a help for packagers to make the right dependency requirements. Julia is broken and can segfault when the versions of CHOLMOD differ so we should really try to avoid mismatch.

[ivarne]

Real users will see this error message when the packagers mess up (which they do).

[andreasnoack]

I'm fine with adding to the error message that the sparse solvers are broken when the versions don't match, but I feel sad that the packaging model of Linux distributions makes it so likely that Julia will end up broken. I'd really prefer that we shipped our own copies of the dependencies. I have space enough left on my harddisk to over 5000 copies of SuiteSparse, so I'd really prefer a couple of extra copies to a potentially broken application.

[tkelman]

Encourage people to use the generic linux binaries instead of from their package manager then. You lose system-managed in-place updates that way, but you gain self-contained installation without needing root.

For distribution packaging we need to either declare more explicit version requirements on dependencies (#6742 is part of that), or register sysimg rebuild hooks. The static compilation work might make this a little less painful if the Julia sysimg could be broken up into more independent pieces, so you wouldn't have to rebuild the whole thing.

[nalimilan]

This shouldn't happen in standard distro packages. Normally, packagers are expected to warn maintainers of all packages that depend on their library when they update to a backward-incompatible version so that they can rebuild their package.

So I'd say the problem comes from the fact that Julia moves very fast, and it's not easy for us to handle stable and development packages without sometimes breaking things. Maybe @staticfloat would better create a separate repo for the dependencies of stable releases, were things are more calm and users can expect more working packages.

[staticfloat]

I agree with @ivarne that we should make the error as user-facing as possible. With all the moving parts we now have with the Julia builds, it's difficult to ensure everything works when problems are silent until the moment of critical failure. The first step toward ensuring that problems like this don't arise is preemptively checking for them, so this is a nice improvement.

[nalimilan]

BTW, @staticfloat, it looks like the policy for Debian packages is to include the SOVERSION in their name, so normally you cannot install Julia with an unsupported dependency version. Sounds like a sane approach to me, that Fedora unfortunately does not follow AFAICT.

[tkelman]

This message should spell out what the consequences of this problem mean. Otherwise it'll confuse people.

Is the exact version number what we actually want to be checking here, or should we specifically be checking as many features as we can? SuiteSparse can be configured in different ways by different packagers...

[andreasnoack]

@tkelman @ivarne I have extended the warnings in this pull request. Could you please take a look and see if the information is useful and if the language is correct.

[ivarne]

The warnings is tremendously more useful now! Thanks!

A few esthetic remarks. My terminal doesn't do a great job with breaking such long lines at word boundries, so I'd appreciate a few newlines at appropriate places. The """ quoted string also have a nice deindentation functionality that allows you to have multiline string literals without breaking the indentation pattern. (see how the leading spaces is removed by the parser.)

begin
    begin
        begin
            warn("""
                     CHOLMOD version incompatibility

                     Julia was compiled with a version of CHOLMOD that supported $(32length(IndexTypes))
                     bit integers, but is currently linked with version that supports $(8intsize) integers. This
                     might cause Julia to terminate when working with sparse matrices for operations involving
                     factorization of a matrix, e.g. solving systems of equations with \\.

                     This problem can be fixed by downloading the OS X or generic Linux binary from
                     www.julialang.org, which are shipped with the correct versions of all dependencies.
            """)
        end
    end
end

[nalimilan]

@andreasnoack Somehow I missed that this commit added a warning even when only SuiteSparse's revision version (z in x.y.z) is changed. Is that absolutely necessary? Couldn't we check only the two first version numbers? It doesn't sound like a good development practice to me to break backward compatibility in an apparently bugfix-only version of a library; ideally, it would not even happen in minor releases, only in major ones. We could discuss this with upstream if needed.

This was raised in #12841, where a bugfix update of SuiteSparse in Fedora triggered the warning in Julia. Printing a scary warning all the time in a distro package isn't reasonable, and neither is updating Julia with every SuiteSparse update. I've never seen such a situation in established programs.

[andreasnoack]

@nalimilan Good point. The test could probably be loosened to only check for match on x and y.

[nalimilan]

Ah, great to hear. That makes it much easier to follow SuiteSparse updates in distros.

[nalimilan]

@andreasnoack See #13000.