PyPlot interferes with LinearAlgebra in a weird way #477
Comments
|
I can't reproduce — it seems to work fine for me on MacOS with Julia 1.4. So it may be a Windows thing, e.g. some strange interference between numpy's LAPACK library and Julia's? |
|
I cannot reproduce this on a windows server 2016 virtual machine. What can I do to figure out what's wrong with my machine? |
|
I also have met similar issues on a single PC with Windows 10 64-bit, with Julia v1.4.2, PyPlot v2.9.0, Polynomials v1.1.3, and CSV v0.7.4. In a script that I first run through some PyPlot plotting commands, Polynomials.fit() may do one of the following:
And sometimes CSV.read() will fail with error message like: ArgumentError: "data.csv" is not a valid file. However, this file has been loaded perfectly fine elsewhere. If I comment out the plotting, all these issues go away. (I can still "using PyPlot" - that doesn't seem to hurt.) See this issue I submitted to Polynomials.jl for details. Tried pointing ENV["PYTHON"] to the MiniConda Python coming with Julia, to an ActivePython 3.7 installation, or to a standard Python 3.8 installation (which has no MKL), and rebuilding PyCall. Doesn't seem to matter for these issues. I have another machine with the same CPU (Intel i9-9900K) and same Julia 1.4.2 and PyPlot v2.9.0, but there I installed MKL.jl so BLAS.vendor() is MKL instead of openblas. (On this machine I have to use the standard Python with no MKL to avoid conflict between Python's numpy MKL and Julia's MKL, which is a known problem.) These issues do not happen on this machine. On older Julia (v1.4.1 and before) I have never seen this. But of course other packages may also be older at that time. |
|
Following up on my comment above: I found the workaround for this particular PC: install MKL.jl. By doing so I limit myself to a standard Python installation without MKL for matplotlib. However, this works around all the weird issues I reported above. |
|
If you are using MKL, there is the danger of a conflict between its BLAS library and the MKL library that is installed by many Python distributions (e.g. Anaconda) for Numpy — see JuliaPy/PyCall.jl#443. With the default Julia OpenBLAS library, however, I didn't think there would be a symbol conflict, at least on 64-bit machines? |
|
@zhopan77 Can you reproduce my original example? BTW: I can reproduce my original issue in julia-1.5.0-rc1. |
|
@stevengj Yes I'm aware of the MKL conflict between Julia and Python. For that purpose I installed the standard Python from https://www.python.org/ and installed only matplotlib on it, and I pointed Julia to this special installation rather than the typical Python distribution I use by default. It's wasteful but works. : -) |
|
@atbug I already switched over to MKL so the issues won't happen anymore... Maybe I will reinstall the openblas Julia and try it some time later. |
|
A list of processors that might reproduce this issue: i5-9265U; i9-9900 (from https://discourse.julialang.org/t/bug-in-linearalgebra-plots-pyplot-combo-on-intel-i9-9900/44097). @zhopan77 what's your processor? |
|
I don't reproduce this with the 1.5.0 release on 9900k OpenBLAS on Windows: _
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.5.0 (2020-08-01)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using LinearAlgebra, PyPlot
julia> xlim([-1, 1]) # a window will pop up, leave the window open
(-1, 1)
julia> H = rand(46, 46)+im*rand(46, 46);
julia> values, vectors = eigen(Hermitian(H))
Eigen{Complex{Float64},Float64,Array{Complex{Float64},2},Array{Float64,1}}
values:
46-element Array{Float64,1}:
-9.648144895450402
-6.403018195231846
-5.567006204994933
-4.651101399349197
-4.360775788309995
-3.924667423840675
-3.629646918347601
-3.4363134351487616
-3.00654617573785
-2.783730791156831
-2.578633073843469
-2.242173850537144
-2.1137776117369507
⋮
2.5661762242605484
2.740609665523724
2.92585837174361
3.2168926169092042
3.424265999066767
3.825152916221416
4.129331637473747
4.434268297449822
4.733944831449584
5.461958227240098
7.553837821773461
29.425237094361222
vectors:
46×46 Array{Complex{Float64},2}:
-0.0380281-0.166063im -0.0560625-0.0569382im … 0.078757+0.165303im 0.00268687-0.134841im
-0.0381843-0.163177im -0.0656187-0.135121im 0.147389+0.193109im 0.00127123-0.154623im
-0.0252568-0.128033im 0.0744523-0.171277im 0.0972979+0.122269im -0.0170815-0.124605im
0.0182624-0.177805im 0.163453-0.153465im 0.153768+0.0704818im -0.00642343-0.130077im
0.0799447-0.109911im 0.170979+0.00691165im 0.226077+0.0746839im 0.0186681-0.114621im
0.0541363-0.100239im 0.164807-0.127734im … 0.149009+0.00977647im -0.0232897-0.147091im
0.0702992-0.129334im 0.0948445+0.109305im 0.0917108-0.0836994im -0.0235026-0.158625im
0.071336-0.133863im 0.148317+0.0471107im 0.204562-0.101519im -0.0233808-0.1493im
0.147327-0.0908953im 0.167219+0.0527419im 0.14576-0.0855678im -0.016402-0.152231im
0.121977-0.125995im 0.00612952+0.202024im 0.0749364-0.010839im -0.0504555-0.156991im
0.0262741-0.0453338im 0.0715243+0.155262im … 0.0703615-0.0880875im -0.0242665-0.14327im
0.11135-0.0668462im 0.0694948+0.0819725im 0.0922752-0.0579878im -0.0450097-0.135566im
0.14415-0.0716435im 0.0426397+0.219534im 0.0467667-0.141799im -0.0338453-0.139842im
⋮ ⋱ ⋮
-0.0331857+0.156023im 0.0373864+0.0535262im -0.0390124+0.132547im -0.128443-0.0774424im
-0.0405406+0.118388im 0.133691+0.00958768im … 0.00290895+0.157096im -0.135272-0.0750328im
-0.0380726+0.15514im 0.161875-0.0538643im 0.00114205+0.0416856im -0.143744-0.0614387im
-0.0630953+0.125791im 0.0520178+0.161339im 0.0394217+0.173115im -0.142329-0.0620126im
-0.0890218+0.142978im 0.0878746+0.0871438im 0.141904+0.0275529im -0.151823-0.0438958im
-0.0815737+0.127161im -0.0594128+0.148887im 0.155414+0.0971197im -0.144271-0.0430016im
-0.114734+0.15381im 0.0261279+0.167901im … 0.0162599-0.0125471im -0.148537-0.0403495im
-0.126208+0.0954169im -0.00961876+0.119196im 0.0637185+0.0440769im -0.157727-0.0288604im
-0.0963627+0.134969im -0.0947695+0.124531im 0.0713737+0.0647373im -0.144469-0.0310239im
-0.117345+0.0680233im -0.0851291+0.15621im 0.176956+0.0579259im -0.156455-0.007943im
-0.124463+0.0792475im -0.0777825-0.0507211im 0.160452-0.0271196im -0.130636-0.000188684im
-0.150394-0.0im -0.152386-0.0im … 0.170812+0.0im -0.138933-0.0im
julia> versioninfo()
Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 16
(@v1.5) pkg> st
Status `C:\Users\alecl\.julia\environments\v1.5\Project.toml`
[bdd23359] ActuaryUtilities v0.3.2
[6e4b80f9] BenchmarkTools v0.5.0
[a93c6f00] DataFrames v0.21.5
[31c24e10] Distributions v0.23.4
[e30172f5] Documenter v0.24.11
[35a29f4d] DocumenterTools v0.1.6
[713c75ef] Franklin v0.9.1 `https://github.com/tlienart/Franklin.jl.git#master`
[7073ff75] IJulia v1.21.2
[c8f0d631] LifeContingencies v0.6.1 `C:\Users\alecl\.julia\dev\LifeContingencies`
[4780e19d] MortalityTables v0.10.2 `C:\Users\alecl\.julia\dev\MortalityTables`
[2bd173c7] NodeJS v1.1.1
[6fe1bfb0] OffsetArrays v1.0.4
[5fb14364] OhMyREPL v0.5.5
[429524aa] Optim v0.21.0
[b98c9c47] Pipe v1.3.0
[91a5bcdd] Plots v1.4.1
[c3e4b0f8] Pluto v0.10.10 `C:\Users\alecl\.julia\dev\Pluto`
[d330b81b] PyPlot v2.9.0
[295af30f] Revise v2.7.2
[2913bbd2] StatsBase v0.32.2
[b8865327] UnicodePlots v1.2.0
[44d3d7a6] Weave v0.10.2
[ade2ca70] Dates |
|
I recently bought a new laptop. This issue is still reproducible on Intel Core i7-1185G7. The OS is Windows 10 20H2. |
|
I can easily reproduce this with the following code, which returns all using PyPlot, LinearAlgebra
fig = figure(1);
x = rand(10,10);
plot(x); xlim(-12,0);
first(qr(x))[1]julia setup: julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-8086K CPU @ 4.00GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
JULIA_CC = C:\mingw-w64\bin\gcc.exe
JULIA_EDITOR = atom.cmd -a
JULIA_NUM_PRECOMPILE_TASKS = 4
JULIA_NUM_THREADS = 6
JULIA_PKG_SERVER = pkg.julialang.org
julia> LinearAlgebra.versioninfo()
BLAS: libopenblas (OpenBLAS 0.3.9 USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=32)
LAPACK: libopenblas64_python setup: |
|
@stevengj Sorry for pinging: I have seen in a few comments that you say this is a julia issue. Is there a way to reproduce this without PyPlot? I tried calling a few numpy linalg functions to see if it would break julia but so far the bug hasn't manifested. If we are able to find a simple repro I can open an issue on |
|
I don't have a Windows machine where I can reproduce this, but I don't see how this could be pyplot-specific — as far as I know, Matplotlib does not use LAPACK or BLAS directly. There must be some numpy call in matplotlib that is triggering the problem. You could try loading matplotlib directly without using PyPlot.jl using PyCall
plt = pyimport("matplotlib.pyplot")
plt.plot(rand(10,10))
plt.xlim(-12,0)
plt.show()to see if you can narrow it down. |
|
Interestingly trying to call directly matplotlib like this fails at the EDIT: and I can't figure out why. Searching suggests some PATH issues but I can't find a culprit. |
|
Try using a different backend. Set the MPLBACKEND environment variable to “tkagg”, for example. |
|
Okay, I can reproduce this with the tkagg backend. At this point you think I should open a julia issue? I not sure how to debug this further. I listed the DLLs used by the process, and as expected there is both openblas and mkl: |
Yes. |
|
I cannot reproduce now with latest Julia and PyPlot. |
Let me start with how to reproduce:
Several observations:
values, vectors = eigen(Hermitian(H))will give the wrong result. If the statement is run a second time, the eigenvalues and eigenvectors are correct.PyPlotis started withAggbackend, the eigenvalues are correct.46is not very special and24,12would also reproduce this. However, if the number is4. I cannot reproduce this on my machine.run.jland doinclude(run.jl)in the REPL, I cannot reproduce this.Python package list:
The text was updated successfully, but these errors were encountered: