Finalize extra packing for macOS bundle #1918

PaulWessel · 2019-10-26T20:50:18Z

This PR finalizes how cpack works on macOS under MacPorts; this time handling the gs link. If there is no gs -> gsc link on brew then we need to work on the script a bit more. It would be good if @seisman on brew could

Temporarily set GMT_PUBLIC_RELEASE true
In top gmt-dev directory, run admin/build-release.sh
Make sure the bundle still works and contains all the goodies.
Examine build/add_macOS_cpack.txt to make sure all is well.

Maybe we should exchange bundles to make sure they run on systems with a different distro.

@seisman

This PR finalizes how cpack works on macOS under MacPorts. IT would be good if @seisman on brew could Temporarily set GMT_PUBLIC_RELEASE true In top gmt-dev directory, run admin/build-rehease.sh Make sure the bundle still works. Maybe he and I should exchange bundles to make sure they run on systems with a different distro.

seisman · 2019-10-26T21:04:37Z

For homebrew, all the commands are located in /usr/local/bin directory, but they are symlinks.

$ which gs gsc
/usr/local/bin/gs
/usr/local/bin/gsc

$ ll /usr/local/bin/gs /usr/local/bin/gsc
lrwxr-xr-x  1 seisman  admin    33B Oct 22 21:36 /usr/local/bin/gs -> ../Cellar/ghostscript/9.50/bin/gs
lrwxr-xr-x  1 seisman  admin    34B Oct 22 21:36 /usr/local/bin/gsc -> ../Cellar/ghostscript/9.50/bin/gsc

PaulWessel · 2019-10-26T21:10:33Z

So given my gs was copied as an existing symlink, we will need the build-macos*.sh script to replace links with the real thing. I see in bash, -L checks if a link, so then we need to do something like

for file in all files; do
  if [ -L $file ]; then # symblink
     echo the name the link points to
  else
    echo $file
done

Better suggestions?

PaulWessel · 2019-10-26T21:12:17Z

Google says use readlink. So we use readlink in that loop.
Please type readlink /usr/local/bin/gs and see if that works.

seisman · 2019-10-26T21:18:02Z

$ readlink /usr/local/bin/gs                                                                                                                                                                       
../Cellar/ghostscript/9.50/bin/gs

PaulWessel · 2019-10-26T21:18:55Z

Do you always get those ..? What if you run the readlink from another directory?

seisman · 2019-10-26T21:20:29Z

Nothing changes.

PaulWessel · 2019-10-26T21:22:20Z

So if we do this in the loop and /usr/local/bin/gs becomes ../Cellar/..., will cpack find things? Or do we need to replace the .. with the actual directory?

seisman · 2019-10-26T21:24:13Z

$ realpath `which gs`
/usr/local/Cellar/ghostscript/9.50/bin/gs

I find realpath very useful, but it's a Linux command, and I installed it via brew install coreutils.

PaulWessel · 2019-10-26T21:25:10Z

OK, I can do that too. Perhaps that is the safest? And we document somewhere in admin that macOS maintainers will need that package?

PaulWessel · 2019-10-26T21:26:20Z

I got this mesage:
coreutils has the following notes:
The tools provided by GNU coreutils are prefixed with the character 'g' by default to distinguish them from the BSD commands.
For example, cp becomes gcp and ls becomes gls.

If you want to use the GNU tools by default, add this directory to the front of your PATH environment variable:
    /opt/local/libexec/gnubin/

can you use grealpath?

PaulWessel · 2019-10-26T21:29:53Z

One more complications. If I blindly do this on all links, I get

/opt/local/bin/gsc
/opt/local/bin/gsc

and no gs. So for macports, need to preserve that gs link. So your gs is the executable?

PaulWessel · 2019-10-26T21:39:12Z

I am modifying the script a bit and making three sets:

# 1a. List of executables whose shared libraries also are needed.
EXEPLUSLIBS="gsc gm ffmpeg"
# 1b. List of symbolic links.
EXELINKS="gs"
# 1c. List of executables whose shared libraries have been included via GDAL
EXEONLY="ogr2ogr gdal_translate"

Do I need a check on Macports vs HomeBrew to make changes here? The EXELINKS will just let which tell us the path, but the other two will go through a greadpath step.

PaulWessel · 2019-10-26T21:45:44Z

Please see if just running this produces acceptable output, i.e.

bash build-macos-external-list.sh

seisman · 2019-10-26T22:42:56Z

This is what I get:

install (PROGRAMS
	/usr/local/Cellar/ffmpeg/4.2.1_1/bin/ffmpeg
	/usr/local/Cellar/ghostscript/9.50/bin/gsc
	/usr/local/Cellar/graphicsmagick/1.3.33/bin/gm
	/usr/local/bin/gs
	DESTINATION ${GMT_BINDIR}
	COMPONENT Runtime)

For me, both gs and gsc are executables, but gs is 13 M and gsc is only 9 Kb.

PaulWessel · 2019-10-26T22:51:51Z

OK, looks reasonable to me, I guess you can try to run the whole thing: Set the GMT_PUBLIC_RELEASE to true and try

bash admin/build-release.sh

and see if it works or not.

seisman · 2019-10-26T22:53:16Z

Nope, /usr/local/bin/gs is wrong. It's a symlink.

PaulWessel · 2019-10-26T22:55:01Z

But /opt/local/bin/gs is a symlink for me and it copes that in as a link.

seisman · 2019-10-26T23:01:43Z

-rwxr-xr-x  1 seisman  staff   266K Oct 26 19:00 ffmpeg
-rwxr-xr-x  1 seisman  staff    16K Oct 26 19:00 gm
-rwxr-xr-x  1 seisman  staff    30K Oct 26 19:00 gmt
-rwxr-xr-x  1 seisman  staff   6.0K Oct 26 18:56 gmt-config
-rwxr-xr-x  1 seisman  staff    18K Oct 21 17:04 gmt_shell_functions.sh
-rwxr-xr-x  1 seisman  staff   5.6K Oct 21 17:04 gmtswitch
lrwxr-xr-x  1 seisman  staff    33B Oct 26 19:00 gs -> ../Cellar/ghostscript/9.50/bin/gs
-rwxr-xr-x  1 seisman  staff   9.0K Oct 26 19:00 gsc
-rwxr-xr-x  1 seisman  staff   1.2K Oct 26 18:56 isogmt

PaulWessel · 2019-10-26T23:03:21Z

OK, not so good. I added $distro as a variable. Perhaps you can play with this by maybe removing the gs and adding it specially as a link to gsc in that same directory?

PaulWessel · 2019-10-26T23:25:00Z

If your actual gs is 13 Mb (in Cellar) while gsc is only 9kB then that seems opposite to what I have (gsc is the program, gs is a link). Very odd.

seisman · 2019-10-27T01:07:02Z

Tried your DMG file. gs doesn't work:

$ gs
GPL Ghostscript 9.27: Can't find initialization file gs_init.ps.

PaulWessel · 2019-10-27T02:13:09Z

Right. So party can be solved by setting GS_LIB to point to it, but of course there is the entirety of /opt/local/share/ghostscript. So would have to add that entire directory into our bundles share directory and have the init set GS_LIB. I also have /opt/local/share/GraphicsMagick-1.3.33. GDAL I think we already have, and there is nothing about ffmpeg. I will try this.

PaulWessel · 2019-10-27T23:53:57Z

Probably, but @seisman says he has built the bundle under brew and got stuff like netcdf included. I am building gs 9.50 from source now, under macports, and will try to build a bundle with everything.

PaulWessel · 2019-10-28T00:31:28Z

So building with port now does not include netcdf either. Apart from the 58 shared libs that build-macos-external-list.sh found, cpack only added the gmt and postscript libs, the gmt plugin dir. No netcdf, pcre, etc. Maybe an error happened. Also, would like to build gs without X11 but it adds Z11 libs now.

PaulWessel · 2019-10-28T00:45:40Z

Well, excluding the macos list and building the bundle as I ahve done for years now yields no siupporting shared libraries. I wonder if it gets confused with both brew and macport files and does nothing.

PaulWessel · 2019-10-28T01:17:32Z

So after removing brew completely, rebooting, and redoing the build-release.sh step, the bundle still has no shared libraries beyond gmt and postscriptlight. How is this possible?

PaulWessel · 2019-10-28T03:43:22Z

Latest version seems to find all dylibs recursively and adds teh ghostscript and gdal share dirs. This has gs 9.50 which may only take GS_LIB. Note: Not set in the shell startup yet so if gs fails please set GS_LIB to the /Application/gmt-6.0.0.app/......./share/ghostscript/9.50/lib or whatever.

PaulWessel · 2019-10-28T05:23:10Z

Am I the only one who thinks

/opt/local/lib/proj6/share

is one stupid place to place share information when there is opt/local/share for that?

SO where to we place this in the bundle? We would have to use GDAL_LIB if that even works for this non-GDAL, probably need to set PROJ_LIB then. So maybe place this ih share/proj6 on the installer.

Also add reference to licenses dir.

PaulWessel · 2019-10-28T17:36:38Z

Last bit that remains is to place all the licenses for gs, gm, ffmpeg, gdal in a licenses folder in the bundle.

PaulWessel · 2019-10-28T17:38:53Z

Should we create a licsenses directory under admin and then just place that directory in the bundle? That may be the simplest. I added a one-liner in the startup script that directs users to that directory for license information.

Will hold all licesnes for executables in the bundle

PaulWessel · 2019-10-28T18:15:56Z

Think this PR is good to be approved now, @seisman ?

PaulWessel · 2019-10-28T19:19:45Z

share/Licenses on the bundle now lists

FFmpeg-LGPL-License.txt GMT-License.txt GNU-LGPL-3.txt GraphicsMagick-License.txt
GDAL-License.txt GNU-GPL-3.txt Ghostscript-AGPL-License.txt

and when you double-click on the GMT logo it prints the gmt splash screen and adds

Note 2: GMT may use Ghostscript, GraphicsMagick, FFmpeg, and GDAL executables; see /Applications/GMT-6.0.0.app/Contents/Resources/share/Licenses for details.

PaulWessel · 2019-10-29T03:28:52Z

Hi @seisman, any reason this cannot be approved?

seisman

Just a few typos, then it's good to merge.

admin/build-macos-external-list.sh

Co-Authored-By: Dongdong Tian <[email protected]>

seisman · 2019-10-29T03:38:31Z

Could you build a bundle after merging this PR and upload it to your FTP? Then I can have more testing on the bundle.

PaulWessel · 2019-10-29T03:38:50Z

Will do now.

PaulWessel · 2019-10-29T03:48:21Z

OK, same ftp link as before. Heading home - let me know if there are any issues.

PaulWessel requested review from seisman and joa-quim October 26, 2019 20:50

Place the windows cpack files in admin

8da8291

PaulWessel mentioned this pull request Oct 26, 2019

Create build-macos-external-list.sh #1917

Merged

Broaden to also handle homebrew distro

9ab2cae

Tell macports, homebrew, fink apart

a9e9d90

Update RELEASE.md

d42f713

Allow hard paths

b4831fe

Add recursive otool

23c31e2

PaulWessel added 2 commits October 27, 2019 17:53

Build otoolr in the build dir

593ffc1

Add fftw single and proj6 share dir

5861522

PaulWessel added 2 commits October 27, 2019 20:19

add proj6 data

dc6f3e3

Set PROJ_LIB in startup script

2dc451b

Also add reference to licenses dir.

Add License dir

2ce0253

Will hold all licesnes for executables in the bundle

PaulWessel added 3 commits October 28, 2019 08:26

Add all licenses to admin/Licenses

871b3c5

Install the licenses

94a11da

Final reseting place

9e181a8

seisman approved these changes Oct 29, 2019

View reviewed changes

admin/build-macos-external-list.sh Outdated Show resolved Hide resolved

admin/build-macos-external-list.sh Outdated Show resolved Hide resolved

admin/build-macos-external-list.sh Outdated Show resolved Hide resolved

PaulWessel and others added 3 commits October 28, 2019 17:37

Update admin/build-macos-external-list.sh

38a681a

Co-Authored-By: Dongdong Tian <[email protected]>

Update admin/build-macos-external-list.sh

bbc7947

Co-Authored-By: Dongdong Tian <[email protected]>

Update admin/build-macos-external-list.sh

90b4b8f

Co-Authored-By: Dongdong Tian <[email protected]>

PaulWessel merged commit e1796e6 into 6.0 Oct 29, 2019

PaulWessel deleted the macospacking branch October 29, 2019 03:38

Finalize extra packing for macOS bundle #1918

Finalize extra packing for macOS bundle #1918

Conversation

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

PaulWessel commented Oct 26, 2019

seisman commented Oct 27, 2019

PaulWessel commented Oct 27, 2019

PaulWessel commented Oct 27, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 28, 2019

PaulWessel commented Oct 29, 2019

seisman left a comment

Choose a reason for hiding this comment

seisman commented Oct 29, 2019

PaulWessel commented Oct 29, 2019

PaulWessel commented Oct 29, 2019