Attempting to codesign a MacOS app corrupts the app

[ndw]

Description

I'm trying to publish a command line MacOS application built with .NET. I've read many of the issues related to this, but none seems to precisely cover my case (at least not in a way that I understand). I did gather that it doesn't work in .NET 5 so I've updated to .NET 6. In order to get it past gatekeeper, I need to sign, notarize, staple, bend, fold, spindle, etc.

I can't seem to get past the first step. If I use codesign to sign the app, the app is then corrupted and won't run.

$ codesign --timestamp -o runtime --force --verify --verbose --sign MYIDHERE SaxonCS
SaxonCS: replacing existing signature
SaxonCS: signed Mach-O thin (x86_64) [SaxonCS]

$ ./SaxonCS
Failed to load /private/tmp/app/libcoreclr.dylib, error: dlopen(/private/tmp/app/libcoreclr.dylib, 0x0001): tried: '/private/tmp/app/libcoreclr.dylib' (code signature in <3ACEB6DA-5249-3A77-A23B-BB471A3797C8> '/private/tmp/app/libcoreclr.dylib' not valid for use in process: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.), '/usr/lib/libcoreclr.dylib' (no such file)
[1]    56791 segmentation fault  ./SaxonCS

I'm slightly confused by the "replacing existing signature" message. I haven't configured .NET to do signing automatically, at least not on purpose, and if it is signing automatically, I don't see how it could be using the right key ID.

On a possibly related note, I'm surprised by the build artifacts that dotnet publish produces. I'm running

dotnet publish SaxonCS.sln --configuration Release -r osx-x64 \
        --self-contained true -p:PublishSingleFile=true \
        -p:PublishReadyToRun=true -p:UseAppHost=true \
        -p:Version=11.2.0 -p:PackageVersion=11.2.0

But I'm not getting a "single file":

$ ls -lA build/cs/bin/Release/net5.0/osx-x64/publish/
.rwxr--r-- 1 2.6M ndw 15 Feb 16:55 libclrjit.dylib
.rwxr--r-- 1 6.9M ndw 15 Feb 17:00 libcoreclr.dylib
.rwxr--r-- 1 962k ndw 15 Feb 16:48 libSystem.IO.Compression.Native.dylib
.rwxr--r-- 1  87k ndw 15 Feb 16:48 libSystem.Native.dylib
.rwxr--r-- 1  36k ndw 15 Feb 16:48 libSystem.Net.Security.Native.dylib
.rwxr--r-- 1  68k ndw 15 Feb 16:48 libSystem.Security.Cryptography.Native.Apple.dylib
.rwxr--r-- 1 172k ndw 15 Feb 16:48 libSystem.Security.Cryptography.Native.OpenSsl.dylib
.rwxr-xr-x 1 105M ndw 11 Mar 14:40 SaxonCS
.rw-r--r-- 1 1.9M ndw 11 Mar 14:40 SaxonCS.pdb
.rw-r--r-- 1 736k ndw 11 Mar 14:40 SaxonCS.xml

(If I don't use the single file option, I get dozens and dozens of files, so it's certainly closer to single file!)

On other occasions, with slightly different publish commands, I get more or less further along. Sometimes I can sign the SaxonCS file and it runs, but complains the other dylib files aren't signed. If I sign them, things crash differently.

I fully expect this is user error, but I cannot find any explanation of either what I should be doing or what I might be doing wrong.

Configuration

$ /usr/local/share/dotnet/dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.201
 Commit:    ef40e6aa06

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  12.2
 OS Platform: Darwin
 RID:         osx.12-x64
 Base Path:   /usr/local/share/dotnet/sdk/6.0.201/

Host (useful for support):
  Version: 6.0.3
  Commit:  c24d9a9c91

.NET SDKs installed:
  6.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

Other information

[carlossanlop]

I think the SDK team might be able to help with this publish related question. Transferring to that repo.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ndw]

Is there anything I can do to help triage this? I've recreated the problem with the five minute sample application from the .NET documentation.

[ndw]

Simple example of reproducing the bug:

$ cd /tmp

/tmp
$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.201
 Commit:    ef40e6aa06

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  12.3
 OS Platform: Darwin
 RID:         osx.12-x64
 Base Path:   /usr/local/share/dotnet/sdk/6.0.201/

Host (useful for support):
  Version: 6.0.3
  Commit:  c24d9a9c91

.NET SDKs installed:
  6.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

/tmp
$ dotnet new console -o MyApp -f net6.0
The template "Console App" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on /private/tmp/MyApp/MyApp.csproj...
  Determining projects to restore...
  Restored /private/tmp/MyApp/MyApp.csproj (in 134 ms).
Restore succeeded.

/tmp took 2s
$ cd MyApp/

/tmp/MyApp via .NET 6.0.201
$ dotnet run
Hello, World!

/tmp/MyApp via .NET 6.0.201 took 3s
$ dotnet publish --configuration Release -r osx-x64 --self-contained true -p:PublishSingleFile=true -p:Version=1.0.0 -p:packageVersion=1.0.0
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /private/tmp/MyApp/MyApp.csproj (in 242 ms).
  MyApp -> /private/tmp/MyApp/bin/Release/net6.0/osx-x64/MyApp.dll
  MyApp -> /private/tmp/MyApp/bin/Release/net6.0/osx-x64/publish/

/tmp/MyApp via .NET 6.0.201 took 7s
$ bin/Release/net6.0/osx-x64/publish/MyApp
Hello, World!

/tmp/MyApp via .NET 6.0.201 took 3s
$ codesign --timestamp -o runtime --force --verify --verbose --sign MYKEYIDHERE bin/Release/net6.0/osx-x64/publish/MyApp
bin/Release/net6.0/osx-x64/publish/MyApp: replacing existing signature
bin/Release/net6.0/osx-x64/publish/MyApp: signed Mach-O thin (x86_64) [MyApp]

/tmp/MyApp via .NET 6.0.201
$ bin/Release/net6.0/osx-x64/publish/MyApp
Failed to create CoreCLR, HRESULT: 0x80004005

I suspect that "replacing existing signature" is relevant, but I can't for the life of me find any documentation about how to prevent dotnet from signing automatically or how to tell it to sign with the correct certificate.

[ndw]

@baronfel is there anything else I can do to help move this along? It's blocking a production release for me.

[baronfel]

I haven't dug in all the way, but based on #24181 it looks like you could have the dylibs included in the single-file version of the app by setting IncludeNativeLibrariesForSelfExtract to true in your publish command. Probably also IncludeSymbolsInSingleFile to true as well to make sure the pdb makes it in as well. That's all coming from a quick peek at the GenerateSingleFileBundle task.

[ndw]

That doesn't seem to work:

$ dotnet publish --configuration Release -r osx-x64 --self-contained true -p:PublishSingleFile=true -p:IncludeNativeLibrariesForSelfExtract=true -p:IncludeSymbolsInSingleFile=true -p:Version=1.0.0 -p:packageVersion=1.0.0
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  MyApp -> /private/tmp/MyApp/bin/Release/net6.0/osx-x64/MyApp.dll
/usr/local/share/dotnet/sdk/6.0.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Publish.targets(98,5): error NETSDK1142: Including symbols in a single file bundle is not supported when publishing for .NET5 or higher. [/private/tmp/MyApp/MyApp.csproj]

[baronfel]

Have you been able to try the same command without the symbols flag?

[ndw]

Builds, but is still corrupted:

$ dotnet publish --configuration Release -r osx-x64 --self-contained true -p:PublishSingleFile=true -p:IncludeNativeLibrariesForSelfExtract=true -p:Version=1.0.0 -p:packageVersion=1.0.0
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  MyApp -> /private/tmp/MyApp/bin/Release/net6.0/osx-x64/MyApp.dll
  MyApp -> /private/tmp/MyApp/bin/Release/net6.0/osx-x64/publish/

/tmp/MyApp via .NET 6.0.201 took 6s
$ codesign --timestamp -o runtime --force --verify --verbose --sign MYKEYIDHERE bin/Release/net6.0/osx-x64/publish/MyApp
bin/Release/net6.0/osx-x64/publish/MyApp: replacing existing signature
bin/Release/net6.0/osx-x64/publish/MyApp: signed Mach-O thin (x86_64) [MyApp]

/tmp/MyApp via .NET 6.0.201
$ bin/Release/net6.0/osx-x64/publish/MyApp
Failed to create CoreCLR, HRESULT: 0x80004005

[baronfel]

@ndw can you try the COREHOST_TRACE=1 that's described in this runtime issue? It looks similar from a top-level and might point to something more fundamentally wrong.

[ndw]

Right. I did this a couple of days ago, but I guess GitHub ate my comment because it was more than 64K.
trace.log

[baronfel]

That's for the publish command itself, I think we'd need to see the same trace logs for actually running MyApp.

[ndw]

Sorry I misunderstood. Here you go.
run.log

[ndw]

Ping?

[baronfel]

Next best thing is something I found in this issue - setting COMPlus_EnableDiagnostics=0 to disable all of the named-pipe generation that's done by the runtime to enable attaching to the running process. If that makes your app run, we should comment on that thread to that effect. Creating these named pipes (potentially on a read-only file system?) may have some edge cases not yet known or tracked.

[devon94]

Also seeing this issue as well. Executable works fine until it is signed.

Update: I found this article and it helped https://learn.microsoft.com/en-us/dotnet/core/install/macos-notarization-issues#default-entitlements. I was signing the dotnet executable with entitlements, but I was missing a few of the required ones. This was the plist that worked for me.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http:https://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
    <key>com.apple.security.cs.disable-executable-page-protection</key>
    <true/>
</dict>
</plist>

[ndw]

I'm returning to this task much later than I anticipated. I'd like to try the plist solution proposed, but I'm unclear on where this plist should be placed and what it should be called...

I think I figured this out: it's passed to codesign with --entitlements. Apologies for the noise.

[devon94]

@ndw We keep an entitlements.plist in the project folder. I believe it can be placed anywhere. codesign's --entitlements arg accepts a path to a plst file.

Heres a simplified version of the command we use that works

codesign --force -s your_identifier --keychain /path/to/your.keychain --entitlements ./entitlements.plist --options runtime --deep /path/to/your.app

[ndw]

Thank you @devon94. I believe I finally wrestled the whole process to the ground. Breadcrumbs for the next traveler: https://dev.saxonica.com/blog/norm/2022/12/21-net6.html