Repository navigation
all: ensure that Go toolchain meets Apple’s notarization requirements #34986
Description
What version of Go are you using (go version)?
$ go version go version go1.13.3 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env GO111MODULE="on" GOARCH="amd64" GOBIN="" GOCACHE="/Users/artyom/Library/Caches/go-build" GOENV="/Users/artyom/Library/Application Support/go/env" GOEXE="" GOFLAGS="-ldflags=-w -trimpath" GOHOSTARCH="amd64" GOHOSTOS="darwin" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/artyom/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/Users/artyom/Library/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/Users/artyom/Library/go/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/artyom/Repositories/artyom/rex/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lb/3rk8rqs53czgb4v35w_342xc0000gn/T/go-build499526677=/tmp/go-build -gno-record-gcc-switches -fno-common"
What did you do?
Installed go from https://dl.google.com/go/go1.13.3.darwin-amd64.tar.gz and then tried to use it to build project. This is on macOS Catalina 10.15:
Darwin MacBook-Air.local 19.0.0 Darwin Kernel Version 19.0.0: Wed Sep 25 20:18:50 PDT 2019; root:xnu-6153.11.26~2/RELEASE_X86_64 x86_64
What did you expect to see?
Go works as usual.
What did you see instead?
On first go call I'm presented with OS pop-up that "macOS cannot verify that this app is free from malware", with options either to discard this window (the process is killed then) or move the binary to trash. After whitelisting this binary via Preferences → Security & Privacy → General → Allow Anyway, go can be called fine.
Next I tried to build project with go install and got the came pop-up for "compile" tool.
Activity
artyom commentedon Oct 18, 2019
To be able to run updated Go 1.13.3 and compile my project with it, I had to whitelist so far these binaries:
Whitelisting each takes the following:
bradfitz commentedon Oct 18, 2019
Good times.
/cc @golang/osp-team
bradfitz commentedon Oct 18, 2019
Related: #34748 (we need a Catalina builder)
networkimprov commentedon Oct 18, 2019
Discussion on Hacker News: https://news.ycombinator.com/item?id=21179970
Also, Catalina is reported to be a disastrous upgrade.
andybons commentedon Oct 18, 2019
I’ll take a look at this.
dmitshur commentedon Oct 18, 2019
I can reproduce on my personal MBP with macOS 10.15 (19A602) when installing from the .pkg installer too, however it's a different (less intrusive) experience.
The "macOS cannot verify that this app is free from malware" prompt appears when first running the installer. After accepting and installing, Go binaries such as
go,compile, do not produce the same prompt. Also, re-running the .pkg installer doesn't ask again.andybons commentedon Oct 21, 2019
In the process of submitting the installer to Apple's notary service, but the process is very slow due to a service issue on Apple's side and our internal tooling keeps timing out. Diagnosing this has been difficult.
It may be that we will need to enable the Hardened Runtime which likely requires a newer version of Xcode that we're using to build releases and an audit of whatever entitlements are required for our binaries to run.
@aclements can you take a look at the link above and let me know which you think we'll need (if any)?
aclements commentedon Oct 21, 2019
@andybons, I don't think we need any of the runtime exceptions from that page. The only one that we may even potentially want is "Disable Library Validation Entitlement", which would let users use the the
pluginpackage without signing their plugins, but it seems like developers probably should sign their plugins.vp2177 commentedon Oct 22, 2019
The (
.pkg) installer doesn't even start for me. Is that the same issue, or unrelated?"go1.13.3.darwin-amd64.pkg can't be opened because Apple cannot check it for malicious software."
Meanwhile, the older Go version I had installed from before the OS X Catalina update, continues to work fine, no warning dialogs.
andybons commentedon Oct 22, 2019
@vp2177 it’s the same issue.
andybons commentedon Oct 22, 2019
Good news: according to Apple, we don’t have to enable the hardened runtime until January 2020. That said, we’ll likely have to start signing the binaries themselves (not just the installer as we do now), so this issue should stay open even if we fix the problem in the short term via notarization of the installer.
44 remaining items
gopherbot commentedon Nov 21, 2019
Change https://golang.org/cl/208227 mentions this issue:
[release-branch.go1.12] cmd/vendor: remove _test.go and testdata filesgopherbot commentedon Nov 22, 2019
Change https://golang.org/cl/208266 mentions this issue:
cmd/release, cmd/releasebot: clean up unused codecmd/release, cmd/releasebot: clean up unused code
[release-branch.go1.12] cmd/vendor: remove _test.go and testdata files
andybons commentedon Nov 26, 2019
I've run an altered
all.bashthat signs (with the hardened runtime enabled) the toolchain binaries and all tests passed on macOS 10.15.1.We plan to enable the hardened runtime for the 1.14 beta (scheduled for early December) and see if anything breaks. If everything looks good, we'll enable it for all releases moving forward.
toothrot commentedon Dec 16, 2019
Just to reiterate what Andy said, the hardened runtime will be enabled for the go1.14 beta release.
cagedmantis commentedon Jan 30, 2020
I've confirmed with @bcmills that #36572 should not be considered a blocker for go1.14rc1. I'm going to recommend that we close this issue. @andybons @dmitshur @toothrot
dmitshur commentedon Feb 4, 2020
Sounds good. With the hardened runtime enabled as of Go 1.14 beta 1, I don't believe there's anything left to do here other than to watch for any issues if they're reported, but new issues can be opened for that.
I'll let one of @andybons or @toothrot confirm and close this issue since they're currently assigned to it.
andybons commentedon Feb 4, 2020
Yes this looks to be fixed. If we discover any issues during the notarization stage of a release we'll open a new issue.