runtime: re-enable TestCgoCallbackGC flaky on FreeBSD

I see a number of recent TestCgoCallbackGC flakes on FreeBSD

amd64:
https://build.golang.org/log/c47d9b4fe69df9c91af66122e7633f77cb623794
https://build.golang.org/log/2c93bb5f172cf8082a80bd567628bcea4f437c20
https://build.golang.org/log/92ad760a01e814bb0021486856b4d3123a4a11b7

386:
https://build.golang.org/log/47a3e8462838b94a69459a3e56de5ea852b5e86e
https://build.golang.org/log/d6604ec508fd4166c46bd6fb3e04a4b0fb945959
https://build.golang.org/log/85e6aab2ff5692ee7372f03dd187fc555f8a685c

Most are:

--- FAIL: TestCgoCallbackGC (93.64s)
    crash_test.go:105: testprogcgo CgoCallbackGC exit status: signal: quit (core dumped)
    crash_cgo_test.go:61: expected "OK\n", but got:
FAIL
FAIL    runtime 147.968s

Some are:

--- FAIL: TestCgoCallbackGC (102.14s)
    crash_test.go:105: testprogcgo CgoCallbackGC exit status: signal: quit (core dumped)
    crash_cgo_test.go:61: expected "OK\n", but got:
panic: test timed out after 5m0s

goroutine 232809 [running]:
panic(0x81f2c80, 0x39220b20)
    /tmp/workdir/go/src/runtime/panic.go:500 +0x334 fp=0x392b1fa0 sp=0x392b1f5c
testing.startAlarm.func1()
    /tmp/workdir/go/src/testing/testing.go:918 +0xef fp=0x392b1fe0 sp=0x392b1fa0
runtime.goexit()
    /tmp/workdir/go/src/runtime/asm_386.s:1612 +0x1 fp=0x392b1fe4 sp=0x392b1fe0
created by time.goFunc
    /tmp/workdir/go/src/time/sleep.go:154 +0x30

goroutine 1 [chan receive]:
runtime.gopark(0x8246704, 0x39632d30, 0x8224051, 0xc, 0x17, 0x3)
    /tmp/workdir/go/src/runtime/proc.go:259 +0x12d fp=0x39233cd4 sp=0x39233cc0
runtime.goparkunlock(0x39632d30, 0x8224051, 0xc, 0x17, 0x3)
    /tmp/workdir/go/src/runtime/proc.go:265 +0x45 fp=0x39233cf0 sp=0x39233cd4
...

Why did these start recently?

I think these are other versions of #15658 and #16136. There is something wrong on FreeBSD but I don't understand what it is and I have not been able to recreate it myself.

If anybody has a FreeBSD machine I can ssh into that would help.

Our FreeBSD builders run on GCE. I can create you an instance there.

Our FreeBSD builders run on GCE. I can create you an instance there.

That would help if you have time. Thanks.

@ianlancetaylor

@ianlancetaylor, done. Emailed you details.

I think I was incorrect in thinking that this was related to the other FreeBSD problems. The other FreeBSD problems are less frequent, and I have not been able to recreate them at all.

I now think this is the same as issue #13926: a problem that arises when there are many simultaneous callbacks from C to Go, which is exactly what the CgoCallbackGC test case does. The problem is that on FreeBSD you can surprisingly large delays during the callback. My theory is that when enough of those delays occur, the test times out. That is consistent with the failures reported on the builder, which are in all cases the program crashing from SIGQUIT; SIGQUIT is sent by the runtime test harness when the test program runs for more than 1 minute (runTestProg in runtime/crash_test.go).

Working on a patch.

CL https://golang.org/cl/25047 mentions this issue.

TestCgoCallbackGC is still failing on FreeBSD on the builders. The change did not fix it, at least not completely.

We may just have to punt on this for Go 1.7, which should be okay since IIUC this is not a regression in Go 1.7. (Or is it?)

To the best of my knowledge this is not a regression in 1.7.

@adg

@adg, @broady, should we at least document this as a known issue in the release notes for Go 1.7? And then punt this to Go 1.8 for actually fixing.

It'd be a pity to ship something with known crashes for a "first class port". I'd rather warn FreeBSD users about this bug so they don't deploy something with cgo callbacks in production and see crashes.

Thanks for trying it so thoroughly. Sounds like this issue can be closed.

@ianlancetaylor

@ianlancetaylor, did you mean to close this? It sounds like we need to at least remove the t.Skip for this test on FreeBSD.

CL https://golang.org/cl/33903 mentions this issue.

CL https://golang.org/cl/33926 mentions this issue.

doc/go1.8: remove mention of #16396

Hello,
I've starting running builds with CGO_ENABLED=1 on freebsd-arm-paulzhol and it fails consistently:

ok  	regexp/syntax	4.107s
--- FAIL: TestCgoCallbackGC (180.11s)
	crash_test.go:105: testprogcgo CgoCallbackGC exit status: exit status 2
	crash_cgo_test.go:61: expected "OK\n", but got:
		SIGQUIT: quit
		PC=0x20263cd0 m=0
		signal arrived during cgo execution
		
		goroutine 166 [syscall, locked to thread]:
		runtime.cgocall(0xc0c78, 0x30a7d7c4, 0x0)
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/cgocall.go:131 +0x130 fp=0x30a7d7ac sp=0x30a7d790
		main._Cfunc_foo()
			_/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/_obj/_cgo_gotypes.go:176 +0x38 fp=0x30a7d7c0 sp=0x30a7d7ac
		main.CgoCallbackGC.func2(0x30a5a040)
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/callback.go:81 +0x14 fp=0x30a7d7d4 sp=0x30a7d7c0
		runtime.goexit()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/asm_arm.s:998 +0x4 fp=0x30a7d7d4 sp=0x30a7d7d4
		created by main.CgoCallbackGC
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/callback.go:83 +0xd8
		
		goroutine 1 [chan receive]:
		main.CgoCallbackGC()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/callback.go:86 +0x114
		main.main()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/main.go:34 +0x1ac
		
		goroutine 17 [runnable, locked to thread]:
		runtime.goexit()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/asm_arm.s:998 +0x4

System is an ALLWINNER A20 running FreeBSD 11.0-RELEASE-p2. I've noticed CGO_ENABLED was off by default because the bootstrap 1.4.3 compilers need to be built with CGO_ENABLED=0.

@paulzhol

@paulzhol This issue is closed and the problem you are reporting has different symptoms. Please open a new issue with the details. Thanks.

@ianlancetaylor

@ianlancetaylor I've opened #18811 but I do have the following in my log which look like the "Most are:" example in the original issue:

ok      regexp/syntax   4.205s
--- FAIL: TestCgoCallbackGC (204.17s)
        crash_test.go:106: testprogcgo CgoCallbackGC exit status: signal: quit
        crash_cgo_test.go:66: expected "OK\n", but got:
FAIL
FAIL    runtime 427.836s
ok      runtime/debug   0.088s

But the builder failed to push that:

2017/01/27 02:25:42 Failed: exit status 1
Build complete, duration 41m53.837957541s. Result: error: exit status 1
 RunTime:41m53.837957541s Builder:freebsd-arm-paulzhol PackagePath:]
2017/01/27 02:25:45 Post https://build.golang.org/result?builder=freebsd-arm-paulzhol&key=XXXXXXXXXXXX&version=1: dial tcp: lookup build.golang.org on 192.168.XXX.XXX:53: server misbehaving

Navigation Menu

Repository navigation

runtime: re-enable TestCgoCallbackGC flaky on FreeBSD #16396

19 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

runtime: re-enable TestCgoCallbackGC flaky on FreeBSD #16396

Description

Activity

ianlancetaylor commented on Jul 17, 2016

bradfitz commented on Jul 17, 2016

ianlancetaylor commented on Jul 18, 2016

bradfitz commented on Jul 18, 2016

ianlancetaylor commented on Jul 19, 2016

gopherbot commented on Jul 19, 2016

ianlancetaylor commented on Jul 21, 2016

adg commented on Jul 25, 2016

ianlancetaylor commented on Jul 26, 2016

bradfitz commented on Jul 27, 2016

19 remaining items

ianlancetaylor commented on Dec 5, 2016

bradfitz commented on Dec 5, 2016

gopherbot commented on Dec 5, 2016

gopherbot commented on Dec 6, 2016

paulzhol commented on Jan 27, 2017

ianlancetaylor commented on Jan 27, 2017

paulzhol commented on Jan 27, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions