Some gotchas with Go and the psx and cap packages
Some gotchas with Go and the psx and cap packages
It turns out that Go sometimes causes OS threads to block indefinitely waiting inside the kernel. This is most severe when the Go runtime decides to declare an *os.File non-pollable and serializes reads inside the kernel. That is a fancy way to say that it devotes a whole thread to running the syscall.Read() function.
Normally, the runtime does a lighter weight thing and has a single thread waiting for all pending reads to be actionable with a polling abstraction, and only calls syscall.Read() when it is guaranteed to promptly return data. This feature allows the go runtime to avoid allocating dormant memory buffers for idle reads, which is a huge win for scalability. It is also a potential gotcha for using the psx package in CGO_ENABLED=0 mode, and by extension the cap package.
What is surprising to learn is that simply accessing (*os.File).Fd() has the undocumented side-effect of forcing that file to be read-blocking. And a blocked thread like this would deadlock the syscall.AllThreadsSyscall() function, which is used by the psx package.
Sometimes doing operations on a file descriptor (the result of calling (*os.File).Fd()) is seemingly needed. One such example use case is that of syscall.SYS_IOCTL. This feature makes slightly abnormal OS features accessed through file descriptors possible. The following is a way to perform an ioctl call on a file descriptor that does not have the side-effect of turning the file into a non-pollable one (it comes from some debugging of a reported deadlock failure - which remains broken in go1.16 and go1.17, but was fixed in go1.18):
//go:uintptrescapes
func ioctlFile(f syscall.Conn, a1, a2 uintptr) error {
s, err := f.SyscallConn()
if err != nil {
return err
}
s.Control(func(fd uintptr) {
if _, _, e := syscall.Syscall(syscall.SYS_IOCTL, fd, a1, a2); e != 0 {
err = e
}
})
return err
}
It is the SyscallConn() abstraction of the syscall.Conn interface (implemented by *os.File, net.Conn etc.) that gives visibility to the file descriptor for an inline function to leverage. In this case that of syscall.SYS_IOCTL functionality.