libusb is so cursed :/
there is no guarantees that udev rules are triggered before calling libusb_open, creating a race condition.
This can be "fixed" by retrying the libusb_open call if it returned LIBUSB_ERROR_ACCESS ...

also libusb segfault if you call libusb_get_device(handle) with a handle that refers to a disconnected device...

edit, this might be my fault