Fix driver removal upon device removal

The driver itself shouldn't be removed but only the device specific
reference to it.
1 file changed