gl: Don't steal callers reference when setting non-floating elements via properties

Introduced by fbef9220d3dc2f785081c4766901aab2ecfaed10. The code there
is only correct for elements we get from signals.
4 files changed