We are told quite clearly, at the point in the ordinances where we are (symbolically) clothed in the Garment that it (the Garment) is to be worn throughout our lives.
What possible reason we would then have for taking it off, other than for pro-functionary things like washing ourselves and the Garment. When we marry many put on a wedding ring. Generally we only take it off to wash, my wife's never comes off. Generally anyone taking of their wedding ring is doing for a nefarious purpose, or is at least seen to be doing for such a reason.
As read at TR interviews, "wearing the Garment is an outward expression of an inner commitment to follow the Saviour, Jesus Christ" (from memory).