This is a great question. Is there a proper winter where you live? i.e. frosts that will kill back most plants?
If not then I’d say leave your plants growing for as long as they will.
If you do get frosts that cause most plants to die back, I personally tend to leave most of the plants and dead plant material around until spring, especially those with seed heads as these tend to make a great perching spot and food source for birds. Also the material tends to create habitat for small creatures to live in over the winter.
You could also cut back some plant material if you wish for it to used as mulch and nutrition to go back into the soil over winter. Or you could use it to mulch heavily around some perennial plants that may be sensitive to the cold, keeping their shallow roots safe from freeze thaw cycles and or cold winds.
The only downside (that I can think of) leaving plant debris over winter is that it can give a place for pests to overwinter (e.g. squash/shield bugs).
So I suppose the answer is that I personally do a bit of a mix, but I do leave most of the perennials with seed heads standing over winter, then tidy them in the spring.