The biggest reasoning behind not washing petals and aerial parts is that it adds more water to the situation. If you are drying the plants, it can be tricky to do while maintaining the best integrity and medicine in the plant, even without having extra water on them. If you are making medicine with the fresh plant that has just been washed the water will be on the surface and added to your medicine, diluting it or causing earlier spoilage. Even with non washed fresh plant material, one needs to be cautious with the water found within the fresh plant in terms of preservation of your final product.
The other factor with washing aerial parts, especially flowers, is that they are quite delicate and can be damaged/start to breakdown very easily. Washing and patting dry could cause this to happen. With roots, they are much more dirty and they are much more sturdy, so can handle a wash, pat dry and more intense ways of drying the plant material.
I completely understand if you feel more comfortable about washing your plants, we also have a myriad of animals traipsing around and through our property and sometimes dust. I don’t wash my aerial parts unless they are particularly dirty or were harvested from low to the ground, which is very rare. For bugs, I let the plant material sit for an hour or two and then follow up with a gentle shake out of the flower heads. I also do a little dust off sometimes.
When you strain your final product, it helps to use a fine mesh or cotton muslin and a coffee filter to really keep dust, dirt, bugs out of the end product.
I hope this gives you the reasoning and you can then decide how to continue on. It’s fine to make the choices that feel better for you as well!