It seems as though you have a specific interest in phytochemistry, which is nice to see as it’s one of the many ways to get to understand and know plants better.
This is quite a complex and detailed topic but to keep it simple, not all saponins are toxic or equally toxic as cardiac glycosides. They are a complex and chemically diverse group of chemical compounds found in plants that create soap bubbles, essentially. They are found quite commonly in plants and nature, for example, have you ever seen foam on a river bend or in the ocean, this is actually caused by saponins that are in the water from organic matter (algae, decaying leaf litter, bogs etc.) that has been aerated when the water gets churned up. While saponins are not always technically toxic, they are often irritating to skin and digestive tissues. They are a broad category of compound and do not all act the same.
The same goes for glycosides, they are an even broader category of chemical compounds of which saponins are one part (as you’ve mentioned). Specifically, cardiac glycosides are very potent and toxic at low doses for someone with a healthy heart. For those with certain cardiac conditions the cardiac glycosides/lily of the valley can be used therapeutically, but I don’t say this lightly, it is NOT a medicine for anyone but a fully trained Herbal Practitioner to use. This is ominous medicine, it can be easily misused and deadly if so. There are pharmaceutical forms of this medicine now that are used reliably due to the accuracy of dose.
To tie this up quite quickly not all glycosides are toxic, but some are definitely toxic, it’s a large category of compounds. It’s important to know the specifics of the compounds found in each plant, how the plant is used and the nuance of who the herb may be useful for. Beyond the mention of lily of the valley in this class, the Home Herbalist and Community Herbalist Programs contain relatively safe herbs and do not teach about plants with narrow therapeutic ranges/toxic plants. These toxic plants should not be used by a herbalist who is not trained as a professional clinician and even then it can be concerning.
If you are personally interested in learning more about phytochemistry in particular, a great book to read is Herbal Constituents (2nd ed.) by Lisa Ganora
I hope this helped to clarify, although i think it may have just opened up more questions I’m sure. Phytochemistry is a full full course on it’s own.