You're mistaking common ragwort for Oxford ragwort. The latter grows only in waste places, never in meadows or pastures. It is unpalatable to animals, like hundreds of plants, but is not a threat. .