A lot of us in the world now get our drnking water through PVC pipes, sit on PVC furniture, wash in PVC sinks and showers. The amount you get through a sex toy are negligible compared to these sources. The reason we use PVC is because it's so chemically inert. Surgical gloves are now made of PVC for those allergic to natural latex.
The phthalates you get from your athletic shoes or from inhalng that new car smell in your SUV are many times what you'd get from a dildo or artifical vagina.
"Jelly" items are made of various silcone ploymers. That stuff they leak is similar to what's in silcone breast inserts, which has been tested to death and found to be innocuous, lawsuits aside.
Unless you have an acute allergy to plasticizers (and you'd probably know it long before you were old enough to fool with sex toys) (and many many more people are allergic to nature's own natural organically grown latex than are allergic to synthetic plastics), I really think this is a non-issue.