If I start fights with a phantasm skill in PVE before the target has otherwise been engaged, the phantasm appears to die on the first hit that it takes. If I otherwise start the fight with an auto attack or other direct damage skill, the phantasms summoned after that tend to last 3-5 hits from most PVE mobs.
This has happened since game release. My mesmer is currently mid-60s.