I just checked and you are right. The fury buff does not refresh either.
For me the buff stays on the phantasms for about 8 seconds and does not refresh.
It is impossible to have both retaliation and fury on the same phantasm.
(even if, by changing the trait, it is possible to have two phantasms with a different buff on each one)