I finally figured out it has nothing to do with my internet, but when I went to play today, it started fine, and then 5 seconds in it lagged when I tried to cast skills, and then my internet went dead (well, it slowed down really badly), I then logged off, left it for a couple of seconds and my internet was fine.
It is good to know that others are having this same problem, as it means it isn’t to do with my usually horrible internet connection.