[Please note: this is from the perspective of a WvW Roamer who usually plays either Guardian or Thief. Also, for the purpose of saving time here, I’m just going to assume Dagger-mainhand for the hypothetical thief I refer to.]
TL;DR Version: Shadow Refuge should be based around giving thieves more sustainability in a fight instead of being a means to simply reset a fight. (Skip to the boldface text below.)
In it’s current state, Shadow Refuge is an extremely boring skill to play against. There is not much counter-play for it (except for a select few skills that can knock an enemy out of it). When traited, Thieves only need to stand in it for 4 seconds, and then they are free to do whatever they want with 16 seconds of stealth. That is a huge amount of stealth for very little “cost” (i.e. using only 1 skill with a 60s cooldown and standing in a relatively large area for only 4 seconds). And even if you damage them a lot while they’re in the Refuge, so what? They still have 16s of stealth and there’s nothing you can do about it as long as they don’t get downed. This essentially gives Thieves the ability to reset any fight without even having to make much of an effort to run away.
As a Guardian who doesn’t particularly like using Shield and a Thief who doesn’t particularly like using Shadow Refuge (in its current state), whenever I see an enemy Refuge go down, I know that the only thing I can really do is run away to prevent them from opening with an unpredictable Backstab. This is how Shadow Refuge is against most classes. When you see it go down you usually just have to accept that the Thief is going to escape or reset.
Shadow Refuge should be changed to help give Thieves more sustainability in a fight rather than simply resetting the fight altogether. Right now it’s all to often just a skill that terribad Thieves use so they can easily escape once they realize you’re not an easy kill (no, I’m not saying everyone who uses Shadow Refuge is bad 
).
As someone who plays Thief often (I usually prefer roaming on Thief instead of Guardian for obvious reasons), I would like Shadow Refuge a lot more if it lasted 5 seconds rather than 4, only gave 7 seconds of stealth total (8 when traited for Meld with Shadows), but healed for 4000+ (or something to that effect). The healing should only be active while stealth (so you can’t put down the Refuge and continue fighting if you want the healing). On a side note: I also do not believe Shadow Refuge should affect downed players, because it often makes it impossible to stomp/kill a downed player before they are revived (especially in outnumbered fights).
Why do I think this would make Shadow Refuge a more interesting skill to play with/against?
1. Thieves interested in staying in a prolonged battle can view this as a huge buff.
2. This version is based around the idea of the Thief staying and fighting, rather than just running away and resetting with 16 seconds of stealth. It gives them more health back, which allows them to return to the fight quickly and have a little more sustainability. Because it would give 2-3 extra seconds of stealth (after its 5 second duration), the Thief would still have a chance to reopen on their opponent with a Backstab.
3. It allows for more counter-play besides just knocking the Thief out with a select few skills. Thieves will have to focus more on evading their opponents damage while remaining in the Refuge so they can benefit from the healing and then reopen. In other words, it requires more skill from the Thief than Shadow Refuge currently does to be beneficial.
4. The Thief is a little more predictable. It will be clear that the thief is probably nearby even after the Refuge ends. Additionally, any Backstab coming from the Refuge’s stealth will be possible for a skilled player to avoid because the Thief will only have a short (and, may I add, reasonable) time-frame in which to pull it off. The Thief can, of course, chain stealth from another source to make this Backstab less predictable.
5. Thieves won’t be able to self revive with 16 seconds of stealth anymore! This means that when I see a Refuge go down I won’t have to spend the next 20 seconds running around looking like an idiot stupidly spamming skill 1 in a vain attempt to prevent a Thief who probably is not even downed from self-reviving! Yay! 
6. It’s actually a… “Refuge.” Currently Shadow Refuge just makes you stand in one spot for 4 seconds and then you can do whatever the kitten you want for 16 seconds. Not much of a “Refuge.” More like an extended vacation from the fight. :P
Anyway, that’s my opinion. What does everyone else think of Shadow Refuge as it is now? All constructive criticism is welcome.