Certain (elite?) skills are bugged when your level scales down. They don’t provide the proper benefit that they do if you don’t scale down.
The reason I know this is true is because if I use an ability like Become the Bear in an area where my level isn’t being affected, my HP goes up a very large amount (around double). If my level is modified AT ALL (even -1) then I only get 1-2k HP and it’s a very small increase. I would assume this is not only affecting vitality, but also the other stat benefits it grants.
Biggest examples I have include:
Become the Bear
Tome of Courage
Likely also affects:
Tome of Wrath
All other Norn forms
Other stat scaling skills