There seems to be a pretty significant bug in the way the tempest and arcane specialization interact.
When overloading, the cooldown of the attunement is correctly reduced to 17.5s or so, but when swapping out of that attunement after finishing the overload, the cooldown is bumped up by 2.5s, cancelling out the arcane reduction.