What about allowing our characters to have the choice of either Attunement Swap OR Weapon Swap?
Keep Attunement Swap as it is now with access to 4 attunements, no change required there.
With Weapon Swap, can have 2 sets of weapon equipped, but locked into 1 attunement. Giving the advantage of more skills within an element, but limited by the overall number of available skills.
The option to choose would be available in the Hero Panel, but only when out of combat, like it is currently when wanting to switch equipment.