The trait is working properly with the glyph of lesser elemental. When the elemental is up the number you see counting down is the remaining time on the elemental. It is not until after the elemental dies or expires that the cool down starts.
That said, it does appear that this trait is bugged with the glyph of elemental (the elite version). When you use the skill it starts the duration countdown from 96 (which is what the recharge time is supposed to be, not the duration) and when the elemental is gone it still takes 120 seconds to recharge (it’s supposed to be 96 with the trait).
