You can enter the Priory basement via one of the grates around that central glowing pillar thing, which gives you a standard instance prompt.
If you want to get back into the big library area, go to Gixx’s office and talk to Ogden, who can transport you there.