The first time I tried this I had the same problems everyone is mentioning above. I even tried to use the cannon, but there was big boat mast in the way and nothing would come in to the small area I was able to shoot at.
The second time I tried it, I cleared the walls as quickly as possible, so that I could get to the courtyard before all the NPCs down there died. I then immediately focused on the summoning acolytes. Having the NPCs alive to keep some of the mob swarms off me was helpful. After that wiping up the rest was fairly simple.
This was a few weeks ago that I did it, so hopefully some changes have been made since.