It’s not a bug, and I don’t believe it’s related to the quality of the code.
It’s a non-trivial game design issue that they haven’t had the time to fully address as of yet.
The biggest problem is that there is an NPC that follows the dungeon starter (not the party leader, as some have suggested, but the player that starts the dungeon). Similar to the personal story, the person that starts the dungeon is essentially who the dungeon is created for. There just happen to be four other people with them. This is most noticeable in the cut scenes, where the dungeon starter and only the dungeon starter is talking with the NPC.
It’s not the easiest problem to solve, as it ideally requires an elegant solution. Do they switch to another person and make that person the one that the dungeon talks with and what-not if the original person leaves? What if no one in the party has the dungeon? What if they just use the entire party, or a random person, in the cut scenes?
These are just some possibilities, but it’s not a small nor trivial game design change, and thus it will take a bit of time. I’m sure it’s on their priority list as something to address, but, as of now, we don’t know how long it will be. We can’t expect it to be a quick fix, no matter how frustrating and seemingly trivial the issue seems.