Actually, for a quick second when you invite someone they can open up their friends menu and invite someone from their list to your party, so chances are the person you invited knew that little trick and that's what happened.
I do that sometimes when I get invited to a new party, I just invite the people that were in my old party to theirs so we have better chances of always having a full party.
When someone invites you to a party just quickly press the 1 button and your friends list will open allowing you to invite people to that party, but it will automatically close when you fully connect so do it quickly lmao it usually doesn't stay open for more than 3 seconds.