Stop trying to hold on to people who don’t want to stay. It may not seem fair and you may not have any idea why they wouldn’t want to stick around, but they have every right to go, if that is their decision. So let them.
Don’t beg them to stay or plead your case for why they shouldn’t leave. Let them go! It is an honor and a privilege to be allowed in your world and you should treat it as such. And if they, on their own, decide they want to leave your kingdom, don’t let the drawbridge hit them on the arse when they leave.