The Prophet of Allah, sallallaahu ‘alayhi wa sallam, never haughtily rejected that one of his servants eat with him; in fact, he encouraged his nation to eat with their servants

“When your servant brings your meals to you then if he does not let him sit and share the meals, he should at least give him a mouthful or two mouthfuls of that meal, or a meal or two meals, as the slave is the one who has prepared it and undergone the heat in cooking it.”

“If one of your servants prepares a meal for you and brings it, while being the one who cooked it and endured the heat and smoke when cooking it, then allow that servant to sit and eat with him, and if the food is insufficient for those who are sitting to eat it, then let one of you spare him at least a morsel or two.”