Rambam teaching astronomy, rejecting a student over a lack of knowledge of astronomy

I’m not here to ask about Rambam’s cosmology or anything, but I’ve heard a story that Rambam rejected a student because the latter wasn’t learned in astronomy at the level that it was taught in Rambam’s beit midrash. I Googled, and couldn’t find anything about the story, just schools that bear Rambam’s name teaching astronomy. Does anyone know where I can find that story, or if it’s apocryphal?