I think that X and Y DNA is a pretty straight forward process, so it is likely true. African Ancestry only shows four groups in Benin (Fon, Adja, Barbia and Somba). It is hard to tell historically where some of these Mande people started their Southern pilgrimage from. It is also possible that one of your grandfathers is actually from a tribe in the the modern country of Benin. It would not be the first time that somebody thought that their ancestors were from one place, but they were in fact from a different place.
my bad it wasn't benin, it was the Ateke from Gabon. Like I said I'm not doubting having Non Dan ancestors, my statements are more so the migration paths dont match up. Now a possibility is that I could have had ancestors from a Tribe that broke off Ateke or their ancestral tribe and they got with one of my Mande ancestors.