It's not even about encroachment. Any black person can go to any country in West Africa and blend in with the indigenous population. All you've to do is pick whatever ethnic group you like, change your name to one of their tribal names, and embrace the culture/people.
However, one thing Africans (just like other indigenous people everywhere else) don't joke about is land. People's heritage, history, culture, and advancements are tied to their ancestral land and it represents who they're.
No, there was NO educated indigenous population in Nigeria when the slaves returned from Brazil, Cuba, and Sierra Leone (Anglo slaves were taken to Freetown in Sierra Leone before they found their way to Nigeria) in the 1800s. The literacy rate of Nigeria in 1920 was less than 10% and it'd be less than 1% in the 1800s. The returnee slaves were actually the first core of educated class in Nigeria and they never used that as a yardstick to oppress the indigenous. Rather, they were at the forefront of everything good that happened to Nigeria, coupled with a lot royal kids and few Nigerian entrepreneurs.
You don't really know much about precolonial/colonial Nigeria. Perhaps one day when I've free time, I might create a thread to discuss it, with some PDF files. The returnee slaves were the unsung heroes of colonial Nigeria and their legacy is what made Lagos what it's today. They built the renaissance there and gave the city the life it has now.

were you alive then nikka?
Stahhhhp it. I can school you on almost every returnee or royal family from egbaland to lagos to abonnema to calabar - which is what I belong too.
When I said an educated elite class- I was obviously talking about the children of Kings, chiefs and merchants. those were who the returnees mixed with not regular farmers and fishermen.
In my own family the indigenous people we married were of royal lineage Alake of egbaland, Obong of Calabar and the Amanyanabo of abonnema. Intermarriage between returnees and non elite did not start happening till 2 generations ago.
If I say an elite indigenous class - I mean the likes of Sir Adetokumbo Ademola- who's father was the Alake of egbaland and mother of Alajika ( Brazilian descent) who married a woman from the Vaughn family (returnee)- who was of AA descent.
^^^ this was more common than a returnee marrying the child of a fisherman or farmer.
point being liberia's indigenous class were not as elitist as
Added - Also racism in America hindered liberias indigenous population from excelling in academics. Liberia being an American colony did not have indigenous people leaving Liberia to study in white schools in America.
Unlike Nigeria, Ghana, Salone- all British colonies- had indigenous elite class of people going to Cambridge, Glasglow, British universities.