Well, after an incident or two, would it not occur to you that this person is just not built for confrontation? Not everybody you hang around has to be a soldier, if you really have love for him let him know that in certain situations you prefer to not have him around because you understand that they're not comfortable getting down like that.
Not everybody is a fighter. Some people are complete pacifists, some only to protect, others only fight when backed into a corner, and then you have those who welcome confrontation and are usually always into it. A part of true friendship is accepting the fact that there's things about that person you don't agree with. You never know, that person may actually want to jump in, but is simply too afraid, that's not who they are and never will be.
Decide what's best for you and keep the most positive people around you. And I hope you also find more peace around you so that you no longer have to risk your health over petty disagreements. If you feel like this person simply never backs YOU up, maybe it's time you move your separate ways. The person you knew 10 years ago no longer exists, and vice-versa, as time passes we learn, grow and change.
The only thing that is constant in life, is change. Make your decision outta love, and do the right thing breh