This isn't as racially tinged as much as it is insensitive. Lets not overplay our cards here.If you feel strongly enough about this, you should file a formal complaint to hr that your boss is making inflammatory, racially-tinged statements that are creating an uncomfortable working environment.
If you dont care, then just suck it up and keep it moving.
What he said was insulting and demoralizing, but not "RACIST"
Remember, what you can PROVE matters.