It's not some made up justification, it's the truth. The wildfire Cersei used to blow up the Sept was just a portion of what the mad king had his pyromancers stash around the city waiting to be used. If Jamie doesn't kill him the city and the half a million people in it would've been burnt to a crisp.
Like I said, if Daeny can't see the contradiction in pressing Jamie over that when she killed Sam's father for less Snow will give her that pill she needs to swallow.
I think what he's trying to say is Jamie uses the wildfire explanation as his excuse for breaking his oath when the real reason he killed the Mad King was to protect his house because he didn't want to kill his father. I don't agree with that because Jamie's dialog seemed sincere to Brienne.