Anytime I did that shyt I had my pistol on me, especially during the winter time where I can have it ready inside my hoodie or coat pocket, I can already have it aimed in that scenario.
There's small spots of rural area in Detroit suburbs that last a few blocks, I was looking for a specific location one time and a white dude stepped on his porch asking me questions, he didn't even know I had my pistol already aimed at him thru my coat pocket. He was cool as hell though and he showed me the proper location.
In the scenario in the OP, I would have killed that man as soon as I seen him walk out with a rifle. I can properly up and aim my pistol much quicker than he can position his rifle