I go back and forth with looking at the person and just general surroundings. I think as long as you’re trying to keep conversation going people (who are trying to be friends) aren’t going to be too upset if you look away, it’s pretty natural. Looking past someone helps me too. It’s enough for me to reset the ‘too much eye contact’ timer in my brain and enough for them to feel like you’re still engaged