Eye movements and other gestures have different impact in different cultures. Constant eye contact in some parts of the world is understood as uncomfortable indication while in some regions it is taken as good indication of high confidence.
Though it feels uncomfortable when person doesn't make sensible eye contact while talking. Repeated movement of eye from side to side while having conversation suggest that a person is thinking about something which is not the part of conversation. That person is not interested in a talk with you or in the case of confrontation, he is avoiding eye contact.
Other reasons :
May be the person having eye defect.
It can be someone's habit.
May be their eyeballs rest that way and were trying to bring in centre so that they could mimic you.
May be counting people passing by seems to be more interesting than your conversation.
May be the person is uncomfortable.
Read More at : /answers.mindstick.com
