Humans can estimate the time and position of a moving object's arrival. However, numerous studies have demonstrated superior position estimation accuracy for descending objects compared with ascending objects. We tested whether the accuracy of position estimation for ascending and descending objects differs between the upper and lower visual fields. Using a head-mounted display, participants observed a target object ascending or descending toward a goal located at 8.7° or 17.1° above or below from the center of the monitor in the upper and lower visual fields, respectively. Participants pressed a key to match the time of the target's arrival at the goal, with the gaze kept centered. For goals (8.7°) close to the center, ascending and descending objects were equally accurate, whereas for goals (17.1°) far from the center, the ascending target's position estimation in the upper visual field was inferior to the others. Targets moved away from the center for goals further from the center and closer to the center for goals nearer to the center. As the positional accuracy of ascending and descending objects was not assessed for each of the four goals, it remains unclear which was more important for impaired accuracy: the proximity of the target position or direction of the upward or downward motion. However, taken together with previous studies, we suggest that estimating the position of objects moving further away from the central fovea of the upper visual field may have contributed to the asymmetry in position estimation for ascending and descending objects.