Automatically delivering objects to in-car passengers has many potential applications. Such a system generally consists of two sub-systems: a perception system and an action system. The perception system basically looks for the targets’ positions and the action system delivers objects to the targets. In this paper, we propose a novel perception system, which contains two major functions: estimation of reaching points and discovering potential risks. The reaching points are the locations where robot arms needs to reach. Moreover, it should be able to reach with comfort by passengers and keep a safe distance from the car body. In order to achieve this, all the vehicle components (side surfaces, side mirrors etc.), which may cause collision, need to be detected. Potential risks are usually caused by moving objects or changing door state (close to open) during the operation. It is necessary to monitor these two situations to avoid any potential risks during operation. Our offline test shows that the accuracy of reaching points estimation can reach up to 94% and the response time for moving objects detection or door state changes is less than 1 millisecond.