15, April 2025
Vision-Based Perception Systems for Autonomous — Advancements, Challenges, and the Road Ahead
Author(s): Keval Bopaliya, Chintan D, Niraj Bhagchandani
Authors Affiliations:
1Undergraduate, B. Tech Information Technology, Atmiya University, Rajkot, India
2Undergraduate, B. Tech Information Technology, Atmiya University, Rajkot, India
3Assistant Professor, B. Tech Information Technology, Atmiya University, Rajkot, India
DOIs:10.2015/IJIRMF/202504020 | Paper ID: IJIRMF202504020 Abstract
Keywords
Cite this Article/Paper as
References
Computer vision is at the core of enabling driverless vehicles, as it enables machines to see and comprehend the world in the same way that human drivers do. Computer vision includes a wide range of abilities in detecting objects, detecting lane markings, detecting signs, and following pedestrians, all of which are critical to safety and efficiency in transport. This article discusses how computer vision is utilized in autonomous cars, its unseen techniques, latest advancements, and challenges that it still poses. Owing to advanced expert systems like artificial intelligence and deep learning, autonomous systems are becoming advanced, processing real-time data and nanosecond-level decisions. Nevertheless, the performance of such systems is primarily dependent on the quality of the training data, weather, and model architecture. Furthermore, safety needs and morality are making up non-technical barriers. The present study is going to present an in-depth account of how computer vision is changing the future of transport and what should be considered to shift from semi-autonomous to fully autonomous transportation systems. Computer vision systems will transform transport by minimising traffic accidents, streamlining traffic flow, and increasing the inclusivity of the mobility system through ongoing industry-academia research collaboration.
Computer Vision, Self-Driving Cars, Object Detection, Lane Detection, Neural Networks, Autonomous Vehicles, Deep Learning, Sensor Fusion
Keval Bopaliya, Chintan D, Niraj Bhagchandani(2025); Vision-Based Perception Systems for Autonomous Vehicles — Advancements, Challenges, and the Road Ahead, International Journal for Innovative Research in Multidisciplinary Field, ISSN(O): 2455-0620, Vol-11, Issue-4, Pp.145-150.     Available on –  https://www.ijirmf.com/
1. Anderson, J. M., Kalra, N., Stanley, K. D., Sorensen, P., Samaras, C., & Oluwatola, O. A. (2016). Autonomous vehicle technology: A guide for policymakers. RAND Corporation.
2. Litman, T. (2020). Autonomous vehicle implementation predictions: Implications for transport planning. Victoria Transport Policy Institute.
3. Fagnant, D. J., & Kockelman, K. (2015). Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice, 77, 167–181. https://doi.org/10.1016/j.tra.2015.04.003
4. Szeliski, R. (2022). Computer vision: Algorithms and applications. Springer.
5. Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848. https://doi.org/10.1109/TPAMI.2017.2699184
6. Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3354–3361). https://doi.org/10.1109/CVPR.2012.6248074
7. Cordts, M., Omran, M., Ramos, S., et al. (2016). The Cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3213–3223). https://doi.org/10.1109/CVPR.2016.350
8. Janai, J., Güney, F., Behl, A., & Geiger, A. (2020). Computer vision for autonomous vehicles: Problems, datasets and state-of-the-art. Foundations and Trends in Computer Graphics and Vision, 12(1–3), 1–308. https://doi.org/10.1561/0600000079
9. Bijelic, M., Gruber, T., Mannan, F., et al. (2020). Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11682–11692). https://doi.org/10.1109/CVPR42600.2020.01170
10. Badue, C., Guidolini, R., Carneiro, R. V., et al. (2021). Self-driving cars: A survey. Expert Systems with Applications, 165, 113816. https://doi.org/10.1016/j.eswa.2020.113816
11. Gonzalez, R. C., & Woods, R. E. (2018). Digital image processing (4th ed.). Pearson.
12. Duda, R. O., & Hart, P. E. (1972). Use of the Hough transformation to detect lines and curves in pictures. Communications of the ACM, 15(1), 11–15. https://doi.org/10.1145/361237.361242
13. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
14. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 779–788). https://doi.org/10.1109/CVPR.2016.91
15. Liu, W., Anguelov, D., Erhan, D., et al. (2016). SSD: Single shot multibox detector. In European Conference on Computer Vision (ECCV) (pp. 21–37). https://doi.org/10.1007/978-3-319-46448-0_2
16. Sun, P., Kretzschmar, H., Dotiwalla, X., et al. (2020). Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2446–2454). https://doi.org/10.1109/CVPR42600.2020.00252
17. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI (pp. 234–241). https://doi.org/10.1007/978-3-319-24574-4_28
18. Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In European Conference on Computer Vision (ECCV) (pp. 801 818). https://doi.org/10.1007/978-3-030-01234-2_49
19. Caltagirone, L., Scheidegger, S., Svensson, L., & Wahde, M. (2019). Fast LIDAR-based road detection using fully convolutional neural networks. In 2019 IEEE Intelligent Vehicles Symposium (IV) (pp. 66–71). https://doi.org/10.1109/IVS.2019.8814014
20. Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018). Joint 3D proposal generation and object detection from view aggregation. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1–8). https://doi.org/10.1109/IROS.2018.8594445
21. Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3354–3361). https://doi.org/10.1109/CVPR.2012.6248074
22. Caesar, H., Bankiti, V., Lang, A. H., et al. (2020). nuScenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11621–11631). https://doi.org/10.1109/CVPR42600.2020.01164
23. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(60). https://doi.org/10.1186/s40537-019-0197-0
24. Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934. https://arxiv.org/abs/2004.10934
25. Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016). ENet: A deep neural network architecture for real time semantic segmentation. arXiv preprint arXiv:1606.02147. https://arxiv.org/abs/1606.02147
26. Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767. https://arxiv.org/abs/1804.02767
27. Chen, L. C., et al. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. ECCV. https://doi.org/10.1007/978-3-030-01234-2_49
28. NVIDIA Corporation. (2020). NVIDIA Jetson AGX Xavier: Developer kit. https://developer.nvidia.com/embedded/jetson-agx-xavier-developer-kit
29. Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018). Joint 3D proposal generation and object detection from view aggregation. IROS. https://doi.org/10.1109/IROS.2018.8594445
30. Feng, D., Haase-Schuetz, C., Rosenbaum, L., Hertlein, H., Glaeser, C., Timm, F., … & Dietmayer, K. (2020). Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2020.2993626
31. Sivaraman, S., & Trivedi, M. M. (2013). Looking at vehicles on the road: A survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Transactions on Intelligent Transportation Systems, 14(4), 1773–1795. https://doi.org/10.1109/TITS.2013.2266661
32. Janai, J., Güney, F., Behl, A., & Geiger, A. (2020). Computer vision for autonomous vehicles: Problems, datasets and state of the art. Foundations and Trends® in Computer Graphics and Vision. https://doi.org/10.1561/0600000079 33. Samek, W., Wiegand, T., & Müller, K. R. (2017). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296. https://arxiv.org/abs/1708.08296
34. Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052
35. Sun, J., & Wang, B. (2021). Privacy protection in autonomous vehicles: Challenges and solutions. ACM Computing Surveys (CSUR). https://doi.org/10.1145/3446384
36. Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015). DeepDriving: Learning affordance for direct perception in autonomous driving. In IEEE ICCV. https://doi.org/10.1109/ICCV.2015.416
36. Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015). DeepDriving: Learning affordance for direct perception in autonomous driving. In IEEE ICCV. https://doi.org/10.1109/ICCV.2015.416
37. Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017). Scene parsing through ADE20K dataset. In CVPR. https://doi.org/10.1109/CVPR.2017.660
38. Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. https://arxiv.org/abs/1702.08608
39. Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., … & Beijbom, O. (2020). nuScenes: A multimodal dataset for autonomous driving. CVPR. https://doi.org/10.1109/CVPR42600.2020.01164
40. McBride, N. (2021). The ethics of driverless cars. Communications of the ACM, 64(11), 20–22. https://doi.org/10.1145/3488663