Objective Assessment of Loose Gravel Condition using Machine Learning with Audio-visual Observation

Abstract: A well-maintained road network is essential for sustainable economic development, providing vital transportation routes for goods and services while connecting communities. Sweden's public road network includes a significant portion of gravel roads, particularly cost-effective for less populated areas with lower traffic volumes. However, gravel roads deteriorate quickly, leading to accidents, environmental pollution, and vehicle tire wear when not adequately maintained. The Swedish Road Administration Authority (Trafikverket) assesses gravel road conditions using subjective methods, analysing images taken during snow-free periods. Due to cost constraints, this labour-intensive process is prone to errors and lacks advanced techniques like road profilometers.This thesis explores the field of assessing gravel road conditions. It commences with a comprehensive review of manual gravel road assessment methods employed globally and existing data-driven smart methods. Subsequently, it harnesses machine hearing and machine vision techniques, primarily focusing on enhancing road condition classification by integrating sound and image data.The research examines sound data collected from gravel roads, exploring machine learning algorithms for loose gravel conditions classification with potential road maintenance and monitoring implications. Another crucial aspect involves applying machine vision to categorise image data from gravel roads. The study introduces an innovative approach using publicly available resources like Google Street View for image data collection, demonstrating machine vision's adaptability in assessing road conditions.The research also compares machine learning methods with manual human classification, specifically regarding sound data. Automated approaches consistently outperform manual methods, providing more reliable results. Furthermore, the thesis investigates combining audio and image data to classify road conditions, particularly loose gravel scenarios. Early feature fusion using pre-trained models significantly improves classifier accuracy.The research proposes using cost-effective devices like mobile phones with AI applications attached to car windshields to collect audio and visual data on gravel road conditions. This approach can provide more accurate and efficient data collection, resulting in real-time mapping of road conditions over considerable distances. Such information can benefit drivers, travellers, and road maintenance agencies by identifying problematic areas with loose gravel, enabling targeted and efficient maintenance efforts, and minimising disruptions to traffic flow during maintenance operations.