Abstract: In recent years, the development of ground robots with human-like perception capabilities has led to the use of multiple sensors, including cameras, lidars, and radars, along with deep learning techniques for detecting and recognizing objects and estimating distances. This paper proposes a computer vision-based navigation system that integrates object detection, segmentation, and monocular depth estimation using deep neural networks to identify predefined target objects and navigate towards them with a single monocular camera as a sensor. Our experiments include different sensitivity analyses to evaluate the impact of monocular cues on distance estimation. We show that this system can provide a ground robot with the perception capabilities needed for autonomous navigation in unknown indoor environments without the need for prior mapping or external positioning systems. This technique provides an efficient and cost-effective means of navigation, overcoming the limitations of other navigation techniques such as GPS-based and SLAM-based navigation. Graphical Abstract: [Figure not available: see fulltext.]
The European Union (EU) transport policy recognizes the importance of the waterborne transport systems as key elements for sustainable growth in Europe. By 2030, 30% of total road freight over 300 km should shift to rail or waterborne transport, and more than 50% by 2050. Thus far, this ambition has failed but there have been several project initiatives within the EU to address these issues. In one of these projects, we consider a new waterborne transport system for Europe that is green, robust, flexible, more automated and autonomous, and able to connect both rural and urban terminals. The purpose of this paper is to describe work and preliminary results from this project. To that effect, and in order to assess any solutions contemplated, a comprehensive set of Key Performance Indicators (KPIs) has been defined, and three specific use cases within Europe are examined and evaluated according to these KPIs. KPIs represent the criteria under which the set of solutions developed are evaluated, and also compared to non-autonomous solutions. They are grouped under economic, environmental and social KPIs. KPIs have been selected after a consultation process involving project partners and external Advisory Group members. Links to EU transport and other regulatory action are also discussed.
The European Union (EU) transport policy recognizes the importance of the waterborne transport systems as key elements for sustainable growth in Europe. By 2030, 30% of total road freight over 300 km should shift to rail or waterborne transport, and more than 50% by 2050. Thus far, this ambition has failed but there have been several project initiatives within the EU to address these issues. In one of these projects, we consider a new waterborne transport system for Europe that is green, robust, flexible, more automated and autonomous, and able to connect both rural and urban terminals. The purpose of this paper is to describe work and preliminary results from this project. To that effect, and in order to assess any solutions contemplated, a comprehensive set of Key Performance Indicators (KPIs) has been defined, and three specific use cases within Europe are examined and evaluated according to these KPIs. KPIs represent the criteria under which the set of solutions developed are evaluated, and also compared to non-autonomous solutions. They are grouped under economic, environmental and social KPIs. KPIs have been selected after a consultation process involving project partners and external Advisory Group members. Links to EU transport and other regulatory action are also discussed.
Marine autonomy research has focused on algorithmic and technical developments, targeting autonomous craft in restricted areas where international rules and regulations are not prioritised. This paper addresses the system engineering aspect of a highly complex system in which the seamless, predictable, and secure interoperability of vendorspecific hardware and software subsystems is a fundamental requirement for designing and implementing cyber-physical systems with artificial intelligence to assist or replace the navigating officer, such as autonomous marine surface vehicles. It addresses international rules in the sector and exhibits a system architecture that can fulfil the criteria for safe behaviour in foreseen occurrences and the capacity to request human aid if the autonomous system cannot manage a problem. The system thinking and engineering provided in this article have been applied to The GreenHopper, a harbour bus currently under construction and intended to undergo certification and enter commercial service.
Highly reliable situation awareness is a main driver to enhance safety via autonomous technology in the marine industry. Groundings, ship collisions and collisions with bridges illustrate the need for enhanced safety. Authority for a computer to suggest actions or to take command, would be able to avoid some accidents where human misjudgement was a core reason. Autonomous situation awareness need be conducted with extreme confidence to let a computer algorithm take command. The anticipation of how a situation can develop is by far the most difficult step in situation awareness, and anticipation is the subject of this article. The IMO International Regulations for Preventing Collisions
at Sea (COLREGS), describe the regulatory behaviours of marine vessels relative to each other, and correct interpretation of situations is instrumental to safe navigation. Based on a breakdown of COLREGS rules, this article presents a framework to represent manoeuvring behaviours that are expected when all vessels obey the rules. The article shows how nested finite automata can segregate situation assessment from decision making and provide a testable and repeatable algorithm. The suggested method makes it possible to anticipate own ship and other vessels’ manoeuvring in a multi-vessel scenario. The framework is validated using scenarios from a full-mission simulator.
Offshore pipelines and structures require regular marine growth removal and inspection to ensure structural integrity. These operations are typically carried out by Remotely Operated Vehicles (ROVs) and demand reliable and accurate feedback signals for operating the ROVs efficiently under harsh offshore conditions. This study investigates and quantifies how sensor delays impact the expected control performance without the need for defining the control parameters. Input-output (IO) controllability analysis of the open-loop system is applied to find the lower bound of the H-infinity peaks of the unspecified optimal closed-loop systems. The performance analyses have shown that near-structure operations, such as pipeline inspection or cleaning, in which small error tolerances are required, have a small threshold for the time delays. The IO controllability analysis indicates that off-structure navigation allow substantial larger time delays. Especially heading is vulnerable to time delay; however, fast-responding sensors usually measure this motion. Lastly, a sensor comparison is presented where available sensors are evaluated for each ROV motion’s respective sensor-induced time delays. It is concluded that even though off-structure navigation have larger time delay tolerance the corresponding sensors also introduce substantially larger time delays.
Autonomous marine surface vehicles rely on computer systems with computer intelligence making decisions to assist or replace the navigating officer. A fundamental requirement for the design and implementation of such a cyber-physical system is seamless, predictable, and secure interoperability between vendor-specific hardware and software subsystems. The article describes a system design that includes mechanisms to mitigate the risks and consequences of software defects, individual component malfunction, and harmful cyber interference. It addresses international regulations in the field and demonstrates a system design that can meet the requirements for safe behaviour in foreseeable events while also having the ability to call for human assistance if the autonomous system is unable to handle a situation. The paper presents a design for highly automated vessels with several inherent risk-reducing features, including the ability to isolate and encapsulate abnormal behaviours, built-in features to support resilience to unexpected events, and mechanisms for internal defence against cyber-attacks. The article shows how this is provided by a novel middleware that supports risk mitigation, dependability, and resilience.
The European maritime transport policy recognizes the importance of the waterborne transport systems as key elements for sustainable growth in Europe. A major goal is to transfer more than 50% of road transport to rail or waterways within 2050. To meet this challenge waterway transport needs to get more attractive and overcome its disadvantages. Therefore, it is necessary to develop new knowledge and technology and find a completely new approach to short sea and inland waterways shipping. A key element in this is automation of ships, ports and administrative tasks aligned to requirements of different European regions. One main goal in the AEGIS project is to increase the efficiency of the waterways transport with the use of higher degrees of automation corresponding with new and smaller ship types to reduce costs and secure higher frequency by feeders and provide multimodal green logistics solutions combining short sea shipping with rail and road transport.
Highly reliable situation awareness is a main driver to enhance safety via autonomous technology in the marine industry. Groundings, ship collisions and collisions with bridges illustrate the need for enhanced safety. Authority for a computer to suggest actions or to take command, would be able to avoid some accidents where human misjudgement was a core reason. Autonomous situation awareness need be conducted with extreme confidence to let a computer algorithm take command. The anticipation of how a situation can develop is by far the most difficult step in situation awareness, and anticipation is the subject of this article. The IMO International Regulations for Preventing Collisions
at Sea (COLREGS), describe the regulatory behaviours of marine vessels relative to each other, and correct interpretation of situations is instrumental to safe navigation. Based on a breakdown of COLREGS rules, this article presents a framework to represent manoeuvring behaviours that are expected when all vessels obey the rules. The article shows how nested finite automata can segregate situation assessment from decision making and provide a testable and repeatable algorithm. The suggested method makes it possible to anticipate own ship and other vessels’ manoeuvring in a multi-vessel scenario. The framework is validated using scenarios from a full-mission simulator.
In near coast navigation, buoys and beacons convey essential information about dangers. At night-time, selected buoys send out individual blink-sequences that are marked in sea charts. International regulations require that navigation officer on watch makes visual confirmation of objects and their type in order to navigate safely. With rapid developments of highly automated vessels, this duty needs be carried out by algorithms that detect and locate objects without human intervention. At night-time, this requires algorithms that decode blink sequences and are able to classify from this information. The paper investigates this problem and suggests an algorithm that solves the problem. Convolutional Neural Networks (CNN) with Gated Recurrent Units (GRU) are developed for classification. A dedicated architecture is suggested that includes both temporal and color decoding to obtain unique precision. We demonstrate how networks are trained on synthetically generated data, and the paper shows, on real-world data, how the suggested approach yields 100.0% accurate results on 44 real-world recordings while being robust to inaccuracy in actual blink sequences. Comparison with baseline signal processing and with a recent state-of-the-art 3D CNN model shows that the new blink-sequence classifier outperforms alternative algorithms. A showcase of the results of this work is available in this video: https://youtu.be/KEi8qNnKV2w.