Current status of visual lookouts by seafarers

To avoid dangerous objects and collisions with other vessels while sailing, the navigator uses binoculars to visually recognize dangerous objects. Nautical instruments such as radar are also used, and decisions are then made to change course if necessary. The new system will be installed on a trial basis to verify whether the safety of the ship's operation can be improved by automating the task of recognizing dangerous objects. This system apparently can automatically recognize dangerous targets and other vessels that may be overlooked by the human eye, especially at night and in congested waters.

Research on automatic identification by image analysis has been underway, and automatic recognition is possible if the image can be acquired. However, ships are exposed to wind and rain. In such an environment, there is no camera that has been able to shoot day and night, and no system that could measure the distance from the captured image to the target with a certain degree of accuracy.

The new system uses a camera unit that can shoot day and night to automatically recognize ships and targets and measure the distance to them. Information obtained from navigational equipment, including vessel names, distance, and time when the ship is closest to the target, can be superimposed and displayed in an integrated manner to a tablet or touch-panel monitor display.

In addition, the system is epoch-making because it can independently recognize small fishing boats and small markers that are not captured by radar and not equipped with AIS. The system measures the distance to these targets and notifies the person on duty of danger of collision.

Surrounding images taken are analyzed using artificial intelligence (AI) on Orca AI's server, which makes use of machine learning and then remotely updates the onboard software. This mechanism improves performance such as recognition rate through continuing use. In addition to the captured video, navigation instrument information is sent to Orca AI's server and displayed together with the video data. This makes it possible to monitor the movement of the ship and check the situation from the land office.

Details of verification

In this verification, the system has been installed on a trial basis on a ship operated by the NYK Group to verify detection capability and contribution to lookout work, improve the target detection algorithm through data collection and machine learning, and enhance the recognition rate.

In addition, in joint research conducted by NYK and MTI in advance with Orca AI, the system was installed on domestic vessels to collect information on the Japan coast, in addition to fishing boats, fishing gear, and buoys peculiar to Japan. We are also researching how to use the system for future autonomous operations by improving its recognition rate.